HttpServletRequest stream data cannot be read repeatedly

preface

Some services may require multiple reads of HTTP request parameters, such as pre-api signature verification. At this point we might implement this logic in interceptors or filters, but if we try, we’ll find that if we read arguments from getInputStream() in the interceptor, we won’t be able to read them again in the Controller, and will throw the following exceptions:

HttpMessageNotReadableException: Required request body is missing
IllegalStateException: getInputStream() can't be called after getReader()
Copy the code

This time we need to cache the requested data. This article details how to solve this problem, starting with ServletRequest data encapsulation principles. If you do not want to see the principle, you can directly read the best solution.

ServletRequest data encapsulation principles

Normally when we accept HTTP request parameters, it is basically wrapped in SpringMVC.

To POST form-data parameters, you can use the entity class directly, or you can use the Controller method directly to fill in the parameters. Manually, you can passrequest.getParameter()In order to get.
When you POST JSON, it is added to the entity class@RequestBodyArgument or direct callrequest.getInputStream()Get stream data.

We can see that different methods are called when retrieving data in different data formats, but reading the source code, we can see that the underlying data source is the same, but SpringMVC helps us to do some processing. Let’s take a look at how ServletRequest data encapsulation works.

In fact, the parameters we transfer over HTTP are stored in the InputStream of the Request object, which is the final implementation of the ServletRequest, provided by Tomcat. The data in InputStream is then encapsulated at different times for different data formats.

Spring MVC’s encapsulation of different types of data

The data for a GET request is typically a Query String, directly after the URL, and requires no special processing
For example, POST or PUTmultipart/form-dataFormatted data

// Remove extraneous code from the source appropriately
// SpringMVC processes this data in the doDispatch() method of DispatchServlet. The specific processing process is as follows:
// org.springframework.web.servlet.DispatcherServlet.java
protected void doDispatch(HttpServletRequest request, HttpServletResponse response) throws Exception {
    HttpServletRequest processedRequest = request;
    HandlerExecutionChain mappedHandler = null;
    boolean multipartRequestParsed = false; WebAsyncManager asyncManager = WebAsyncUtils.getAsyncManager(request); processedRequest = checkMultipart(request); multipartRequestParsed = (processedRequest ! = request);// Determine handler for the current request.
    // other code...
}
// 1. Call checkMultipart(request) to check whether the requested data type is multipart/form-data
protected HttpServletRequest checkMultipart(HttpServletRequest request) throws MultipartException {
    if (this.multipartResolver ! =null && this.multipartResolver.isMultipart(request)) {
		return this.multipartResolver.resolveMultipart(request);
    }
    return request;
}
/ / 2. If it is, call the multipartResolver resolveMultipart (request), which returns a StandardMultipartHttpServletRequest object.
// org.springframework.web.multipart.support.StandardMultipartHttpServletRequest.java
public StandardMultipartHttpServletRequest(HttpServletRequest request) throws MultipartException {
    this(request, false);
}
public StandardMultipartHttpServletRequest(HttpServletRequest request, boolean lazyParsing) throws MultipartException {
    super(request);
    if (!lazyParsing) {
        parseRequest(request);
    }
}
/ / 3. In constructing StandardMultipartHttpServletRequest object invokes the parseRequest (request), which will further encapsulation is the data flow in the InputStream.
// Form-data is a form-data package containing fields and files.
Copy the code

For example, POST or PUTapplication/x-www-form-urlencodedFormatted data

// Non-form-data data will be stored in the InputStream of HttpServletRequest.
// The first time getParameterNames() or getParameter() is called,
// The parseParameters() method is called to wrap the parameters, read the data from InputStream, and wrap it into a Map.

//org.apache.catalina.connector.Request.java
public String getParameter(String name) {
    if (!this.parametersParsed) {
        this.parseParameters();
    }
    return this.coyoteRequest.getParameters().getParameter(name);
}
Copy the code

For example, POST or PUT application/jsonFormatted data

// The data will be stored directly in the InputStream of HttpServletRequest via request.getinputStream () or getReader().
Copy the code

Problems with reading parameters

We now have a pretty good idea of how SpringMVC encapsulates HTTP request parameters. According to the previous description, if we try to read parameters repeatedly in the interceptor and Controller, we will get the following exception:

HttpMessageNotReadableException: Required request body is missing
IllegalStateException: getInputStream() can't be called after getReader()
Copy the code

This is due to the particularity of InputStream data. When reading InputStream data in Java, it reads byte data one by one through the movement of a pointer. After reading it once, the pointer will not reset, so there will be problems in the second reading. As mentioned earlier, the HTTP Request parameters are also wrapped in the Request object InputStream, so the second call to getInputStream() will throw the above exception. The specific problem can be broken down into a number of cases:

Request mode:multipart/form-data, manually called in the interceptorrequest.getInputStream()

// As mentioned above, it will be processed at doDispatch(), so there will be no value here
log.info("input stream content: {}".new String(StreamUtils.copyToByteArray(request.getInputStream())));
Copy the code

Request mode:application/x-www-form-urlencoded, manually called in the interceptorrequest.getInputStream()

// The first time can get the value
log.info("input stream content: {}".new String(StreamUtils.copyToByteArray(request.getInputStream())));
// The first call to getParameter() calls parseParameters(), which calls getInputStream()
// There is no value here
log.info("form-data param: {}", request.getParameter("a"));
log.info("form-data param: {}", request.getParameter("b"));
Copy the code

Request mode:application/json, manually called in the interceptorrequest.getInputStream()

// The first time can get the value
log.info("input stream content: {}".new String(StreamUtils.copyToByteArray(request.getInputStream())));
// Call getInputStream() anywhere else after that and get no value, raising an exception
Copy the code

To retrieve HTTP request parameters multiple times, we need to cache the data in the InputStream stream.

Best solution

Through access to information, oneself actually springframework corresponding wrapper to solve this problem, in the org. Springframework. Web. The util package under a ContentCachingRequestWrapper class. This class caches InputStream into ByteArrayOutputStream, which is repeatable by calling getContentAsByteArray().

/**
 * {@link javax.servlet.http.HttpServletRequest} wrapper that caches all content read from
 * the {@linkplain #getInputStream() input stream} and {@linkplain #getReader() reader},
 * and allows this content to be retrieved via a {@link #getContentAsByteArray() byte array}.

 * @see ContentCachingResponseWrapper
 */
Copy the code

In use, only need to add a Filter, to pack it into ContentCachingResponseWrapper returned to the interceptor and the Controller is ok.

@Slf4j
@WebFilter(urlPatterns = "/*")
public class CachingContentFilter implements Filter {
    private static final String FORM_CONTENT_TYPE = "multipart/form-data";

    @Override
    public void init(FilterConfig filterConfig) {}@Override
    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
        String contentType = request.getContentType();
        if (request instanceof HttpServletRequest) {
            HttpServletRequest requestWrapper = new ContentCachingRequestWrapper((HttpServletRequest) request);
            / / # 1
            if(contentType ! =null && contentType.contains(FORM_CONTENT_TYPE)) {
                chain.doFilter(request, response);
            } else {
                chain.doFilter(requestWrapper, response);
            }
            return;
        }
        chain.doFilter(request, response);
    }

    @Override
    public void destroy(a) {}}// Add the scan filter annotation
@ServletComponentScan
@SpringBootApplication
public class SeedApplication {
    public static void main(String[] args) { SpringApplication.run(SeedApplication.class, args); }}Copy the code

In the interceptor, get the request parameters:

// Stream data fetch, such as JSON
/ / # 2
String jsonBody = IOUtils.toString(wrapper.getContentAsByteArray(), "utf-8");
// form-data and urlencoded data
String paramA = request.getParameter("paramA");
Map<String,String[]> params = request.getParameterMap();
Copy the code

Tips:

So we need to make a distinction here based on contentType, encountermultipart/form-dataData is wrapped into a Map using a MultipartResolver without a wrapper, which can also be used flexibly in the interceptor.
Wrapper can be used for specific usegetContentAsByteArray()To get the data and pass throughIOUtilsConvert to String. Use as little as possiblerequest.getInputStream(). Because InputStream can only be read once even though it’s wrapped, and the HttpMessageConverter argument conversion needs to be called before the argument goes to the Controller’s method, it’s okay to leave it there.

conclusion

Encounter this problem also consulted many blogs, some use ContentCachingRequestWrapper, also some implements a Wrapper. The Wrapper constructor reads the stream data directly into byte[] data, which can cause problems with multipart/form-data. Because the wrapper is executed before the MultipartResolver is called, there is no data to read when it is called again.

So the blogger took a look at the Spring source code and implemented this solution, which can basically handle a variety of common data types.

reference

How do I reuse inputStream?
Read stream twice
[Fixed] Request.getinputStream () can only be read once
Http Servlet request lose params from POST body after read it once
Spring Boot series two: One diagram to understand the request processing process

The appendix code

package com.example.seed.common.config;

import lombok.extern.slf4j.Slf4j;
import org.springframework.web.util.ContentCachingRequestWrapper;

import javax.servlet.*;
import javax.servlet.annotation.WebFilter;
import javax.servlet.http.HttpServletRequest;
import java.io.IOException;

/ * * *@author Fururur
 * @date2020/5/6 - all * /
@Slf4j
@WebFilter(urlPatterns = "/*")
public class CachingContentFilter implements Filter {
    private static final String FORM_CONTENT_TYPE = "multipart/form-data";

    @Override
    public void init(FilterConfig filterConfig) {}@Override
    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
        String contentType = request.getContentType();
        if (request instanceof HttpServletRequest) {
            HttpServletRequest requestWrapper = new ContentCachingRequestWrapper((HttpServletRequest) request);
            if(contentType ! =null && contentType.contains(FORM_CONTENT_TYPE)) {
                chain.doFilter(request, response);
            } else {
                chain.doFilter(requestWrapper, response);
            }
            return;
        }
        chain.doFilter(request, response);
    }

    @Override
    public void destroy(a) {}}Copy the code

package com.example.seed;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.web.servlet.ServletComponentScan;

@ServletComponentScan
@SpringBootApplication
public class SeedApplication {
    public static void main(String[] args) { SpringApplication.run(SeedApplication.class, args); }}Copy the code

@RequestMapping("/query")
public void query(HttpServletRequest request) {
    ContentCachingRequestWrapper wrapper = new ContentCachingRequestWrapper(request);
    log.info("{}".new String(wrapper.getContentAsByteArray()));
}
Copy the code

HttpServletRequest stream data cannot be read repeatedly

preface

ServletRequest data encapsulation principles

Spring MVC’s encapsulation of different types of data

Problems with reading parameters

Best solution

conclusion

reference

The appendix code

Related Posts

Simple and easy to use efficient tools | Go theme month

0005: Algorithm — Merge sort

Personal Hematemesis series – Summary Redis