A Bug from HTTPS to HTTP and they put up with it for 2 years? I’m sorry I can’t accept it. Overtime has been changed!

Today this article tells you a story and process of tracking down bugs. Personally, I always think that there is a demon when things go wrong, and so is the Bug in the program.

We hope that through this Bug investigation story, we can not only learn a series of knowledge points, but also learn how to solve the problem, how to do things more professionally. The way and thinking of problem solving are more important than simple technology.

Let ‘s go!

The origin of the story

Shortly after I took over the new project of the new team, when releasing a system, my colleague kindly reminded me that when releasing THE XX system, I should comment out a line of code in the test environment, and then release the comment when launching the system.

Listen to this friendly remind, a surprised: this is what black technology? ! In my experience, there is no system that needs to be handled this way, so I decided to troubleshoot this problem.

Finally spare time, Friday toss for more than half a day, did not solve the problem, the weekend is still thinking about, so work overtime to solve this problem.

Existence and operation of bugs

The project is jSP-based with no back-end separation. A public head.jsp is introduced in the JSP page, which contains this line of code and comments:

<! <meta http-equiv=" content-security-policy "Content ="upgrade-insecure-requests" />Copy the code

A friendly reminder from a colleague is that the operation is commented out in the test environment (otherwise it cannot be accessed), and the production environment needs to be released or it cannot be accessed (go round and round). This is probably used to solve HTTPS related problems.

So, what is the reason for doing this? Is there an easier way to do it? People are just doing it. No one is looking for the root of the problem. No one has the answer.

HTTP request in HTTPS

Let’s take a look at what is used to configure META elements.

The http-equiv content-security-policy (CSP) is used to protect against XSS attacks.

The usual way to use this is to define it in HTML with meta tags:

<meta http-equiv="content-security-policy" content=" policy" > <meta http-equiv="content-security-policy-report-only" The content = "strategy" >Copy the code

In content, you can specify various restriction policies related to security.

Upgrade-insecure requests is one of the limiting policies used in the project, which automatically replaced all HTTP links on the web page that loaded external resources with HTTPS.

This line of code was originally written to cast an HTTP request into an HTTPS request.

However, this problem does not occur as long as you configure HTTP to HTTPS in Nginx or SLB, and the system has the corresponding configuration.

So, another online service experiment, comment out this section of code, part of the function is really in the circle, sincere not deceiving me!

Why are HTTP requests not allowed in HTTPS

Checking the request in the browser, we found that the loop was caused by the following error:

Mixed Content: The page at 'https://example.com' was loaded over HTTPS, but requested an insecure stylesheet 'http://example.com/xxx'. This request has been blocked; the content must be served over HTTPS.
Copy the code

Mixed Content refers to Mixed Content. So-called mixed content usually occurs when the original HTML content is loaded over HTTPS, but other resources (for example, CSS styles, JS, images, etc.) are loaded over insecure HTTP requests. In this case, the same page uses BOTH HTTP and HTTPS content, and the HTTP protocol reduces the security of the entire page.

Therefore, modern browsers warn against HTTP requests in HTTPS, block the request, and throw the above exception information.

Now, the cause of the problem is pretty clear: AN HTTP request appears in an HTTPS request.

Well, there are several solutions:

• Scheme 1: Add meta tags to the HTML to forcibly convert HTTP requests into HTTPS requests. This is the same method as above, but the disadvantage of this method is obvious. In a test environment without HTTPS, you need to comment it out manually. Otherwise, it cannot be accessed normally.
• Scheme 2: Convert HTTP requests to HTTPS requests through Nginx or SLB configuration.
• Solution 3: The stupidest way is to find HTTP request problems in the project and fix them one by one.

Preliminary transformation, slightly effective

The first scheme used at present is obviously not up to the requirements, and the second scheme has been configured, but part of the page still does not work. So, what’s the alternative?

After a lot of investigation, it was found that the project did not work because redirect was widely used in the project.

@RequestMapping(value = "delete") public String delete(RedirectAttributes redirectAttributes) { //.. Do something addMessage(redirectAttributes, "Delete XXX succeeded "); return "redirect:" + Global.getAdminPath() + "/list"; }Copy the code

Redirect packets are redirected to HTTP in HTTPS environments, causing unreachable packets.

No wonder the HTTP to HTTPS Settings above are complete, and some pages are not working.

The root cause of this problem is the HTTP 1.0 protocol compatibility of Spring’s ViewResolver.

In view of this problem, it can be solved by closing it. There are two specific transformation schemes.

Redirect: RedirectView: RedirectView

modelAndView.setView(new RedirectView(Global.getAdminPath() + "/list", true, false));
Copy the code

If the RedirectView parameter is set to false, the http10Compatible switch is turned off.

Solution 2: Configure the redirectHttp10Compatible property of The Spring ViewResolver. With this scheme, global shutdown can be achieved.

<bean id="viewResolver" class="org.springframework.web.servlet.view.InternalResourceViewResolver"> <property name="viewClass" value="org.springframework.web.servlet.view.JstlView" /> <property name="prefix" value="/" /> <property  name="suffix" value=".jsp" /> <property name="redirectHttp10Compatible" value="false" /> </bean>Copy the code

Because of the heavy use of redirect in projects, the second option was adopted. After modification, we found that most of the problems were solved.

In order to prevent omissions, a few more pages, and even a fish!

Shiro interceptors are at work again

After solving the redirection problem and thinking everything was all right, the page involving Shiro redirects had a similar problem. The reason is simple: Some pages require Shiro to authenticate their permissions, but Shiro blocks HTTPS requests and redirects them to HTTP requests.

So why doesn’t setting redirectHttp10Compatible to false in the view layer work?

Tracing the code in Shiro’s interceptor, Shiro sets redirectHttp10Compatible to true by default in the interceptor

View the source code can be found that Shiro login filter FormAuthenticationFilter method called saveRequestAndRedirectToLogin method:

protected void saveRequestAndRedirectToLogin(ServletRequest request, ServletResponse response) throws IOException { saveRequest(request); redirectToLogin(request, response); } // Then call redirectToLogin protected void redirectToLogin(ServletRequest Request, ServletResponse response) throws IOException { String loginUrl = getLoginUrl(); WebUtils.issueRedirect(request, response, loginUrl); IssueRedirect public static void issueRedirect(ServletRequest Request, ServletResponse Response, ServletResponse response, ServletResponse response) String url) throws IOException { issueRedirect(request, response, url, (Map)null, true, true); } public static void issueRedirect(ServletRequest Request, ServletResponse Response, String url, Map queryParams, boolean contextRelative, boolean http10Compatible) throws IOException { RedirectView view = new RedirectView(url, contextRelative, http10Compatible); view.renderMergedOutputModel(queryParams, toHttp(request), toHttp(response)); }Copy the code

As you can see from the code trace above, issueRedirect is eventually called twice in the issueRedirect method of WebUtils, and the http10Compatible parameter defaults to true.

Get to the root of the problem and fix it easily. Rewrite the FormAuthenticationFilter interceptor:

public class CustomFormAuthenticationFilter extends FormAuthenticationFilter { @Override protected boolean onAccessDenied(ServletRequest request, ServletResponse response) throws Exception { if (isLoginRequest(request, response)) { if (isLoginSubmission(request, response)) { return executeLogin(request, response); } else { return true; } } else { saveRequestAndRedirectToLogin(request, response); return false; } } protected void saveRequestAndRedirectToLogin(ServletRequest request, ServletResponse response) throws IOException { saveRequest(request); redirectToLogin(request, response); } protected void redirectToLogin(ServletRequest request, ServletResponse response) throws IOException { String loginUrl = getLoginUrl(); WebUtils.issueRedirect(request, response, loginUrl, null, true, false); }}Copy the code

In this example, change the http10Compatible parameter in onAccessDenied that requires the webutils. issueRedirect method to be called to false.

The above is just an example, and in fact includes not only successful pages, but also failed pages, etc., which need to be re-implemented. Finally, configure a custom interceptor in the shiroFilter.

<! Login - custom filters - > < bean id = "customFilter" class = "com. Senzhuang. Shiro. CustomFormAuthenticationFilter" / > < bean id="shiroFilter" class="org.apache.shiro.spring.web.ShiroFilterFactoryBean"> <property name="securityManager" ref="securityManager" /> <property name="loginUrl" value="/login.html"></property> <property name="unauthorizedUrl" value="/refuse.html"></property> <property name="filters"> <map> <entry key="authc" value-ref="customFilter"/> </map> </property> </bean>Copy the code

After the above modification, the problem with HTTP requests in HTTPS has been solved.

In order to prevent omission, and one by one point some pages, and sent a problem! Ah, how so hand owe…

The pit of LayUI

I thought if we solved the above problem, we’d be done with it, and we’d have a barbecue to celebrate. As a result, a similar error was found in the front page. But in this case the error message comes from the path to the login page:

http://example.com/a/login
Copy the code

Why does the business operations page request the Login page after successfully logging in? And the redirect is an HTTP request, not an HTTPS request.

Check the login request result:

After checking the relevant business code, after the login is completed, there is no request for login request again ah, why will request login again? Could it be that access to some resources is limited and redirected to the login page?

Therefore, check the “Initiator” of the HTML call:

When LayUI requests the corresponding layer. CSS resource, the login operation is triggered.

The first thing that came to mind was that Shiro did not allow the interception of static resources, so Shiro allowed Layui to intercept, but the problem was already there.

After checking again, I found that the page did not actively introduce the layer. CSS file, so I actively introduced the layer. CSS file, but the problem still exists.

I had no choice but to look at layui.js to see why the request was made. At this point, you also notice the word “undefinedcss” in the request path.

Undefined is the uninitialized default value of a js variable, similar to null in Java.

Search for “CSS/” in layui.js and you’ll find this code:

return layui.link(o.dir + "css/" + e, t, n)
Copy the code

In contrast, the value of O.dir is “undefined”, which becomes “undefinedCSS” when connected to the following CSS. This path does not exist, and is not configured with Shiro permissions. By default, it goes to the login screen. This is an internal asynchronous redirect request, which does not appear on the page until you see the browser error message.

Layui link method parameters to modify:

// return layui.link(o.dir + "CSS /" + e, t, n) // Return layui.link((O.dir? o.dir:"/static/sc_layui/") +"css/"+e, t, n)Copy the code

The basic idea is: if there is a value of O.dir (js value is true), then use the value of O.dir; If O.dir is undefined, the specified default value is used.

“/static/sc_layui/” is the path where layui components are stored in the project. Since layui.js may be compressed JS, search “CSS/” or” layui.link “to find the corresponding code.

Restart the project, clear the browser cache, visit the page again, and the problem is completely resolved.

You can eat kebabs in peace

It took me half a day over the weekend to finally fix the problem once and for all, and now it’s safe to celebrate with a kebab.

Finally, review the process and see what you can learn from it:

• Problem: Code needs to be changed manually for different environments (HTTP and HTTPS);
• Looking for problems: For security, HTTP requests are not allowed in HTTPS;
• Fix the problem: Two ways to closehttp10Compatible;
• Shiro Problem: Shiro is disabled by defaulthttp10Compatible, rewrite Filter to realize the close operation;
• LayUI Bug fixes: LayUI code Bug that causes HTTP (login) requests to be initiated. Fix this Bug;

During this process, if you just settle for the status quo, “follow the rules,” and change the file every time you go live, it takes time and effort, and you don’t know why you’re doing it.

But if you follow through, as I did, you’ll learn a number of things:

• CSP, upgrade-insecure-requests configuration for HTTP requests
• Why cannot AN HTTP request be initiated in HTTPS?
• Configured in the Spring view parserhttp10Compatible;
• Disadvantages of redirect view returns;
• How to convert HTTP requests to HTTPS requests in Nginx;
• Mixed Content concepts and errors in HTTP requests;
• HTTP 1.0, HTTP 1.1, HTTP2.0 protocol differences;
• Shiro interceptor custom Filter;
• Shiro interceptor filters specified URL access;
• Shiro interceptor configuration and part of the source code implementation;
• [Fixed] A bug in LayUI
• Other techniques used or learned in troubleshooting the problem;

Did you learn these techniques? Have you learned the ideas and ways to solve problems? If something in this article inspires you, I’ll share it with you, and so will you.

About the blogger: Author of the technology book SpringBoot Inside Technology, loves to delve into technology and writes technical articles.

Public account: “program new vision”, the blogger’s public account, welcome to follow ~

Technical exchange: Please contact the weibo user at Zhuan2quan

\