Series of articles:

SpringCloud source series (1) – registry initialization for Eureka

SpringCloud source code series (2) – Registry Eureka service registration, renewal

SpringCloud source code series (3) – Registry Eureka crawl registry

SpringCloud source code series (4) – Registry Eureka service offline, failure, self-protection mechanism

SpringCloud source series (5) – Registry EurekaServer cluster for Eureka

SpringCloud source Code Series (6) – Summary of the Registry Eureka

Load balancing

RestTemplate

In exploring the eureka source code, we define a RestTemplate with the @loadBalanced tag in the Demo-Consumer service, The RestTemplate is then used to invoke the remote service Demo-Producer in the form of the service name. The request is then polled on both Demo-Producer instances.

RestTemplate is a network request framework in Spring Resources that accesses third-party RESTful apis. RestTemplate is used to consume REST services, so the main methods of RestTemplate are closely related to REST Http methods, such as HEAD, GET, POST, PUT, DELETE, and OPTIONS. These methods correspond to headForHeaders(), getForObject(), postForObject(), PUT (), and delete() in the RestTemplate class.

The RestTemplate itself is not LoadBalanced. If the @loadbalanced flag is not used, an error will be reported if the RestTemplate is called using the service name. With the @loadBalanced flag, the REST method that calls the RestTemplate is routed to a service instance using a load balancing policy. The Ribbon is responsible for load balancing. We’ll see how @loadbalanced makes the RestTemplate LoadBalanced later.

@SpringBootApplication
public class ConsumerApplication {

    @Bean
    @LoadBalanced
    public RestTemplate restTemplate(a) {
        return new RestTemplate();
    }

    public static void main(String[] args) { SpringApplication.run(ConsumerApplication.class, args); }}@RestController
public class DemoController {
    private final Logger logger = LoggerFactory.getLogger(getClass());

    @Autowired
    private RestTemplate restTemplate;

    @GetMapping("/v1/id")
    public ResponseEntity<String> getId(a) {
    	// called by the service name
        ResponseEntity<String> result = restTemplate.getForEntity("http://demo-producer/v1/uuid", String.class);
        String uuid = result.getBody();
        logger.info("request id: {}", uuid);
        returnResponseEntity.ok(uuid); }}Copy the code

Ribbon and load balancing

1. Load balancing

Load balancing refers to the distribution of load among multiple execution units. Load balancing can be divided into centralized load balancing and in-process load balancing:

• Centralized load balancing: stands between the Internet and the execution unit, and is responsible for forwarding network requests to the execution units, such as Nginx and F5. Centralized load balancing can also be called server load balancing.
• Intra-process load balancing: integrates load balancing logic into the client, which maintains a list of instances of service providers, typically retrieved from a registry such as Eureka. With a list of instances, you can achieve load balancing by allocating requests to multiple service providers through a load balancing policy. In-process load balancing is also known as client load balancing.

The Ribbon is a client load balancer that controls HTTP and TCP client load balancing behavior. The Ribbon is an open source load balancing component of Netflix that has been integrated into the SpringCloud ecosystem. It is an indispensable component in the SpringCloud ecosystem. Without it, the service cannot scale horizontally.

2. Ribbon Module

The Ribbon has many sub-modules. According to the official documentation, Netflix’s Ribbon is mainly used in production environment as follows:

• ribbon-loadbalancer: Load balancer API that can be used independently or with other modules.
• ribbon-eurekaThe Ribbon combines the Eureka API to provide dynamic service registry information for load balancers.
• ribbon-core: Core API of the Ribbon

3. Integration of SpringCloud and Ribbon

Similar to eureka’s integration into Spring Cloud, Spring Cloud provides a corresponding spring-Cloud-starter-Netflix-Eureka-client (server) dependency package, The ribbon is integrated into spring-Cloud-starter-Netflix-ribbon. There is no need to introduce the ribbon dependency package separately. Spring-cloud-starter-netflix-eureka-client already relies on spring-cloud-starter-Netflix-ribbon. Therefore, we introduced spring-cloud-starter-Netflix-Eureka-client to make use of the Ribbon functions.

4. Ribbon is integrated with RestTemplate

In Spring Cloud’s microservice system, Ribbon serves as a load balancer for service consumers. It can be used in two ways, one is combined with RestTemplate and the other is combined with Feign. Now that we’ve demonstrated the use of RestTemplate with load balancing, here’s a diagram to look at remote calls to RestTemplate based on the Ribbon.

RestTemplate Load balancing

@ LoadBalanced annotations

Take RestTemplate as a starting point to look at the core principle of load balancing in the Ribbon. So let’s first look at how the @loadbalanced annotation makes the RestTemplate LoadBalanced.

First look at the @loadBalanced annotation definition, we get the following information:

• This annotation uses@QualifierThe LoadBalanced annotation bean object can be injected elsewhere.
• As you can see from the comments, the RestTemplate or WebClient of the @loadBalanced tag will use itLoadBalancerClientTo configure the bean object.

/** * Annotation to mark a RestTemplate or WebClient bean to be configured to use a LoadBalancerClient. */
@Target({ ElementType.FIELD, ElementType.PARAMETER, ElementType.METHOD })
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Inherited
@Qualifier
public @interface LoadBalanced {

}
Copy the code

Note that @loadBalanced is under the loadBalancer package under the Spring-Cloud-Commons module.

RestTemplate Automatic load balancing configuration

In @ LoadBalanced with package, have a LoadBalancerAutoConfiguration automation configuration class, as can be seen from the comments also, this is the client load balance Ribbon automation configuration class.

From this automated configuration class you can get the following information:

• The first step is to have a dependency and definition for the RestTemplateLoadBalancerClientObject, which corresponds to the RestTemplate LoadBalancerClient configuration.
• You can then see that the class is injected with aRestTemplate object with the @loadBalanced annotationIs to increase the load balancing capability for these objects.
• fromSmartInitializingSingletonAfter the bean initialization is complete, use theRestTemplateCustomizerCustomize the RestTemplate.
• Further down, you can see that the RestTemplateCustomizer is actually added to the RestTemplateLoadBalancerInterceptorThis interceptor.
• The LoadBalancerInterceptor build requires thatLoadBalancerClient and LoadBalancerRequestFactory, LoadBalancerRequestFactory through LoadBalancerClient and LoadBalancerRequestTransformer constructed.

/** * Auto-configuration for Ribbon (client-side load balancing). */
@Configuration(proxyBeanMethods = false)
@ConditionalOnClass(RestTemplate.class) // Has a RestTemplate dependency
@ConditionalOnBean(LoadBalancerClient.class) // The bean object that defines LoadBalancerClient
@EnableConfigurationProperties(LoadBalancerRetryProperties.class)
public class LoadBalancerAutoConfiguration {

    // Inject the RestTemplate object with the @loadBalanced flag
    @LoadBalanced
    @Autowired(required = false)
    private List<RestTemplate> restTemplates = Collections.emptyList();

    @Autowired(required = false)
    private List<LoadBalancerRequestTransformer> transformers = Collections.emptyList();

    @Bean
    public SmartInitializingSingleton loadBalancedRestTemplateInitializerDeprecated(
            final ObjectProvider<List<RestTemplateCustomizer>> restTemplateCustomizers) {
        return () -> restTemplateCustomizers.ifAvailable(customizers -> {
            for (RestTemplate restTemplate : LoadBalancerAutoConfiguration.this.restTemplates) {
                for (RestTemplateCustomizer customizer : customizers) {
                    // Use RestTemplateCustomizer to customize the restTemplatecustomizer.customize(restTemplate); }}}); }@Bean
    @ConditionalOnMissingBean
    public LoadBalancerRequestFactory loadBalancerRequestFactory( LoadBalancerClient loadBalancerClient) {
        return new LoadBalancerRequestFactory(loadBalancerClient, this.transformers);
    }

    @Configuration(proxyBeanMethods = false)
    @ConditionalOnMissingClass("org.springframework.retry.support.RetryTemplate")
    static class LoadBalancerInterceptorConfig {

        / / create LoadBalancerInterceptor need LoadBalancerClient and LoadBalancerRequestFactory
        @Bean
        public LoadBalancerInterceptor ribbonInterceptor( LoadBalancerClient loadBalancerClient, LoadBalancerRequestFactory requestFactory) {
            return new LoadBalancerInterceptor(loadBalancerClient, requestFactory);
        }

        @Bean
        @ConditionalOnMissingBean
        public RestTemplateCustomizer restTemplateCustomizer(
                final LoadBalancerInterceptor loadBalancerInterceptor) {
            return restTemplate -> {
                List<ClientHttpRequestInterceptor> list = new ArrayList<>(
                        restTemplate.getInterceptors());
                // Add the LoadBalancerInterceptor interceptor to restTemplatelist.add(loadBalancerInterceptor); restTemplate.setInterceptors(list); }; }}}Copy the code

RestTemplate interceptor LoadBalancerInterceptor

LoadBalancerAutoConfiguration automation configuration is mainly to the RestTemplate added a load balancing LoadBalancerInterceptor interceptor. As can be seen from the setInterceptors parameters, the type of the interceptor is ClientHttpRequestInterceptor, if we want to customized RestTemplate can implement this interface to customize, You can then mark the sequence of interceptors with @order.

public void setInterceptors(List<ClientHttpRequestInterceptor> interceptors) {
    if (this.interceptors ! = interceptors) {this.interceptors.clear();
        this.interceptors.addAll(interceptors);
        // Sort according to the Order of @order annotations
        AnnotationAwareOrderComparator.sort(this.interceptors); }}Copy the code

The Interceptors are set in the parent class of The RestTemplate, InterceptingHttpAccessor, whose class structure is shown below.

From the restTemplate. GetForEntity (” http://demo-producer/v1/uuid “, String class) this GET request check, is how to use LoadBalancerInterceptor. As you work your way through, you can see that you’re finally getting into the doExecute method.

In the doExecute method, you first create a ClientHttpRequest based on the URL, method, and then use the ClientHttpRequest to initiate the request.

protected <T> T doExecute(URI url, @Nullable HttpMethod method, @Nullable RequestCallback requestCallback,
        @Nullable ResponseExtractor<T> responseExtractor) throws RestClientException {
    ClientHttpResponse response = null;
    try {
        // Create a ClientHttpRequest
        ClientHttpRequest request = createRequest(url, method);
        if(requestCallback ! =null) {
            requestCallback.doWithRequest(request);
        }
        // Call the execute() method of ClientHttpRequest
        response = request.execute();
        // Process the result
        handleResponse(url, method, response);
        return(responseExtractor ! =null ? responseExtractor.extractData(response) : null);
    }
    catch (IOException ex) {
        // ...
    }
    finally {
        if(response ! =null) { response.close(); }}}protected ClientHttpRequest createRequest(URI url, HttpMethod method) throws IOException {
    ClientHttpRequest request = getRequestFactory().createRequest(url, method);
    initialize(request);
    if (logger.isDebugEnabled()) {
        logger.debug("HTTP " + method.name() + "" + url);
    }
    return request;
}
Copy the code

InterceptingHttpAccessor overrides the parent HttpAccessor getRequestFactory method. The parent class is the default requestFactory SimpleClientHttpRequestFactory.

In the rewritten getRequestFactory method, if the interceptor is not empty, Based on the parent class default SimpleClientHttpRequestFactory founded InterceptingClientHttpRequestFactory and interceptors.

public ClientHttpRequestFactory getRequestFactory(a) {
    List<ClientHttpRequestInterceptor> interceptors = getInterceptors();
    if(! CollectionUtils.isEmpty(interceptors)) { ClientHttpRequestFactory factory =this.interceptingRequestFactory;
        if (factory == null) {
            / / incoming SimpleClientHttpRequestFactory and ClientHttpRequestInterceptor interceptors
            factory = new InterceptingClientHttpRequestFactory(super.getRequestFactory(), interceptors);
            this.interceptingRequestFactory = factory;
        }
        return factory;
    }
    else {
        return super.getRequestFactory(); }}Copy the code

That is called the createRequest method to create ClientHttpRequest InterceptingClientHttpRequestFactory. Inside you can see, the actual type is InterceptingClientHttpRequest ClientHttpRequest.

protected ClientHttpRequest createRequest(URI uri, HttpMethod httpMethod, ClientHttpRequestFactory requestFactory) {
    return new InterceptingClientHttpRequest(requestFactory, this.interceptors, uri, httpMethod);
}
Copy the code

InterceptingClientHttpRequest class structure is as follows:

In the doExecute RestTemplate call request. The execute () is invoked the InterceptingClientHttpRequest AbstractClientHttpRequest of parent The execute method. Step by step in to can find finally is actually called InterceptingClientHttpRequest executeInternal method.

In InterceptingClientHttpRequest executeInternal method, founded the InterceptingRequestExecution to execute the request. In InterceptingRequestExecution the execute method, first traverse to perform all interceptors, then initiated by ClientHttpRequest real HTTP requests.

protected final ClientHttpResponse executeInternal(HttpHeaders headers, byte[] bufferedOutput) throws IOException {
    / / create InterceptingRequestExecution
    InterceptingRequestExecution requestExecution = new InterceptingRequestExecution();
    // Request invocation
    return requestExecution.execute(this, bufferedOutput);
}

private class InterceptingRequestExecution implements ClientHttpRequestExecution {
    private final Iterator<ClientHttpRequestInterceptor> iterator;

    public InterceptingRequestExecution(a) {
        // Interceptor iterator
        this.iterator = interceptors.iterator();
    }

    @Override
    public ClientHttpResponse execute(HttpRequest request, byte[] body) throws IOException {
        if (this.iterator.hasNext()) {
            ClientHttpRequestInterceptor nextInterceptor = this.iterator.next();
            / / by using interceptors to intercept processing and the incoming InterceptingRequestExecution
            return nextInterceptor.intercept(request, body, this);
        }
        else {
            // The interceptor starts making the actual HTTP request after traversing
            HttpMethod method = request.getMethod();
            ClientHttpRequest delegate = requestFactory.createRequest(request.getURI(), method);
            request.getHeaders().forEach((key, value) -> delegate.getHeaders().addAll(key, value));
            / /...
            returndelegate.execute(); }}}Copy the code

The LoadBalancerInterceptor intercepts the service name from the original address of the request and calls the execute method of the loadBalancer. LoadBalancerClient.

As you can imagine, loadbalancer.execute is to get a specific instance based on the service name and replace the original address with the IP address of the instance. What loadBalancer is?

public ClientHttpResponse intercept(final HttpRequest request, final byte[] body, final ClientHttpRequestExecution execution) throws IOException {
    // Original address: http://demo-producer/v1/uuid
    final URI originalUri = request.getURI();
    // host is the service name: demo-producer
    String serviceName = originalUri.getHost();
    return this.loadBalancer.execute(serviceName, this.requestFactory.createRequest(request, body, execution));
}
Copy the code

Load balancing client LoadBalancerClient

When in the configuration LoadBalancerInterceptor, need two parameters, LoadBalancerClient and LoadBalancerRequestFactory, LoadBalancerRequestFactory already know is how to create. Where is the LoadBalancerClient created? Through IDEA search, it can be found that the RibbonAutoConfiguration under the Spring-Cloud-Netflix-Ribbon module is configured. The actual type of the LoadBalancerClient is RibbonLoadBalancerClient.

The configuration class order is EurekaClientAutoConfiguration, RibbonAutoConfiguration, LoadBalancerAutoConfiguration, The load balancing capability of RestTemplate requires the LoadBalancerInterceptor interceptor, and the LoadBalancerClient is required to create the LoadBalancerInterceptor. To obtain an instance based on the service name, LoadBalancerClient requires an instance library, such as a configuration file or a registry. As you can see from this, the RibbonLoadBalancerClient acquires the instance from the Eureka registry by default.

@Configuration
@Conditional(RibbonAutoConfiguration.RibbonClassesConditions.class)
@RibbonClients
/ / after EurekaClientAutoConfiguration configuration
@AutoConfigureAfter(name = "org.springframework.cloud.netflix.eureka.EurekaClientAutoConfiguration")
/ / before LoadBalancerAutoConfiguration configuration
@AutoConfigureBefore({ LoadBalancerAutoConfiguration.class, AsyncLoadBalancerAutoConfiguration.class })
@EnableConfigurationProperties({ RibbonEagerLoadProperties.class, ServerIntrospectorProperties.class })
public class RibbonAutoConfiguration {

    @Autowired(required = false)
    private List<RibbonClientSpecification> configurations = new ArrayList<>();

    @Bean
    @ConditionalOnMissingBean
    public SpringClientFactory springClientFactory(a) {
        SpringClientFactory factory = new SpringClientFactory();
        factory.setConfigurations(this.configurations);
        return factory;
    }

    @Bean
    @ConditionalOnMissingBean(LoadBalancerClient.class)
    public LoadBalancerClient loadBalancerClient(a) {
        return newRibbonLoadBalancerClient(springClientFactory()); }}Copy the code

LoadBalancerClient provides three interfaces:

public interface LoadBalancerClient extends ServiceInstanceChooser {

    // Find a Server from LoadBalancer to send the request
    <T> T execute(String serviceId, LoadBalancerRequest<T> request) throws IOException;

    // Take the Server from the incoming ServiceInstance to send the request
    <T> T execute(String serviceId, ServiceInstance serviceInstance, LoadBalancerRequest<T> request) throws IOException;

    // Refactor the original URI
    URI reconstructURI(ServiceInstance instance, URI original);
}
Copy the code

Enter the Execute method of RibbonLoadBalancerClient.

• Obtain the load balancer corresponding to the service based on the service nameILoadBalancer.
• Then an instance is selected from ILoadBalancer based on certain policiesServer.
• Then encapsulate information such as the server and serviceId toRibbonServerIs a ServiceInstance, ServiceInstance.
• LoadBalancerRequest is finally calledapplyAnd pass ServiceInstance to replace the service name in the address with the real IP address.

public <T> T execute(String serviceId, LoadBalancerRequest<T> request) throws IOException {
    return execute(serviceId, request, null);
}

public <T> T execute(String serviceId, LoadBalancerRequest<T> request, Object hint)
        throws IOException {
    // Obtain a load balancer based on the service name
    ILoadBalancer loadBalancer = getLoadBalancer(serviceId);
    // Use the load balancer to obtain the instance Server
    Server server = getServer(loadBalancer, hint);
    if (server == null) {
        throw new IllegalStateException("No instances available for " + serviceId);
    }
    // Encapsulate instance information: The parent class of RibbonServer is ServiceInstance
    RibbonServer ribbonServer = new RibbonServer(serviceId, server,
            isSecure(server, serviceId),
            serverIntrospector(serviceId).getMetadata(server));
    return execute(serviceId, ribbonServer, request);
}

@Override
public <T> T execute(String serviceId, ServiceInstance serviceInstance, LoadBalancerRequest
       
         request)
        throws IOException {
    Server server = null;
    if (serviceInstance instanceof RibbonServer) {
        server = ((RibbonServer) serviceInstance).getServer();
    }
    if (server == null) {
        throw new IllegalStateException("No instances available for " + serviceId);
    }

    try {
        // Process the address, replacing the service name with the real IP address
        T returnVal = request.apply(serviceInstance);
        return returnVal;
    } catch (Exception ex) {
        // ...
    }
    return null;
}
Copy the code

LoadBalancerRequest is an anonymous class created in the Intercept of LoadBalancerInterceptor. In its functional interface, Request is wrapped in a layer with the ServiceRequestWrapper decorator.

public LoadBalancerRequest<ClientHttpResponse> createRequest(final HttpRequest request, final byte[] body, final ClientHttpRequestExecution execution) {
    return instance -> {
        HttpRequest, ServiceRequestWrapper overrides the getURI method.
        HttpRequest serviceRequest = new ServiceRequestWrapper(request, instance, this.loadBalancer);
        if (this.transformers ! =null) {
            for (LoadBalancerRequestTransformer transformer : this.transformers) { serviceRequest = transformer.transformRequest(serviceRequest, instance); }}// Continue to execute interceptor
        return execution.execute(serviceRequest, body);
    };
}
Copy the code

Equestwrapper (loadBalancer) ¶ ServiceRequestWrapper (loadBalancer) ¶ Replace the service name in the original address with the real IP address and port address of the Server.

@Override
public URI getURI(a) {
    / / refactoring URI
    URI uri = this.loadBalancer.reconstructURI(this.instance, getRequest().getURI());
    return uri;
}
Copy the code

public URI reconstructURI(ServiceInstance instance, URI original) {
    Assert.notNull(instance, "instance can not be null");
    / / service name
    String serviceId = instance.getServiceId();
    RibbonLoadBalancerContext context = this.clientFactory.getLoadBalancerContext(serviceId);

    URI uri;
    Server server;
    if (instance instanceof RibbonServer) {
        RibbonServer ribbonServer = (RibbonServer) instance;
        server = ribbonServer.getServer();
        uri = updateToSecureConnectionIfNeeded(original, ribbonServer);
    }
    else {
        server = new Server(instance.getScheme(), instance.getHost(), instance.getPort());
        IClientConfig clientConfig = clientFactory.getClientConfig(serviceId);
        ServerIntrospector serverIntrospector = serverIntrospector(serviceId);
        uri = updateToSecureConnectionIfNeeded(original, clientConfig, serverIntrospector, server);
    }
    // Reconstruct the address
    return context.reconstructURIWithServer(server, uri);
}
Copy the code

ReconstructURIWithServer:

public URI reconstructURIWithServer(Server server, URI original) {
    String host = server.getHost();
    int port = server.getPort();
    String scheme = server.getScheme();

    if (host.equals(original.getHost())
            && port == original.getPort()
            && scheme == original.getScheme()) {
        return original;
    }
    if (scheme == null) {
        scheme = original.getScheme();
    }
    if (scheme == null) {
        scheme = deriveSchemeAndPortFromPartialUri(original).first();
    }

    try {
        StringBuilder sb = new StringBuilder();
        sb.append(scheme).append(": / /");
        if(! Strings.isNullOrEmpty(original.getRawUserInfo())) { sb.append(original.getRawUserInfo()).append("@");
        }
        sb.append(host);
        if (port >= 0) {
            sb.append(":").append(port);
        }
        sb.append(original.getRawPath());
        if(! Strings.isNullOrEmpty(original.getRawQuery())) { sb.append("?").append(original.getRawQuery());
        }
        if(! Strings.isNullOrEmpty(original.getRawFragment())) { sb.append("#").append(original.getRawFragment());
        }
        URI newURI = new URI(sb.toString());
        return newURI;
    } catch (URISyntaxException e) {
        throw newRuntimeException(e); }}Copy the code

RestTemplate Load balancing summary

At this point, we’ve pretty much figured out how a simple @loadbalanced annotation can make the RestTemplate LoadBalanced. This section concludes.

1. How does RestTemplate get load balancing capability

• 1) First, RestTemplate is the next network request framework of the Spring-Web module to access third-party RESTful apis
• 2) in the spring cloud microservices architecture, you can make the RestTemplate LoadBalanced by marking it with @loadbalanced
• 3) The core component that makes RestTemplate load balanced isLoadBalancerAutoConfigurationAdded to it in the configuration classLoadBalancerInterceptorLoad balancing interceptor
• 4) RestTemplate customizes by iterating through all interceptors before making an HTTP call. LoadBalancerInterceptor replaces the service name in the URI with the real IP address of the instance. Once the customization is complete, a real HTTP request is made.
• 5) The LoadBalancerInterceptor mainly uses the LoadBalancerClient to reconstruct the URI. The LoadBalancerClient can find an available instance based on the service name and reconstruct the URI.

2. Core components

There are multiple modules involved here. Here is the module to which the core component belongs:

Spring – the web:

• RestTemplate
• InterceptingClientHttpRequest: perform interceptors, and launched the final HTTP calls

Spring – the cloud – Commons:

• @LoadBalanced
• LoadBalancerAutoConfiguration
• LoadBalancerRequestFactory: create decoration class ServiceRequestWrapper replace the original HttpRequest, overloading getURI method.
• LoadBalancerInterceptor: load balancing interceptor
• LoadBalancerClient: load balancing client interface

Spring – the cloud – netflix – ribbon:

• RibbonLoadBalancerClient: Implementation class of LoadBalancerClient, the load balancing client of the Ribbon
• RibbonAutoConfiguration

Ribbon – loadbalancer:

• ILoadBalancer: load balancer
• Server: an instance

3. Finally, use another diagram to clarify the relationship of the RestTemplate block