Spring Cloud Kubernetes Ribbon source code analysis

The purpose of using the Ribbon in a project is to achieve load balancing on the client side (service consumer side).

In our last article on Spring Cloud OpenFeign source code analysis, we analyzed why using OpenFeign without configuring urls and importing Ribbon dependencies will cause errors. This article continues with an analysis of how OpenFeign integrates with the Ribbon, how the Ribbon achieves load balancing, and how the Ribbon obtains services from the registry.

`OpenFeign`with`Ribbon`Integrated interface invocation process

The process of integrating OpenFeign and Ribbon to implement the load balancing call interface is as follows:

Spring – the cloud – openfeign – the core module:

1, callLoadBalancerFeignClienttheexecuteCall a remote method;
2, callFeignLoadBalancertheexecuteWithLoadBalancerMethod to implement load balancing calls.

Ribbon – the core module:

3, callLoadBalancerCommandthesubmitMethod to implement asynchronous calls to block synchronously waiting for results.
4, callLoadBalancerCommandtheselectServerMethod to load balance a call from multiple service providers;

Ribbon – loadbalancer module:

5, callILoadBalancerthechooseServerMethod service selection;
6, callIRulethechooseMethods Select a service according to some algorithm, such as random algorithm, polling algorithm;

`OpenFeign`How to`Ribbon`Integration of

SCK – demo project project address: https://github.com/wujiuye/share-projects/tree/master/sck-demo.

When using OpenFeign, if the @FeignClient URL attribute is not configured, then we need to import the spring-Cloud-starter – Kubernetes-ribbon dependency and use LoadBalancerFeignClient to call the interface. If we don’t need to use the Ribbon for load balancing, we can simply set @feignClient’s URL property to http://{serviceId} without Ribbon dependencies.

Add spring-cloud-starter- Kubernetes-ribbon dependencies to sck-Demo projects. Add spring-Cloud-starter – Netflix-ribbon dependencies to non-Spring Cloud Kubernetes projects.

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-kubernetes-ribbon</artifactId>
</dependency>
Copy the code

When we added the spring-cloud-starter-Kubernetes-ribbon dependency configuration to our project, Both spring-Cloud-starter-Netflix-ribbon and Spring-Cloud-Kubernetes-ribbon will be imported into the project, as shown in the following figure.

When openFeign is used in a project and spring-cloud-starter- Netflix -ribbon is added, the Ribbon can be automatically configured to integrate with OpenFeign and inject ILoadBalancer implementation class instances into the project. The default is ZoneAwareLoadBalancer, which is a class under Spring-Cloud-Netflix-ribbon.

The spring.factories file in the meta-INF directory of the spring-cloud-netflix-ribbon looks like this:

org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
org.springframework.cloud.netflix.ribbon.RibbonAutoConfiguration
Copy the code

Spring-cloud-netflix-ribbon is an implementation of the Loadbalancer interface for Spring-Cloud-Commons.

RibbonAutoConfiguration injects a LoadBalancerClient, a load balancing interface defined by spring-cloud-commons. RibbonLoadBalancerClient is the Ribbon implementation class for the Spring-Cloud-Commons load balancing interface LoadBalancerClient. It is provided to code using the @loadBalanced annotation.

    @Bean
	@ConditionalOnMissingBean(LoadBalancerClient.class)
	public LoadBalancerClient loadBalancerClient(a) {
		return new RibbonLoadBalancerClient(springClientFactory());
	}
Copy the code

Call the springClientFactory method when creating the RibbonLoadBalancerClient to create the springClientFactory:

    @Bean
	public SpringClientFactory springClientFactory(a) {
		SpringClientFactory factory = new SpringClientFactory();
		factory.setConfigurations(this.configurations);
		return factory;
	}
Copy the code

SpringClientFactory is a subclass of NamedContextFactory, its building method calls the superclass constructor when introduced into a configuration class RibbonClientConfiguration. Class, This is why RibbonClientConfiguration configuration class effect.

public class SpringClientFactory extends NamedContextFactory<RibbonClientSpecification> {

	static final String NAMESPACE = "ribbon";

	public SpringClientFactory(a) {
		super(RibbonClientConfiguration.class, NAMESPACE, "ribbon.client.name"); }}Copy the code

SpringClientFactory creates an ApplicationContext for each service provider, implements bean isolation, resolves bean name conflicts, and implements different configurations.

When creating ApplicationContext registered defaultConfigType to bean factory, is the defaultConfigType constructor passed RibbonClientConfiguration. Class.

protected AnnotationConfigApplicationContext createContext(String name) {
        / / create the ApplicationContext
		AnnotationConfigApplicationContext context = newAnnotationConfigApplicationContext(); .// Register multiple Configuration classes
		context.register(PropertyPlaceholderAutoConfiguration.class,
		                this.defaultConfigType); .// Call the Refresh method of ApplicationContext
		context.refresh();
		return context;
	}
Copy the code

So when is the createContext method called?

Take the service consumer invoking the service provider interface in SCK-Demo as an example:

@Service
public class DemoInvokeServiceImpl implements DemoInvokeService {

    @Resource
    private DemoService demoService;

    @Override
    public ListGenericResponse<DemoDto> invokeDemo(a) {
        returndemoService.getServices(); }}Copy the code

DemoService is the interface declared by the @FeignClient annotation. When we call one of DemoService’s methods, we know from Spring Cloud OpenFeign source code analysis that The execute method of LoadBalancerFeignClient is eventually called.

public class LoadBalancerFeignClient implements Client {
    / /...
	private SpringClientFactory clientFactory;

       Execute ()
	@Override
	public Response execute(Request request, Request.Options options) throws IOException{
	    / /...
	    IClientConfig requestConfig = getClientConfig(options, clientName);
	    / /...}}Copy the code

The execute method calls the getClientConfig method to get the IClientConfig instance from SpringClientFactory, which is the client configuration. The getClientConfig method gets the bean that implements the IClientConfig interface from the service provider’s ApplicationContext factory.

When calling a service provider interface for the first time, because did not initialize AnnotationConfigApplicationContext, so will call createContext method to create ApplicationContext, This method will RibbonClientConfiguration class registered to ApplicationContext, last call context. The refresh (); Will call to RibbonClientConfiguration was @ Bean annotation methods.

@Configuration(proxyBeanMethods = false)
@EnableConfigurationProperties
@Import({ HttpClientConfiguration.class, OkHttpRibbonConfiguration.class,
		RestClientRibbonConfiguration.class, HttpClientRibbonConfiguration.class })
public class RibbonClientConfiguration {
    / /...
    
	@RibbonClientName
	private String name = "client";

	@Autowired
	private PropertiesFactory propertiesFactory;

    // IClientConfig instance to configure client connection timeout and read timeout
	@Bean
	@ConditionalOnMissingBean
	public IClientConfig ribbonClientConfig(a) {
		DefaultClientConfigImpl config = new DefaultClientConfigImpl();
		config.loadProperties(this.name);
		config.set(CommonClientConfigKey.ConnectTimeout, DEFAULT_CONNECT_TIMEOUT);
		config.set(CommonClientConfigKey.ReadTimeout, DEFAULT_READ_TIMEOUT);
		config.set(CommonClientConfigKey.GZipPayload, DEFAULT_GZIP_PAYLOAD);
		return config;
	}

    // Configure the load balancing algorithm used by the Ribbon. By default, ZoneAvoidanceRule is used
	@Bean
	@ConditionalOnMissingBean
	public IRule ribbonRule(IClientConfig config) {
		if (this.propertiesFactory.isSet(IRule.class, name)) {
			return this.propertiesFactory.get(IRule.class, config, name);
		}
		ZoneAvoidanceRule rule = new ZoneAvoidanceRule();
		rule.initWithNiwsConfig(config);
		return rule;
	}

    // Configure the service update to periodically pull services from the registry, started by ILoadBalancer
	@Bean
	@ConditionalOnMissingBean
	public ServerListUpdater ribbonServerListUpdater(IClientConfig config) {
		return new PollingServerListUpdater(config);
	}

    // Configure the ribbon load balancer. The default value is ZoneAwareLoadBalancer
	@Bean
	@ConditionalOnMissingBean
	public ILoadBalancer ribbonLoadBalancer(IClientConfig config, ServerList
       
         serverList, ServerListFilter
        
          serverListFilter, IRule rule, IPing ping, ServerListUpdater serverListUpdater)
        
        {
		if (this.propertiesFactory.isSet(ILoadBalancer.class, name)) {
			return this.propertiesFactory.get(ILoadBalancer.class, config, name);
		}
		return new ZoneAwareLoadBalancer<>(config, rule, ping, serverList,
				serverListFilter, serverListUpdater);
	}
	
	/ /... The others are not to be understood
}
Copy the code

ILoadBalancer is the load balancing interface defined by the Ribbon. ZoneAwareLoadBalancer is a subclass of DynamicServerListLoadBalancer, DynamicServerListLoadBalancer encapsulates the service update logic.

DynamicServerListLoadBalancer service call enableAndInitLearnNewServersFeature method in the way of constructing open updater ServerListUpdater, ServerListUpdater periodically pulls available services from the registry to update the service list cache.

public class DynamicServerListLoadBalancer<T extends Server> extends BaseLoadBalancer {
    protected volatile ServerListUpdater serverListUpdater;
    protected final ServerListUpdater.UpdateAction updateAction = new ServerListUpdater.UpdateAction() {
        @Override
        public void doUpdate(a) { updateListOfServers(); }};public void enableAndInitLearnNewServersFeature(a){
        serverListUpdater.start(updateAction);
    }
    // Call ServerList to get the service
    @VisibleForTesting
    public void updateListOfServers(a) {
        List<T> servers = new ArrayList<T>();
        if(serverListImpl ! =null) {
            servers = serverListImpl.getUpdatedListOfServers();
            // If you need to filter
            if(filter ! =null) { servers = filter.getFilteredListOfServers(servers); } } updateAllServerList(servers); }}Copy the code

`Ribbon`How is load balancing implemented

ServerList We will explain this later. First, we will make clear the entire call link after openFegin and Ribbon are integrated. We continue our analysis from the Execute method of the LoadBalancerFeignClient. (LoadBalancerFeignClient by FeignRibbonClientAutoConfiguration automatic configuration class configuration, if you forget to look down on a paper.)

public class LoadBalancerFeignClient implements Client {
    / /...
	private SpringClientFactory clientFactory;

	@Override
	public Response execute(Request request, Request.Options options) throws IOException {
		try {
			URI asUri = URI.create(request.url());
			// Provider is the name of the service. For example, SCK-Demo-Prodiver
			String clientName = asUri.getHost();
			URI uriWithoutHost = cleanUrl(request.url(), clientName);
			// delegate is: class Default implements Client {}
			FeignLoadBalancer.RibbonRequest ribbonRequest = new FeignLoadBalancer.RibbonRequest(
					this.delegate, request, uriWithoutHost);
			// Get the client configuration first
			IClientConfig requestConfig = getClientConfig(options, clientName);
			return
			    // Load balancing selects a service provider
			    lbClient(clientName)
			    // Call the interface to get the response
				.executeWithLoadBalancer(ribbonRequest, requestConfig).toResponse();
		}
		catch (ClientException e) {
			IOException io = findIOException(e);
			if(io ! =null) {
				throw io;
			}
			throw newRuntimeException(e); }}}Copy the code

LbClient creates a FeignLoadBalancer object and calls the FeignLoadBalancer executeWithLoadBalancer method to implement the load balancing call interface. The execute method of FeignLoadBalancer is eventually called. The Ribbon uses RxJava to make asynchronous calls to block synchronously to get results.

public T executeWithLoadBalancer(final S request, final IClientConfig requestConfig) throws ClientException {
        // Command also encapsulates the implementation logic for load balancing
        LoadBalancerCommand<T> command = buildLoadBalancerCommand(request, requestConfig);
        try {
            return command.submit(
                new ServerOperation<T>() {
                    @Override
                    public Observable<T> call(Server server) {
                        URI finalUri = reconstructURIWithServer(server, request.getUri());
                        S requestForServer = (S) request.replaceUri(finalUri);
                        try {
                            // Call the execute method of FeignLoadBalancer
                            return Observable.just(
                            AbstractLoadBalancerAwareClient.this.execute(requestForServer, requestConfig)
                            );
                        } 
                        catch (Exception e) {
                            return Observable.error(e);
                        }
                    }
                })
                .toBlocking()
                .single();
        } catch(Exception e) { ..... }}Copy the code

The submit method of LoadBalancerCommand has a lot of code and complicated logic, so I will not expand the description.

public Observable<T> submit(final ServerOperation<T> operation) {
    // Use the load balancer
    Observable<T> o = (server == null ? selectServer() : Observable.just(server))
}
Copy the code

The selectServer method returns an Observable

, which is an RxJava API that we’ll skip.

private Observable<Server> selectServer(a) {
        return Observable.create(new OnSubscribe<Server>() {
            @Override
            public void call(Subscriber<? super Server> next) {
                try {
                    / / call LoadBalancerContext getServerFromLoadBalancer method
                    Server server = loadBalancerContext.getServerFromLoadBalancer(loadBalancerURI, loadBalancerKey);   
                    next.onNext(server);
                    next.onCompleted();
                } catch(Exception e) { next.onError(e); }}}); }Copy the code

SelectServer method called LoadBalancerContext getServerFromLoadBalancer method to obtain a service provider, This LoadBalancerContext is actually FeignLoadBalancer (the answer can be found in the buildLoadBalancerCommand method).

GetServerFromLoadBalancer methods part of the code is as follows:

public Server getServerFromLoadBalancer(@Nullable URI original, @Nullable Object loadBalancerKey) throws ClientException {
    ILoadBalancer lb = getLoadBalancer();
    if (host == null) {
        if(lb ! =null){
            Server svc = lb.chooseServer(loadBalancerKey);
            return svc;
        } 
        / /...}}Copy the code

Since the Ribbon uses ILoadBalancer by default as ZoneAwareLoadBalancer, the getLoadBalancer method returns ZoneAwareLoadBalancer. The chooseServer that calls the load balancer after obtaining the load balancer selects a service provider.

ZoneAwareLoadBalancer chooseServer method:

    @Override
    public Server chooseServer(Object key) {
        if(! ENABLED.get() || getLoadBalancerStats().getAvailableZones().size() <=1) {
            logger.debug("Zone aware logic disabled or there is only one zone");
            return super.chooseServer(key); }}Copy the code

BaseLoadBalancer chooseServer (BaseLoadBalancer);

public class BaseLoadBalancer extends AbstractLoadBalancer implements
        PrimeConnections.PrimeConnectionListener.IClientConfigAware {
        
        protected IRule rule = DEFAULT_RULE;
        
    public Server chooseServer(Object key) {
        if (counter == null) {
            counter = createCounter();
        }
        counter.increment();
        if (rule == null) {
            return null;
        } else {
            try {
                // Call IRule's Choose method, whose rule was injected through the constructor when the ZoneAwareLoadBalancer was created
                return rule.choose(key);
            } catch (Exception e) {
                logger.warn("LoadBalancer [{}]: Error choosing server for key {}", name, key, e);
                return null; }}}}Copy the code

IRule is a service selector and an implementation of load balancing algorithms. Injected in the RibbonAutoConfiguration configuration class.

    // Configure the load balancing algorithm used by the Ribbon. By default, ZoneAvoidanceRule is used
	@Bean
	@ConditionalOnMissingBean
	public IRule ribbonRule(IClientConfig config) {
		if (this.propertiesFactory.isSet(IRule.class, name)) {
			return this.propertiesFactory.get(IRule.class, config, name);
		}
		ZoneAvoidanceRule rule = new ZoneAvoidanceRule();
		rule.initWithNiwsConfig(config);
		return rule;
	}
Copy the code

Injected through the constructor when the ZoneAwareLoadBalancer is created.

    @Bean
	@ConditionalOnMissingBean
	public ILoadBalancer ribbonLoadBalancer(IClientConfig config, ServerList
       
         serverList, ServerListFilter
        
          serverListFilter, IRule rule, IPing ping, ServerListUpdater serverListUpdater)
        
        {
	        / /...
		return new ZoneAwareLoadBalancer<>(config, rule, ping, serverList,
				serverListFilter, serverListUpdater);
	}
Copy the code

How ZoneAvoidanceRule selects a call from multiple service providers, which is the implementation of the load-balancing algorithm, is not covered in this article.

`Ribbon`How is the service provider obtained from the registry

Our analysis to the front, ZoneAwareLoadBalancer is a subclass of DynamicServerListLoadBalancer, DynamicServerListLoadBalancer encapsulates the service update logic, Periodically call the ServerList getUpdatedListOfServers method to pull services from the registry.

ServerList is a class in the ribbon- Loadbalancer package. It is not a spring-Cloud interface, so it has nothing to do with the spring-Cloud service discovery interface.

public interface ServerList<T extends Server> {

    public List<T> getInitialListOfServers(a);
    
    /** * Return updated list of servers. This is called say every 30 secs * (configurable) by the Loadbalancer's Ping cycle * * /
    public List<T> getUpdatedListOfServers(a);   

}
Copy the code

Based on the analysis of RibbonClientConfiguration, we found that there is a way can register a ServerList < > Server, but this method will not perform.

public class RibbonClientConfiguration {
    
    @Bean
	@ConditionalOnMissingBean
	public ServerList<Server> ribbonServerList(IClientConfig config) {
		if (this.propertiesFactory.isSet(ServerList.class, name)) {
			return this.propertiesFactory.get(ServerList.class, config, name);
		}
		ConfigurationBasedServerList serverList = new ConfigurationBasedServerList();
		serverList.initWithNiwsConfig(config);
		returnserverList; }}Copy the code

Since we used spring-cloud-starter-Kubernetes-ribbon in our SCK-Demo project, let’s take a look at what spring-Cloud-Kubernetes-ribbon is responsible for. Start by finding the auto-configuration class in the spring.factories file in spring-cloud-starter-kubernetes-ribbon.

org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
org.springframework.cloud.kubernetes.ribbon.RibbonKubernetesAutoConfiguration
Copy the code

Automatic configuration class RibbonKubernetesAutoConfiguration source code is as follows:

@Configuration(proxyBeanMethods = false)
@EnableConfigurationProperties
@ConditionalOnBean(SpringClientFactory.class)
@ConditionalOnProperty(value = "spring.cloud.kubernetes.ribbon.enabled",matchIfMissing = true)
@AutoConfigureAfter(RibbonAutoConfiguration.class)
@RibbonClients(defaultConfiguration = KubernetesRibbonClientConfiguration.class)
public class RibbonKubernetesAutoConfiguration {}Copy the code

SpringClientFactory we analyzed, RibbonAutoConfiguration we also analyzed the, only KubernetesRibbonClientConfiguration the configuration class.

KubernetesRibbonClientConfiguration is using the @ RibbonClients annotations to import the configuration of the class, namely through ImportBeanDefinitionRegistrar registration.

@Configuration(proxyBeanMethods = false)
@EnableConfigurationProperties(KubernetesRibbonProperties.class)
public class KubernetesRibbonClientConfiguration {

	@Bean
	@ConditionalOnMissingBean
	publicServerList<? > ribbonServerList(KubernetesClient client, IClientConfig config, KubernetesRibbonProperties properties) { KubernetesServerList serverList;if (properties.getMode() == KubernetesRibbonMode.SERVICE) {
			serverList = new KubernetesServicesServerList(client, properties);
		}
		else {
			serverList = new KubernetesEndpointsServerList(client, properties);
		}
		serverList.initWithNiwsConfig(config);
		returnserverList; }}Copy the code

Spring-cloud-kubernetes-ribbon is responsible for implementing the ribbon’s ServerList

interface. When spring. Cloud. Kubernetes. Ribbon. Mode configuration for the SERVICE, using KubernetesServicesServerList, otherwise use KubernetesEndpointsServerList. The default mode is POD.

@ConfigurationProperties(prefix = "spring.cloud.kubernetes.ribbon")
public class KubernetesRibbonProperties {
	/** * Ribbon enabled,default true. */
	private Boolean enabled = true;
	/ * * * {@link KubernetesRibbonMode} setting ribbon server list with ip of pod or service
	 * name. default value is POD.
	 */
	private KubernetesRibbonMode mode = KubernetesRibbonMode.POD;
	/** * cluster domain. */
	private String clusterDomain = "cluster.local";
}
Copy the code

KubernetesRibbonMode is an enumeration class that supports POD and Service.

public enum KubernetesRibbonMode {

	/** * using pod ip and port. */
	POD,
	/** * using kubernetes service name and port. */
	SERVICE

}
Copy the code

What does that mean? If mode is Service, the Ribbon obtains the service name and port number of the service provider in Kubernetes. If mode is service, the Ribbon uses Kubernetes instead of load balancing. When mode is POD, it is to obtain the IP and port of the pod of the service provider, which is the internal IP of the Kubernetes cluster. As long as the service consumer is deployed in the same Kubernetes cluster, it can access the service provider on the POD through the IP of pod and the port exposed by the service provider.

If we don’t want to use the Ribbon for load balancing, we can add the following configuration items to our configuration file:

spring:
  cloud:
    kubernetes:
      ribbon:
        mode: SERVICE
Copy the code

Did you learn? In the next article we looked at service registration for Spring Cloud Kubernetes.

Spring Cloud Kubernetes Ribbon source code analysis

OpenFeignwithRibbonIntegrated interface invocation process

OpenFeignHow toRibbonIntegration of

RibbonHow is load balancing implemented

RibbonHow is the service provider obtained from the registry

Related Posts

Buried point

Performance optimization of the Android performance optimization installation package

“Java” reflection

`OpenFeign`with`Ribbon`Integrated interface invocation process

`OpenFeign`How to`Ribbon`Integration of

`Ribbon`How is load balancing implemented

`Ribbon`How is the service provider obtained from the registry