This is the 25th day of my participation in the August Challenge

This article uses the source address: simple-rpc

The purpose of load balancing is to distribute requests across multiple machines according to a certain policy so that the system can scale horizontally. Load balancing in distributed service framework is accomplished by software algorithm.

In the distributed service framework, load balancing is implemented by the server, and its realization principle is as follows:

  • The service consumer gets the list of service providers from the service registry at the start of the reference and caches it to the local cache on the service caller.
  • Before initiating the service invocation, the service consumer selects the target machine from the local cache of the service provider list through some policy or algorithm, and then initiates the service invocation to complete the load balancing function.

Load balancing algorithm

The following interfaces are provided. The interface input parameter is the service provider list.

public interface LoadBalanceStrategy {
    Provider select(List<Provider> providers);
}
Copy the code

Weighted stochastic algorithm

First of all, the list of service providers is enlarged according to the weighted number. For example, the weighted number of service provider A is 3, which becomes A,A,A after amplification and is stored in the new list of service providers. Random numbers between the size range of the list of service providers are obtained as the index of the list of service providers to obtain services. The code is as follows:

public class WeightRandomLoadBalanceStrategyImpl implements LoadBalanceStrategy {

    @Override
    public Provider select(List<Provider> providers) {
        for (Provider provider : providers) {
            int weight = provider.getWeight();
            for (int i = 0; i < weight; i++) { providers.add(Provider.copy(provider)); }}int maxLen = providers.size();
        int random = RandomUtils.nextInt(0, maxLen - 1);
        returnproviders.get(random); }}Copy the code

Weighted polling algorithm

Retrieves the data in the list of service providers in sequence and uses a counter to record the index of the used data. If the index reaches the last data, the counter returns to 0 and the cycle begins again. The code is as follows:


public class WeightPollingLoadBalanceStrategyImpl implements LoadBalanceStrategy {

    private final AtomicInteger index = new AtomicInteger(0);
    private final Lock lock = new ReentrantLock();

    @Override
    public Provider select(List<Provider> providers) {
        for (Provider provider : providers) {
            int weight = provider.getWeight();
            for (int i = 0; i < weight; i++) {
                providers.add(Provider.copy(provider));
            }
        }
        lock.lock();
        try {
            if (index.get() >= providers.size()) {
                index.set(0);
            }
            Provider provider = providers.get(index.getAndIncrement());
            if (null == provider) {
                return providers.get(0);
            }
            return provider;
        } finally{ lock.unlock(); }}}Copy the code

Source address hash algorithm

The IP address of the request source is used to obtain the hash value, and then the service provider list index is modeled according to the size of the service provider list, and then the service provider is obtained. The code is as follows:

public class HashLoadBalanceStrategyImpl implements LoadBalanceStrategy {

    @Override
    public Provider select(List<Provider> providers) {
        int hashCode = IpUtil.getLocalIP().hashCode();
        returnproviders.get(hashCode % providers.size()); }}Copy the code

Load balancing engine

We need to integrate the above algorithm implementation into our distributed service framework implementation, so we define a load balancing engine class, use facade mode, provide unified API externally, and select different policy services according to different policy configuration. The policy engine code is implemented as follows:

public static LoadBalanceStrategy getLoadBalanceStrategy(String loadBalanceStrategy) {
    LoadBalanceStrategyEnum strategyEnum = LoadBalanceStrategyEnum.getByCode(loadBalanceStrategy);
    if (strategyEnum == null) {
        log.debug("can't find strategy with property loadBalanceStrategy = {}", loadBalanceStrategy);
        throw new SRpcException("can't find strategy with property loadBalanceStrategy：" + loadBalanceStrategy);
    } else {
        returnLOAD_BALANCE_STRATEGY_MAP.get(strategyEnum); }}Copy the code

summary

We provide several simple load balancing algorithms, as well as a policy engine for easy use. In the real world, however, it’s not as simple as that. Common open source frameworks have more complex algorithms such as consistent hashes, minimum active calls, and so on. The overall goal is the same, that is, to scatter traffic to the service provider machine.