You can subscribe if you search “App Alang” on wechat. Like and see, unlimited power.

This article Github.com/niumoo/Java… And program ape Alang blog has been included, there are many knowledge points and series of articles.

Load balancing is a technology that evenly distributes resources among multiple computers (networks, cpus, and disks) to improve resource utilization. The use of load balancing maximizes service throughput and may minimize response time, and also improves service availability due to the use of multiple server nodes for single point of service.

Load balancing can be realized by software or hardware, such as the famous F5 load balancing device, software such as NGINX load balancing, and Springcloud Ribbon component load balancing.

If you don’t know what load balancing is, you have to put up a picture, because it doesn’t tell you anything.

Load balancing requires that each server receives approximately the same number of requests. However, in actual production, the performance of each machine may be different, so we would like the machine with good performance to bear more requests, which is also normal demand.

If that doesn’t make sense to you, let me give you another example: a row of cute bears (servers).

This is when someone (the user) comes and slaps him in the face.

So how do we get each of these cute little bears to get hit roughly the same number of times?

Or maybe Bear 4 is fat and can take twice as many hits as others. How can we make bear 4 take twice as many hits as others?

Or maybe the strength of each hand is different, some are heavy and some are light, and it just so happens that Bear 4 is always slapped in the face with this great strength. Bear 4 is about to lose the bear, should we continue to hit it?

These are all worth thinking about.

Said so much, dry mouth, my hands have been hungry, can’t wait to masturbate the code.

1. Random interviews

So, in order to balance the load, we have to make sure that bear 1 to Bear 4 gets an even number of shots. Using random access, for example, the more random shots a bear makes, according to mathematical probability theory, the more similar each bear will be hit. Code implementation is also relatively simple, using a random number, random access to a can.

/** Server list */
private static List<String> serverList = new ArrayList<>();
static {
    serverList.add("192.168.1.2 instead");
    serverList.add("192.168.1.3");
    serverList.add("192.168.1.4");
    serverList.add("192.168.1.5");
}

/** * Random routing algorithm */
public static String random(a) {
    // Copy the collection used for traversal to prevent the collection from changing during operation
    List<String> tempList = new ArrayList<>(serverList.size());
    tempList.addAll(serverList);
    // Random number random access
    int randomInt = new Random().nextInt(tempList.size());
    return tempList.get(randomInt);
}
Copy the code

Because you use a non-thread-safe collection, you operate on a copy of the collection when accessing it, as is the idea in the following polling methods.

Write a mock request method, request 10W times, record the result of the request.

public static void main(String[] args) {
    HashMap<String, Integer> serverMap = new HashMap<>();
    for (int i = 0; i < 20000; i++) {
        String server = random();
        Integer count = serverMap.get(server);
        if (count == null) {
            count = 1;
        } else {
            count++;
        }
        / / record
        serverMap.put(server, count);
    }
    // Total result of routing
    for (Map.Entry<String, Integer> entry : serverMap.entrySet()) {
        System.out.println("IP:" + entry.getKey() + ", times:"+ entry.getValue()); }}Copy the code

Run to get the request result.

IP:192.168.1.3, Count: 24979 IP:192.168.1.2, Count: 24896 IP:192.168.1.5, Count: 25043 IP:192.168.1.4, Count: 25082Copy the code

The number of times each server is accessed tends to be about 2.5W, which is kind of load balancing. But randomness is randomness, and there is no guarantee that the number of accesses will be absolutely uniform.

2. Polling access

Polling interview is much easier, take bear 1 to bear 4 above for example, we slap face one after another, bear 1 hit bear 2, bear 2 hit bear 3, bear 4 hit bear 1, finally also achieved the balance. But there’s always a price to pay for uniformity, you need randomness in random access, what do you need in polling access to ensure polling?

/** Server list */
private static List<String> serverList = new ArrayList<>();
static {
    serverList.add("192.168.1.2 instead");
    serverList.add("192.168.1.3");
    serverList.add("192.168.1.4");
    serverList.add("192.168.1.5");
}
private static Integer index = 0;

/** * Random routing algorithm */
public static String randomOneByOne(a) {
    // Copy the collection used for traversal to prevent the collection from changing during operation
    List<String> tempList = new ArrayList<>(serverList.size());
    tempList.addAll(serverList);
    String server = "";
    synchronized (index) {
        index++;
        if (index == tempList.size()) {
            index = 0;
        }
        server = tempList.get(index);;
    }
    return server;
}
Copy the code

As you can see from the code, the location of the last access must be recorded in order to ensure polling, and locking must be performed when using the location record in order to avoid problems in concurrent cases. Obviously, this mutex adds performance overhead.

Use the same test code as above to test the 10W request load.

IP:192.1681.3., the number of times:25000
IP:192.1681.2., the number of times:25000
IP:192.1681.. 5, the number of times:25000
IP:192.1681.4., the number of times:25000
Copy the code

3. Polling weighting

The above demonstrates the polling mode, remember the beginning of the bear 4 is fat, strong ability to resist beating, can withstand other people twice the number of times? Neither of the above methods reflects this characteristic of Bear 4, which is sneaky and painless. But bear 1 to Bear 3 are on the verge of collapse, no, we have to let the fat play more, versatile, improve overall performance.

/** Server list */
private static HashMap<String, Integer> serverMap = new HashMap<>();
static {
    serverMap.put("192.168.1.2 instead".2);
    serverMap.put("192.168.1.3".2);
    serverMap.put("192.168.1.4".2);
    serverMap.put("192.168.1.5".4);
}
private static Integer index = 0;

/** * Weighted routing algorithm */
public static String oneByOneWithWeight(a) {
    List<String> tempList = new ArrayList();
    HashMap<String, Integer> tempMap = new HashMap<>();
    tempMap.putAll(serverMap);
    for (String key : serverMap.keySet()) {
        for (int i = 0; i < serverMap.get(key); i++) { tempList.add(key); }}synchronized (index) {
        index++;
        if (index == tempList.size()) {
            index = 0;
        }
        returntempList.get(index); }}Copy the code

This time the overall performance of each server is recorded and a number is given, with the higher the number, the better the performance. The more requests you can handle, you can see that the server 192.168.1.5 performs at 4, twice as well as the other servers, still testing 10W requests.

IP:192.1681.3., the number of times:20000
IP:192.1681.2., the number of times:20000
IP:192.1681.. 5, the number of times:40000
IP:192.1681.4., the number of times:20000
Copy the code

192.168.1.5 took twice as many requests.

4. Random weighting

The method of random weighting is basically the same as the method of polling weighting, except that the method of polling using mutex is replaced by random access. According to probability theory, when the number of visits increases, the service access will also reach load balance.

/** Server list */
private static HashMap<String, Integer> serverMap = new HashMap<>();
static {
    serverMap.put("192.168.1.2 instead".2);
    serverMap.put("192.168.1.3".2);
    serverMap.put("192.168.1.4".2);
    serverMap.put("192.168.1.5".4);
}
/** * Weighted routing algorithm */
public static String randomWithWeight(a) {
    List<String> tempList = new ArrayList();
    HashMap<String, Integer> tempMap = new HashMap<>();
    tempMap.putAll(serverMap);
    for (String key : serverMap.keySet()) {
        for (int i = 0; i < serverMap.get(key); i++) { tempList.add(key); }}int randomInt = new Random().nextInt(tempList.size());
    return tempList.get(randomInt);
}
Copy the code

Still 10 W request test, 192.168.1.5 weight is approximately twice that of other servers,

IP:192.168.1.3, Count: 19934 IP:192.168.1.2, Count: 20033 IP:192.168.1.5, Count: 39900 IP:192.168.1.4, Count: 20133Copy the code

5. IP-Hash

The above methods either use random numbers or polling, and ultimately achieve load balancing of requests. However, there is also an obvious disadvantage, that is, multiple requests from the same user may not be processed by the same service, then the problem arises. If your service depends on session, the session will be lost because of different services, which is not what we want. So there is a way to determine which server the request goes to by hashing the IP of the requesting side. This ensures that requests from the same user land on the same service.

private static List<String> serverList = new ArrayList<>();
static {
    serverList.add("192.168.1.2 instead");
    serverList.add("192.168.1.3");
    serverList.add("192.168.1.4");
    serverList.add("192.168.1.5");
}

/** * IP hash routing algorithm */
public static String ipHash(String ip) {
    // Copy the collection used for traversal to prevent the collection from changing during operation
    List<String> tempList = new ArrayList<>(serverList.size());
    tempList.addAll(serverList);
    // Hash the requested server
    int index = ip.hashCode() % serverList.size();
    return tempList.get(Math.abs(index));
}
Copy the code

6. Summary

The above four methods seem good, but does this really reflect the load balancing mentioned in the beginning? The answer is not necessarily. Like the last question above.

Or maybe the strength of each hand is different, some are heavy and some are light, and it just so happens that Bear 4 is always slapped in the face with this great strength. Bear 4 is about to lose the bear, should we continue to hit it?

The same is true for servers, where each requested operation may consume different resources. For example, some operations are relatively high CPU usage, which is also normal. Therefore, load balancing sometimes cannot simply be based on the load of requests as the sole basis for load balancing. It can also be combined with the current number of connections of services, recent response time and other dimensions for overall balancing, in a word, to achieve load balancing of resource usage.

Hello world:) I’m Aaron, a tech tool guy on the front line. The article continues to update, can pay attention to the public number “program ape A Lang” grow together. This article has been received at Github.com/niumoo/Java… And “Unread Code,” lots of knowledge and a series of articles.