Stage one, build website alone
In the early days of the site, we used to run all our programs and software on a single computer. At this point we use a container such as Tomcat, Jetty, jBOOS, and then directly use JSP/servlet technology, Or use some open source frameworks such as Maven + Spring +struct+ Hibernate, Maven + Spring + SpringMVC + Mybatis; Finally, select a database management system to store data, such as mysql, sqlserver, oracle, and then connect and operate the database through JDBC.
Put all the above software on the same machine, the application runs, is also a small system. The system result is as follows:
Phase 2: Separating the application server from the database
With the online of the website, the number of visits is gradually rising, the load of the server is slowly improving, when the server is not overloaded, we should be ready to improve the load capacity of the website. If our code level has been difficult to optimize, without improving the performance of a single machine, adding machines is a good way, not only can effectively improve the load capacity of the system, but also cost-effective.
What are the additional machines for? At this point, we can split the database and Web server, which not only improves the load capacity of a single machine, but also improves the disaster recovery capacity.
The architecture of the application server separated from the database is as follows:
Phase 3: Application server cluster
As traffic continues to increase, a single application server can no longer meet the demand. Assuming no strain on the database server, we can switch from one application server to two or even more, spreading user requests among different servers to increase load capacity. There is no direct interaction between multiple application servers, which rely on databases to provide services externally. Keepalived is a switch mechanism similar to layer3, 4 and 7. Keepalived is not exclusive to a specific failover application, but is a product that can be applied to a variety of applications. Keepalived, coupled with IPVSADm, is a magic tool for load balancing.
Taking the addition of an application server as an example, the system structure diagram after the addition is as follows:
When the system evolves to this point, the following four problems will occur:
Who forwards user requests to specific application servers
What are the forwarding algorithms
How does the application server return user requests
If users access different servers each time, how can session consistency be maintained
Let’s look at the solution to the problem:
1. The first problem is the problem of load balancing. Generally, there are five solutions:
1. HTTP redirection. HTTP redirection is request forwarding at the application layer. The user’s request is actually sent to the HTTP redirection load balancing server. The server redirects the user based on the algorithm. After receiving the redirection request, the user requests the real cluster again
Pros: Simple.
Disadvantages: Poor performance.
2. DNS domain name resolution load balancing. When a user requests the DNS server for an IP address corresponding to a domain name, the DNS server directly provides the IP address of the load balancing server.
Advantages: DNS, we don’t have to maintain the load balancing server.
Disadvantages: When an application server is down, DNS cannot be notified in a timely manner, and control of DNS load balancing is in the domain name service provider, so the site cannot be improved and managed more strongly.
3. Reverse proxy server. When the user’s request reaches the reverse proxy server (has reached the website room), the reverse proxy server forwards it to the specific server according to the algorithm. Common Apache, Nginx can act as a reverse proxy server.
Advantages: Simple deployment.
Disadvantages: Proxy servers can be a performance bottleneck, especially when uploading large files at once.
4. IP layer load balancing. After the request reaches the load balancer, the load balancer changes the destination IP address of the request to forward the request for load balancing.
Advantages: Better performance.
Disadvantages: The bandwidth of the load balancer becomes a bottleneck.
5. Data link layer load balancing. After the request reaches the load balancer, the LOAD balancer modifies the MAC address of the request to implement load balancing. Unlike IP load balancing, the load balancer directly returns the request to the customer after accessing the server. Without going through the load balancer.
2. The second problem is the cluster scheduling algorithm. There are 10 common scheduling algorithms.
1. Rr polling scheduling algorithm. As the name implies, polling distribution requests.
Advantages: Simple implementation
Disadvantages: Does not consider the processing power of each server
2. WRR weighted scheduling algorithm. We assign weight to each server, and the load-balancing scheduler schedules the server according to the weight. The number of times the server is called is proportional to the weight.
Advantages: Different server processing capacity is taken into account
3. Sh original address hash: extract the user IP, obtain a key according to the hash function, and check the corresponding value according to the static mapping table, that is, the IP address of the target server. If the target machine is overloaded, it returns null.
4. Dh target address hash: Same as above, except that the TARGET IP address is extracted for hashing.
Advantages: Both algorithms can enable the same user to access the same server.
5, LC least connection. Priority is given to forwarding requests to servers with fewer connections.
Advantages: The load is more evenly distributed among servers in the cluster.
6, WLC weighted least connection. Add weights to each server based on LC. The algorithm is :(number of active connections *256+ number of inactive connections) ÷ weight, the server with the smallest calculated value will be selected first.
Advantages: Requests can be allocated based on server capacity.
7, sed minimum expected delay. Sed is similar to WLC except that the number of inactive connections is not considered. The algorithm is :(active connection number +1)*256÷ weight, the server with the same calculated value is preferentially selected.
8. Nq never waits in line. Improved SED algorithm. If the number of connections to a server is 0, the equalizer will forward the request directly to it, without going through sed calculation.
9. LBLC is based on local minimum connections. According to the destination IP address of the request, the equalizer finds out the server whose IP address is used recently and forwards the request to the server. If the server is overloaded, the least number of connections algorithm is adopted.
LBLCR minimum local based connections with replication. According to the destination IP address of the request, the equalizer finds out the server group recently used by the IP address, which is not a specific server. Then, the equalizer uses the minimum number of connections to select a specific server from the group and forwards the request to the server. If the server is overloaded, then according to the least number of connections algorithm, in the cluster of servers not in the local server group, find a server to join the local server group, and then forward the request to it.
3. The third problem is the cluster mode problem, and there are generally three solutions:
1. NAT: The load balancer receives a user’s request and forwards it to a specific server. After the server processes the request, the load balancer returns the request to the user.
2. DR: The load balancer receives the user’s request and forwards it to a specific server. After the server plays the request, it directly returns it to the user. The system needs to support IP Tunneling, which is difficult to cross-platform.
3. TUN: Same as above, but without IP Tunneling protocol, which is cross-platform and can be supported by most systems.
4. The fourth problem is the session problem. Generally, there are 4 solutions:
1, Sticky Session. Session sticky means to allocate the requests of the same user in a certain session to a fixed server, so that we do not need to solve the problem of cross-server session. Common algorithms include IP_hash, namely the two hash algorithms mentioned above.
Advantages: Simple implementation.
Disadvantages: The session disappears after the application server restarts.
Session Replication. Session replication replicates sessions in a cluster so that each server holds session data for all users.
Advantages: Reduces the load balancing server’s stress and does not need to implement the IP_hasP algorithm to forward requests.
Disadvantages: High bandwidth overhead during replication. If the number of visits is large, the session occupies large memory and is wasteful.
3. Centralized storage of Session data: Centralized storage of Session data uses databases to store Session data, which decouples the Session from the application server.
Advantages: Much less bandwidth and memory stress between clusters compared to Session Replication.
Disadvantages: The database that stores sessions needs to be maintained.
Cookie Base: The Cookie Base is to store the session in the Cookie, and the browser can tell the application server what my session is, which also realizes the decoupling of session and application server.
Advantages: Simple implementation, basic maintenance free.
Disadvantages: Cookie length limitation, low security, broadband consumption.
It is worth mentioning that:
Nginx currently supports load balancing algorithms such as WRR, SH (which supports consistent hashing), and FAIR (which I think boils down to LC). But nginx as an equalizer can also be used as a static resource server.
Keepalived + IPVSADm is powerful and supports rr, WRR, LC, WLC, LBLC, SH, and DH algorithms
Keepalived supports NAT, DR and TUN cluster modes
Nginx itself does not provide a solution for session synchronization, whereas Apache provides session sharing support.
We recommend a Java advanced Technology group: 619881427, where all the architectural techniques used in this article can be shared and downloaded for free. If you’re interested in learning, you can add it.
Ok, after solving the above problems, the structure of the system is as follows:
Stage 4: Database read and write separation
Above we always assume that the database load is normal, but as the number of visits increases, so does the database load. Then someone may immediately think of the same as the application server, a database in two load balancing. But for databases, it’s not that simple. If we simply split the database in two, and then load the requests to machine A and machine B respectively, it will obviously cause the data inconsistency between the two databases. In this case, we can first consider using read/write separation.
The database system structure after read/write separation is as follows:
This structural change also brings two problems:
Data synchronization between primary and secondary databases
Application selection of data sources
Solution:
We can use MYSQL master+slave to implement master/slave replication.
Use third-party database middleware, such as MyCAT. Mycat grew out of Cobar, alibaba’s open-source database middleware, which was later discontinued. Mycat is a relatively good mysql open source database sub-database sub-table middleware in China.
Stage 5: Use search engines to relieve library reading pressure
If the database does read library, it is often powerless to find the fuzzy, even if it does read and write separation, this problem has not been solved. Taking the transaction website as an example, the published goods are stored in the database, and the most commonly used function of users is to find the goods, especially to find the corresponding goods according to the title of the goods. For such requirements, we generally use the like function to achieve, but this way is very expensive. At this point we can use the search engine’s inverted index to complete.
Search engines have the following advantages:
It can greatly improve query speed.
The introduction of a search engine also brings the following overhead:
With a lot of maintenance work, we needed to implement the index build process ourselves and design full/incremental build methods to handle non-real-time and real-time query requirements.
Search engine clusters need to be maintained
Search engine can not replace database, it solves the “read” problem in some scenarios, whether to introduce search engine, need to consider the needs of the whole system. The system structure after the introduction of search engine is as follows:
Stage 6: Use caching to relieve library read stress
1. Cache of background application layer and database layer
As the number of visits increases, there is a tendency for many users to access the same part of the content, and for these popular content, there is no need to read from the database every time. We can use caching technology, such as Google’s open source caching technology Guava or Memcacahe for the application layer cache, or Redis for the database layer cache.
In addition, in some cases, a relational database is not very suitable. For example, I want to create a “limit on the number of incorrect password entries per day” function. The idea is that when a user logs in, if the login is incorrect, the IP address of the user and the number of incorrect password entries are recorded. If placed in memory, then obviously will take up too much content; If it is placed in a relational database, it is necessary to establish database tables, resume corresponding Java beans, and write SQL and so on. The data we want to store is nothing more than key:value data like {IP :errorNumber}. For this kind of data, we can use NOSQL database instead of traditional relational database.
Page caching
In addition to data caching, there is page caching. Such as localstroage or cookies in HTML5.
Advantages:
Reduce stress on your database
Greatly improve access speed
Disadvantages:
A cache server needs to be maintained
Increased coding complexity
It is worth mentioning that:
The scheduling algorithm for the cache cluster is different from the application server and database mentioned above. It is best to use a “consistent hashing algorithm” to improve the hit ratio. I won’t expand on this, but if you are interested, you can refer to relevant materials.
Structure after adding cache:
Stage 7: database horizontal split and vertical split
Our site has evolved so far that transactions, products, and user data are all in the same database. Despite the adoption of increased cache, read and write separation, but as the pressure of the database continues to increase, the bottleneck of the database is more and more prominent, at this time, we can have data vertical split and horizontal split two options.
We recommend a Java advanced Technology group: 619881427, where all the architectural techniques used in this article can be shared and downloaded for free. If you’re interested in learning, you can add it.
7.1 Data vertical split
Vertical split means to split the different business data in the database into different databases. In this example, the data of transactions, goods, and users is separated.
Advantages:
The stress of having all your business in a single database is resolved.
More optimization can be made based on the characteristics of the business
Disadvantages:
Multiple databases need to be maintained
Question:
Original cross-business transactions need to be considered
Joins across databases
Solution:
We should try to avoid cross-database things at the application layer, and if we must, try to control them in the code.
We can solve this problem through third-party applications, such as mycat mentioned above. Mycat provides a rich cross-library join scheme. Please refer to the official documentation of Mycat for details.
The vertical split structure is as follows:
7.2. Horizontal split of data
Horizontal data splitting is the splitting of data from the same table into two or more databases. Horizontal data splitting occurs when the volume of data or updates for a business reaches the bottleneck of a single database, and the table can be split into two or more databases.
Advantages:
If we can overcome these problems, then we will have a good handle on the data volume and write volume growth.
Question:
Applications that access user information need to solve the problem of SQL routing because the user information is now split between two databases and they need to know where the data to be manipulated is.
Primary keys are also handled differently, for example from increment fields, and can no longer simply be used.
If you need paging, you’re in trouble.
Solution:
We can also address third-party middleware, such as MyCat. Mycat parses our SQL through the SQL parsing module and forwards requests to a specific database based on our configuration.
This can be solved with UUID guaranteed unique or custom ID schemes.
Mycat also provides rich paging query schemes, such as paging queries from each database, merging data for a paging query, and so on.
Data level split structure:
Phase 8: Application split
8.1. Split applications
As businesses grow, there are more businesses and more applications. We need to think about how to avoid making apps bloated. This requires breaking down applications from one to two or more. Using our example above, we can separate users, goods, and transactions. Into “user, goods” and “user, transaction” two subsystems.
Split structure:
Question:
After this split, there may be some identical code, such as user-related code, goods and transactions require user information, so the code that operates on user information remains similar in both systems. How to ensure that this code is reusable is a problem that needs to be solved.
Solve a problem:
By taking the route of service to solve
8.2. Take the road of service
In order to solve the problems caused by the above split application, we split the common services into a service-oriented pattern, referred to as SOA.
System structure after servitization:
Advantages:
Instead of having the same code scattered across different applications, these implementations are placed in service centers, making the code easier to maintain.
We put the interaction with the database in the service center, making the “front-end” Web application more focused on the interaction with the browser.
We recommend a Java advanced Technology group: 619881427, where all the architectural techniques used in this article can be shared and downloaded for free. If you’re interested in learning, you can add it.
Question:
How do I make a remote service invocation
Solutions:
We can solve this problem by introducing message middleware below
Phase 9: Introduction of message-oriented middleware
As the site continues to grow, there may be sub-modules developed in different languages and sub-systems deployed on different platforms. We need a platform to deliver reliable, platform – and language-independent data, to make load balancing transparent, to collect call data during calls and analyze it, to predict the site’s growth rate and so on, and to make predictions about how the site should grow. The open source messaging middleware includes Ali’s Dubbo, which can be combined with Google’s open source distributed program coordination service ZooKeeper to realize server registration and discovery.
Structure with message-oriented middleware:
Ten,
The above evolution process is only an example, which is not suitable for all websites. In practice, the evolution process of websites is closely related to their own business and different problems, and there is no fixed model. Only by careful analysis and continuous exploration can we find a suitable architecture for our website.