1, an overview of the

Taking Taobao as an example, this paper introduces the evolution process of server architecture from one hundred concurrency to ten million concurrency, and lists relevant technologies encountered in each evolution stage, so that we can have an overall understanding of the evolution of architecture. At last, the paper summarizes some principles of architecture design.

2. Basic concepts

Before introducing architecture, in order to avoid confusion among some of the readers, I will introduce some of the most basic concepts in architecture design. 1) What is distributed? If multiple modules in the system are deployed on different servers, it can be called a distributed system. For example, Tomcat and database are deployed on different servers, or two Tomcats with the same function are deployed on different servers. 2) What is high availability? If some nodes in the system fail and other nodes continue to provide services, the system is considered highly available. 3) What is a cluster? Software in a particular domain is deployed on multiple servers and provides a class of services as a whole, called a cluster. For example, the Master and Slave of Zookeeper are deployed on multiple servers to provide centralized configuration services. In a common cluster, clients can connect to any node to obtain services. When a node in the cluster goes offline, other nodes can automatically take over its services, indicating that the cluster has high availability. 4) What is load balancing? When a request is sent to the system, the request is evenly distributed to multiple nodes in some way so that each node can handle the request load evenly. The system is considered to be load balanced. 5) What are forward and reverse proxies? When the system needs to access the external network, it forwards the request through a proxy server. In the view of the external network, the access is initiated by the proxy server. In this case, the proxy server implements forward proxy. When an external request enters the system, the proxy server forwards the request to a server in the system. For the external request, only the proxy server interacts with it. At this time, the proxy server implements reverse proxy. In simple terms, forward proxy refers to the process that the proxy server accesses the external network instead of the internal system, and reverse proxy refers to the process that the external request to access the system is forwarded to the internal server through the proxy server.

3. Architecture evolution

3.1 Single-node Architecture

Take Taobao as an example: At the beginning of the website, the number of applications and users were small. Tomcat and database could be deployed on the same server. When a browser sends a request to www.taobao.com, the DNS server converts the domain name to an actual IP address, and the browser accesses Tomcat with the IP address. Architecture bottleneck: As the number of users increases, Tomcat and database compete for resources, and the stand-alone performance is insufficient to support services. 3.2 First Evolution: Tomcat and database are deployed separately

Tomcat and database monopolize server resources respectively, significantly improving their respective performance. Architectural bottlenecks: As the number of users grows, concurrent reading and writing to the database becomes a bottleneck. Tips: Welcome to follow wechat official account: Java backend, get more technical blog posts push. 3.3 Evolution 2: Local cache and distributed cache are introduced

Add local cache on the same Tomcat server or in the same JVM, and add distributed cache externally to cache popular product information or HTML pages of popular products. Caching can intercept most requests before they are read or written to the database, greatly reducing the database pressure. The techniques involved include memcached as a local cache, Redis as a distributed cache, cache consistency, cache penetration/breakdown, cache avalanche, hot data set failures, etc. Architecture bottleneck: The cache can resist most of the access requests. As the number of users increases, the concurrency pressure mainly falls on the stand-alone Tomcat, and the response gradually slows down. 3.4 Third Evolution: Reverse proxy is introduced to implement load balancing

Tomcat is deployed on multiple servers and requests are evenly distributed to each Tomcat using reverse proxy software (Nginx). The assumption here is that Tomcat supports up to 100 concurrent requests and Nginx supports up to 50,000 concurrent requests, so in theory Nginx can handle 50,000 concurrent requests by distributing requests to 500 Tomcats. The technologies involved include: Nginx and HAProxy, both of which are reverse proxy software working in the seventh layer of the network. They mainly support HTTP protocol, and also involve session sharing and file uploading and downloading. Architectural bottlenecks: Reverse proxies greatly increase the amount of concurrency that the application server can support, but the increase in concurrency also means that more requests penetrate the database, and stand-alone databases eventually become bottlenecks. 3.5 Fourth Evolution: Read and write separation of databases

A database can be divided into read libraries and write libraries. Multiple read libraries are synchronized to the read library through the synchronization mechanism. If the latest written data needs to be queried, you can write a copy to the cache to obtain the latest data from the cache. The technologies involved include: Mycat, which is database middleware, which can be used to organize the separate read and write of the database and the sub-database sub-table, and the client can access the lower database through it. It also involves data synchronization and data consistency. Architecture bottleneck: As the number of services increases, there is a large gap between the visits of different services. Different services directly compete for databases and affect each other’s performance. 3.6 Evolution 5: The database is separated by service

Data of different services is stored in different databases to reduce resource competition among services. For services with heavy traffic, more servers can be deployed to support them. As a result, cross-business tables cannot be directly associated with analysis, which needs to be solved by other means. However, this is not the focus of this paper. Interested parties can search for their own solutions. Architecture bottlenecks: As the number of users increases, stand-alone write libraries will gradually reach performance bottlenecks. 3.7 Sixth Evolution: Split a large table into a small table

For example, comment data can be hashed according to product IDS and routed to corresponding tables for storage. For payment records, tables can be created on an hourly basis, and each hourly table continues to be split into smaller tables, using user ids or record numbers to route the data. As long as the amount of real-time table data is small enough and requests are distributed evenly across small tables on multiple servers, the database can improve performance through horizontal scaling. Mycat, mentioned earlier, also supports access control when large tables are split into smaller tables. This approach significantly increases the difficulty of database operation and maintenance, and has higher requirements for DBAs. When the database is designed to this structure, it can already be called a distributed database, but it is only a logical database as a whole, the different components of the database are independently implemented by different components, such as the management of the sub-database and sub-table and request distribution, implemented by Mycat, SQL parsing is implemented by the standalone database, Read and write separation may be implemented by gateways and message queues, query results may be summarized by the database interface layer, etc. This architecture is actually a kind of IMPLEMENTATION of MPP (massively parallel processing) architecture. At present, there are a lot of MPP databases in both open source and commercial, among which Greenplum, TiDB, Postgresql XC and HAWQ are popular. Commercial MPP databases, such as NTNU GBase, Ruifan Snowball DB and Huawei LibrA, have different emphases. For example, TiDB is more focused on distributed OLTP scenarios, Greenplum is more focused on distributed OLAP scenarios. These MPP databases basically provide SQL standard support capabilities like Postgresql, Oracle, and MySQL. Can make a query parsing for the implementation of a distributed parallel execution plan to distribute to each machine, the final summary data returned by the database itself provides such as rights management, depots table, transaction, a copy of the data, such as ability, and most can support more than 100 nodes of the cluster, greatly reduce the cost of database operations, And make the database can also achieve horizontal expansion. Architectural bottlenecks: Both the database and Tomcat can scale horizontally, and the concurrency that can be supported increases dramatically. As users grow, stand-alone Nginx will eventually become a bottleneck. 3.8 Evolution 7: Load balancing of multiple Nginx using LVS or F5

Because the bottleneck is in Nginx, multiple Nginx load balancing cannot be achieved through two layers of Nginx. Figure of the LVS and F5 is working in the fourth layer network load balancing solutions, including the LVS is a software that runs on the operating system kernel mode, to TCP forwarding requests or higher level of network protocols, so support agreement is richer, and the performance is much higher than Nginx, can assume that single LVS can support hundreds of thousands of concurrent forward requests; The F5 is a load balancing hardware that is similar to the capabilities provided by LVS, with higher performance than LVS, but at an expensive price. LVS is standalone software. If the LVS server is down, the entire backend system cannot be accessed. Therefore, a standby node is required. Keepalived can be used to simulate a virtual IP address and bind the virtual IP address to multiple LVS servers. When the browser accesses the virtual IP address, the router redirects it to the real LVS server. When the main LVS server is down, Keepalived automatically updates the routing table in the router. Redirect virtual IP addresses to another normal LVS server to achieve high availability of the LVS server. It is important to note that the drawing from Nginx layer to Tomcat layer does not mean that all Nginx will forward requests to all Tomcat. In actual use, it may be several Nginx followed by a part of Tomcat. Keepalived is implemented between these Nginx layer and high availability. Other Nginx are connected to other Tomcat, so the number of Tomcat connections can be multiplied. Architecture bottleneck: Since LVS is also a standalone server, when the number of concurrent users increases to hundreds of thousands, THE LVS server will eventually reach the bottleneck. At this time, the number of users reaches tens of millions or even hundreds of millions. The users are distributed in different regions and the distance from the server room is different, resulting in significantly different access delays. 3.9 Evolution Eight: DNS polling is used to implement load balancing between equipment rooms

On the DNS server, you can configure a domain name to correspond to multiple IP addresses. Each IP address corresponds to a virtual IP address in a different equipment room. When a user accesses www.taobao.com, the DNS server uses polling policies or other policies to select an IP address for the user to access. This way can achieve load balancing between machine rooms at this point, the system can achieve the level of machine room expansion, tens of millions to hundreds of millions of levels of concurrency can be solved by increasing the machine room, the system entrance request concurrency is no longer a problem. Architecture bottleneck: With the development of data richness and business, the demand for retrieval and analysis is becoming more and more abundant. Relying on database alone cannot solve such rich demand. 3.10 Evolution 9: NoSQL database and search engine technologies are introduced

When the amount of data in the database reaches a certain size, the database is not suitable for complex queries, and can only meet the scenarios of common queries. For statistical report scenarios, results may not be generated when there is a large amount of data, and other queries may be slowed down when complex queries are run. For full-text search and variable data structure scenarios, databases are inherently unsuitable. Therefore, appropriate solutions need to be introduced for specific scenarios. For massive file storage, you can use HDFS. For key value data, you can use HBase and Redis. For full-text search, you can use search engines such as ElasticSearch. Solutions such as Kylin or Druid are available. Of course, the introduction of more components will also increase the complexity of the system, the data stored by different components need to be synchronized, the problem of consistency needs to be considered, and more means of operation and maintenance need to manage these components. Architecture bottleneck: The introduction of more components to solve the rich requirements, business dimensions can be greatly expanded, resulting in an application containing too much business code, business upgrade iterations become difficult. 3.11 Evolution 10: A large application is split into a small application

Application code is divided by service block, so that the responsibilities of each application are clearer and each application can be upgraded independently. In this case, some common configurations may be involved between applications. You can use the Zookeeper distributed configuration center to solve this problem. Architecture bottleneck: Multiple copies of the same code exist when modules are shared between different applications. As a result, all application codes must be upgraded when common functions are upgraded. 3.12 Evolution 11: Reuse functions are separated into microservices

If user management, order, payment, authentication and other functions exist in multiple applications, the codes of these functions can be extracted separately to form a single service to manage such services, which is the so-called micro service. Common services are accessed between applications and services through HTTP, TCP or RPC requests and other ways. Each individual service can be managed by a separate team. In addition, Dubbo, SpringCloud and other frameworks can realize service governance, traffic limiting, circuit breaker, downgrade and other functions to improve the stability and availability of services. Architecture bottlenecks: Different services have different interface access modes, and application code needs to adapt to multiple access modes to use the services. In addition, applications can access services and services may also access each other, so the call chain becomes very complex and the logic becomes confused. 3.13 Evolution 12: The ESB, an enterprise service bus, is introduced to mask service interface access differences

The ESB implements unified access protocol transformation. Applications access back-end services through ESB, and services invoke each other through ESB to reduce the coupling degree of the system. The so-called SOA (Service Oriented) architecture, in which a single application is split into multiple applications, common services are isolated to manage, and the enterprise message bus is used to decouple services from each other, is easily confused with microservices architecture because of its similar presentation. Personally, microservice architecture refers to the idea of extracting public services from the system for independent operation and maintenance management, while SOA architecture refers to an architectural idea of splitting services and unifying service interface access. SOA architecture contains the idea of microservices. Architectural bottlenecks: Business development, applications and services is likely to become steadily more, application and service deployment is complicated, the same server deployment of multiple services to solve the problem of running environment conflict In addition, for that need dynamic scalability, such as big promote capacity of scenes, the performance of the need for extending the service needs to be ready to run on the new service environment, deployment of services, etc., Operations will become very difficult. 3.14 Evolution: Container technology was introduced to implement operating environment isolation and dynamic service management

At present, the most popular containerization technology is Docker, and the most popular container management service is Kubernetes(K8S). Applications/services can be packaged as Docker images, which can be dynamically distributed and deployed through K8S. Docker image can be understood as a minimum operating system that can run your application/service, which puts the application/service running code, and the running environment is set up according to the actual needs. After the whole “operating system” is packaged as an image, it can be distributed to the machines where relevant services need to be deployed. Directly starting the Docker image can get the service up, making the deployment, operation and maintenance of the service simple. Before the promotion, the existing machine cluster can be divided into servers to start the Docker image. After the promotion of service performance, the image can be shut down without affecting other services on the machine. (Before Section 18, the system configuration of the service running on the newly added machine needs to be modified to fit the service. This results in the destruction of the running environment required by other services on the machine). Architecture bottleneck: After the use of containerization technology, the problem of service dynamic expansion and shrinkage can be solved, but the company still needs to manage the machine itself. In non-large-scale promotion, a large number of idle machine resources are still needed to deal with the large-scale promotion, and the machine cost and operation and maintenance cost are very high, and the resource utilization rate is low. 3.15 Evolution: Use the cloud platform to host the system

System can be deployed to the public cloud, the use of public cloud mass machine resources, solve the problem of dynamic hardware resources In promoting the time periods, temporary application more resources in the cloud platform, combining with the Docker and K8S to rapid deployment services, at the end of the great promoting release resources, truly on-demand, resource utilization is greatly increased, Meanwhile, the operation and maintenance costs are greatly reduced. So-called cloud platform, is the huge machine resources, through a unified resource management, abstracted as a resource as a whole Dynamic application on the cloud platform can be on-demand hardware resources (such as CPU, memory, network, etc.), and provides a common operating system above, provide the commonly used technology components (such as Hadoop technology stack, MPP databases, etc.) for the use of the user, Users of developed applications can even solve their needs (such as audio and video transcoding services, email services, personal blogs, etc.) without caring what technology is used inside the application. The following concepts are involved in cloud platforms:

  1. IaaS: Infrastructure as a service. Corresponding to the above mentioned machine resources unified into a resource whole, can dynamically apply for hardware resources level;

  2. PaaS: Platform as a service. Provide common technical components to facilitate the development and maintenance of the system as mentioned above;

  3. SaaS: Software as a service. To provide developed applications or services as described above, pay for functionality or performance requirements.

So far: there are solutions to all of the problems mentioned above, from high concurrent access issues to service architecture and system implementation. At the same time, it should be realized that the above introduction has intentionally omitted practical issues such as cross-machine room data synchronization, distributed transaction implementation, and so on, which will be discussed separately at a later time.

4. Summary of architecture design

1) Does the adjustment of the architecture have to follow the above evolution path? No, the architecture evolution sequence mentioned above is only a separate improvement for a certain side. In actual scenarios, there may be several problems to be solved at the same time, or another aspect may reach the bottleneck first. In this case, it should be solved according to the actual problems. For example, in the scenario where the amount of concurrency in the government class may be small but the business may be very rich, high concurrency is not the key problem to be solved. In this case, the solution to enrich the demand may be the priority. 2) How far should the architecture be designed for the system to be implemented? For single-implementation systems with well-defined performance metrics, it is sufficient that the architecture is designed to support the performance metrics of the system, but with interfaces to extend the architecture in case it is needed. Evolving systems, such as e-commerce platforms, should be designed to meet the requirements of the next phase of user volume and performance indicators, and the architecture should be upgraded iteratively based on the growth of the business to support higher concurrency and richer business. 3) What is the difference between server architecture and big data architecture? The so-called “big data” is a general term for solutions in massive data collection, cleaning, conversion, data storage, data analysis, and data service scenarios. Each scenario contains various optional technologies, such as Flume, Sqoop, and Kettle for data collection, and HDFS and FastDFS for data storage. NoSQL databases, such as HBase and MongoDB. Data analysis includes Spark technology stack and machine learning algorithm. In general, big data architecture is an architecture that integrates various big data components based on business requirements and generally provides distributed storage, distributed computing, multidimensional analysis, data warehouse, machine learning algorithm and other capabilities. On the other hand, server-side architecture refers more to the architecture at the application organization level, and the underlying capabilities are often provided by big data architecture. 4) Are there any architectural design principles?

  • N+1 design: every component in the system should be free of single points of failure;

  • Rollback design: To ensure that the system is forward compatible, there should be a way to roll back the version when the system is upgraded;

  • Disable design: a configuration should be provided to control whether specific functions are available and to quickly bring the function offline in the event of a system failure;

  • Monitoring design: in the design stage to consider the means of monitoring;

  • Multi-active data center design: If the system needs high availability, it should be considered to implement multi-active data center in more than one place. The system can still be used when at least one machine room is powered off.

  • Adopt mature technologies: New or open source technologies often have many hidden bugs, and failure without commercial support can be a disaster.

  • Resource isolation design: to avoid a single business to occupy all resources;

  • Architecture should be able to expand horizontally: only when the system can expand horizontally can bottleneck problems be effectively avoided;

  • Non-core purchase: If non-core functions need to occupy a large amount of R&D resources to solve, consider purchasing mature products;

  • Use of commercial hardware: commercial hardware can effectively reduce the probability of hardware failure;

  • Rapid iteration: the system should develop small functional modules quickly and go online for verification as soon as possible, so as to find problems early and greatly reduce the risk of system delivery;

  • Stateless design: The service interface should be stateless, so that access to the current interface does not depend on the state of the last access to the interface.

Author: sky _2cd3 links: www.jianshu.com/p/f4a907fe1… The copyright of the book belongs to the author. Commercial reprint please contact the author for authorization, non-commercial reprint please indicate the source.