How do I start the ZooKeeper Server

We use zkServer.sh start to start the ZK server. Looking at the code of this script, we find that the main method of QuorumPeerMain is actually called

We explain the working principle of ZK cluster mode, first look at the picture below is the overall architecture of the server startup process components working principle diagram, then I will roughly explain the major core components

Core components that run at startup time

Above we can see the component diagram in three colors

  • Yellow: monitor port 2181, probably should know this port is responsible for communication with the client, around this piece of the core component isNioServerCnxnFactory
  • Blue: port 3888 is monitored. This series of components are mainly used for cluster election. The core component around this part is responsible for communicationQuirumCnxManagerIn charge of electionsFastLeaderElection
  • Purple: port 2888 is monitored. This series of components are mainly used for data exchange between the leader and followers after the role of each server is confirmed after the election. The core component of this component is the leader (follower) component

At the same time, we can see that this series of components are contained in the central component of QuorumPeer, which can be said to represent a ZK server. It can be imagined that each major component is each organ of a person, thus forming the complete person of QuorumPeer

In the figure above, the core components of the Leader are completely drawn. The components of followers are the same as those of the Leader. The difference is that the followers create not the Leader component but the Follower component after the election

ZKDatabase represents an in-memory database. All data written to the ZK is recorded in the memory, and the disk data is restored to the memory after a restart

Overall process combing

Start the ZK server and create a NioServerCnxnFactory for network communication with the client

Create a ZKDatabase by using FileTxnSnapLog to load the disk snapshot file + proposal log file to recover an in-memory database

Create a QuirumCnxManager to transfer voting information between clusters. Each node sends the zxID, epoch, and myID values of the current machine to each other

Create a FastLeaderElection to read and archive the voting data of each node and finally decide who will be the Leader, Follower, and Observer

Finally, Leader, Follower, and Observer are created according to the role of the current node. Another port is bound for data synchronization between clusters, and a thread is created for each connection. In order to avoid repeated connection binding creation, Only a server with a large ID is allowed to initiate a connection to a server with a small ID. If a small ID is sent to a large one, the connection is forcibly closed

thinking

From the zK server startup, we can see that for network call processing, in which case is suitable for BIO, in which case is suitable for NIO, and different port system, logic independent, easy to expand, maintenance, and higher fault tolerance

When communicating with clients, there may be a large number of client connections and high concurrent read requests. By default, the maximum number of connected clients can be adjusted to 60. If the BIO is used, one thread needs to be created for each client, which will occupy thread resources even if the client does not request

Communication between clusters is handled using BIO, which creates a thread for each connection between clients, since there are relatively few cluster nodes

Port 3888 is used for communication election between clusters, and port 2888 is used for information synchronization between the leader and followers. The two are isolated from each other even if one party has errors, the other party is not affected