This is the 16th day of my participation in the First Challenge 2022

The basic concept

• Zookeeper can act as a registry and distributed lock
• Zookeeper is a member of the Hadoop system
• An odd number of servers are used to build a Zookeeper cluster
• Primitives:
  • Terms for operating systems and computer networks
  • A process consisting of several instructions used to perform a function
  • Primitives are indivisible. That is, the execution of primitives must be continuous and cannot be interrupted during execution
• Zookeeper is an Apache open source distributed coordination service framework:
  • Complex and error-prone distributed consistent services are encapsulated into an efficient set of primitives that are provided to callers in a series of easy-to-use interfaces
  • Mainly provides coordination services for distributed applications
• 
  $Zookeeper = File system + notification mechanism$
  • Zookeeper is used to maintain configuration information, name, and provide centralized services for distributed synchronization and group services that distributed applications can use
  • Zookeeper stores and manages the configuration data required for these services. If the status of the configuration data changes,Zookeeper notifies the services registered with Zookeeper to respond in a timely manner
• ZookeeperProvides high availability, high performance, stable distributed data consistency solutions, usually used in the following aspects:
  • Publish and subscribe to data
  • Unified Naming Service
  • Distributed notification and coordination
  • Cluster management
  • Master the election
  • Load balancing
  • A distributed lock
  • Distributed queue
• Zookeeper stores data in memory, providing high performance. Applications with read operations over write operations perform better. Read operations over write operations are a typical scenario for coordinating services, because write operations result in synchronization between all servers
• ZookeeperContains the following advantages:
  • Simple data structure:
    • The data model in the memory of the Zookeeper server consists of a series of ZNodes
    • Zookeeper enables distributed applications to coordinate with each other through a shared tree-structured namespace
    • Zookeeper stores all data in memory, which improves server throughput and reduces latency
    • Due to memory limitations, keep the amount of data stored by ZNodes in Zookeeper as small as possible
  • Building a cluster:
    • A Zookeeper cluster usually consists of an odd number of machines
    • Each machine in the Cluster in Zookeeper maintains the current server state in memory, and each machine can communicate with each other
  • Sequential access:
    • Zookeeper assigns an increasing global unique number to each update request from the client
    • Zookeeper identifies the sequence of all transaction operations based on this id
  • High performance:
    • Similar to Redis, Zookeeper stores all data in memory
    • Zookeeper provides high throughput and low latency

Zookeeper characteristics

• The atomic broadcast protocol in Zookeeper is a consistency protocol
• Zookeeper is an efficient, scalable service
• Both read and write operations in Zookeeper are fast. The read operation is faster than the write operation

Sequential Consistency

• Update requests from a client are executed sequentially
• Transaction requests from the same client are eventually applied to Zookeeper in strict order
• Sequentiality includes the following two types:
  • Global ordering: If a message on one server is published before another message, then that message on all servers is published before another message
  • Partial ordering: If another message is published by the same publisher after one message, it must precede another message

Atomic Atomicity

• The results of all transaction requests are applied consistently across all machines in the cluster
• That is, all the machines in the cluster either successfully applied a transaction or did not apply a transaction. There are no intermediate states
• Updates either succeed or fail, and there is no partial success

Single System Image

• No matter which Server the client connects to, the obtained system image is the same
• No matter which Zookeeper server the client is connected to, the data obtained from the Zookeeper server is in the same tree structure
• Single system mirroring is guaranteed through final consistency
• inZABIn the agreement:
  • The write operation does not guarantee that updates will be confirmed by all followers immediately
  • Some followers and the Leader can read the latest data, so the latest data can not be read by the rest of the followers
  • At this point, you can use the sync synchronization method before a read operation

Reliability, Reliability

• If the update is valid, it continues to be valid until the update is overwritten
• Once a change request is applied, the result of the change is persisted until overwritten by the next change
• An update operation accepted is not accidentally lost unless overwritten by other update operations

Real-time Timeliness

• Ensure that the system information displayed by each client is consistent within a certain period of time
• Zookeeper ensures that the client can obtain server update information or server failure information within a specified interval
• Due to network latency,Zookeeper cannot ensure that both clients receive immediately updated data. If you need the latest data, you can use the sync() interface before reading the data

Waiting has nothing to do with wait-free

• Slow failing clients must not interfere with fast client requests, allowing each client to wait efficiently

Zookeeper application scenarios

• Zookeeper is used to store data. However, Zookeeper is not suitable for storing large amounts of data, because Zookeeper stores all data in the memory

Data publishing and subscription

• Configuration file management and configuration file synchronization in distributed environment:
  • In a cluster, the configuration information of all nodes is consistent. For example, Hadoop cluster, database configuration in the cluster and other global configuration information
  • The modified information in the configuration file must be quickly synchronized to each node
• Publish and subscribe model, also known as configuration center
  • Publishers publish data to the Zookeeper node, and subscribers can obtain data dynamically from the Zookeeper node. In this way, data information can be centrally managed and dynamically updated
• It can be used in distributed environmentsZookeeperImplement configuration management:
  • Write the configuration information to Znode of Zookeeper
  • Other nodes can listen on this node, Znode
  • If Znode configuration information is modified,Zookeeper notifies each listening node of the modified configuration information
• Zookeeper’s Watcher mechanism makes it easy to publish and subscribe data
• Publish data to the monitored Zookeeper node, and other machines can dynamically update the configuration by listening for the changes of Zookeeper nodes

Unified Naming Service

• Unified naming of services is often required in distributed environments:
  • When multiple copies of a service are deployed, specific services cannot be invoked directly because it is uncertain which service can handle requests with high performance
  • In this case, multiple services can be uniformly named, and then the internal load balancing call, so as to achieve the optimal call to the service. Such as Dubbo service
• The unified naming service is implemented as follows:
  • The publisher writes its own address list to the Zookeeper node
  • Subscribers can get a list of addresses from a node with a fixed name and link to the publisher for related communication
• Generate globally unique ids from Zookeeper sequence nodes

Distributed notification and coordination

• By using the Watcher registration and asynchronous notification mechanism of Zookeeper, the notification and coordination between different systems in the distributed environment is well realized, and the real-time processing of data is realized

Cluster management

• Cluster management includes information such as online rate and node online and offline notification
• In distributed environment, it is necessary to know the status of each node in real time:
  • Monitor the status of each machine in the cluster
  • Collect runtime state data for each machine in the cluster
  • Collect dynamic online and offline notifications of servers
• ZookeeperHow to achieve cluster management:
  • Write the node information to Znode in Zookeeper
  • Listen on this nodeZnodeGets the real-time state change of this node
    • Example: Master status monitoring and election in HBase

Master the election

• MasterElection application scenarios:
  • In a distributed environment, the same service applications are distributed on different machines in the cluster. Some service logic, such as time-consuming computing and network I/O processing, is used. Only one machine in the cluster is required to perform the execution, and the other machines share the execution results
  • This greatly reduces the number of repeated operations and improves system performance
• MasterElection implementation method:
  • By taking advantage of the strong consistency of Zookeeper, the nodes created in distributed and high-concurrency scenarios must be globally unique
  • When there are multiple client requests to create a currentMaster node, only one client request will succeed in creating the node
  • Zookeeper allows you to create a Master node based on its unique global features. Other clients use Wacther to monitor whether the Master node is alive
  • If the Master node goes down, the rest of the machines can continue to create a new Master node for a new election

Load balancing

• Software load balancers include:
  • Production of message-oriented middleware
  • Consumer load balancing

A distributed lock

• Through the use ofZookeeperThe use of temporary nodes to achieve distributed locking
  • Obtain distributed locks by creating unique nodes
  • The lock is released when the party that acquired the lock finishes executing the relevant code or after an outage
• Lock services are divided into:
  • Exclusive locks:
    • All clients acquire the lock, but ultimately only one client can acquire the lock
    • An exclusive lock uses a temporary node
  • Control timing lock:
    • All clients that acquire locks will acquire them sequentially
    • Controlling the timing lock uses a temporary sequential child node under a node

Distributed queue

• Distributed queues include the following two types:
  • FIFO:
    • Similar to control timing in distributed lock services
    • Implemented using temporary sequential nodes
  • Wait for the elements in the distributed queue to come together and execute in order:
    • First create a queue(num) node in the queue and copy the size of the queue
    • Check whether the queue is full or whether the condition meets the requirement of execution by monitoring the change of node points in the queue
    • This mode applies to conditional tasks. The task can be executed only when the conditions are met

Working principles of Zookeeper

• ZookeeperDesign patterns in:
  • Zookeeper is a distributed service management framework based on the observer pattern
  • Zookeeper stores and manages configuration data and important data of interest, and receives registrations from observers
  • If the status of the data in Zookeeper changes,Zookeeper is responsible for notifying the observers registered with Zookeeper to react, implementing the master-slave mode in the cluster
• ZookeeperThe core of the: Atomic broadcast
  • The Atomic Broadcast mechanism ensures synchronization between servers and is implemented using the Zookeeper Atomic Broadcast Protocol (ZAB)
• ZABProtocol:
  • Recovery mode: RecoveryChoose the main
    • When the server starts or the leader goes down,ZAB goes into recovery mode
    • The recovery mode ends when the leader leader is elected and most servers have synchronized with the leader leader status
    • State synchronization ensures that the leader and server have the same system state
  • Broadcast mode: Broadcast synchronization
• zxid:
  • Zookeeper uses the incrementing transaction ID zxID to identify transactions to ensure the consistency of transaction order. All proposals have a ZXID when they are proposed
  • zxidIs a64The number of bits:
    • The higher 32 bits are epoch, which is used to identify whether the relationship between leaders and leaders has changed. Each leader elected will have a new epoch, which identifies the current leader under the designated leader
    • The lower 32 bits are an increasing counter

The write operation

• leaderThe write operation includes the following five steps:
  • The client sends a write request to the leader
  • The leader sends the write request in the form of a proposal to all followers and waits for an ACK
  • The follower sends an ACK to the leader after receiving the proposal
  • The leader gets more than half of the ACKS, including its own default ACK, and sends a commit to all followers and observers
  • The leader returns the processing results to the client
• Note:
  • The Observer does not have voting rights, and the leader does not need to receive an ACK from the Observer
  • The leader only needs to obtain more than half of the ACKS, but does not need to obtain all the acks of the followers. It should be noted that the leader also contains an ACK
  • The Observer has no voting rights and synchronizes the leader’s data to ensure that it returns as new data as possible when processing read requests
• followerorobserverWrite operation of:
  • In a write operation by a follower or observer, both the follower and observer receive write requests but forward them to the leader for processing
  • The follower or Observer write operation has an extra step to request forwarding, and the rest of the process is the same as the leader write operation

A read operation

• Both the leader and follower and Observer can process read requests, reading data from local memory and sending it back to the client
• Servers do not need to interact with each other to process read requests. The number of followers and observers increases the number of read requests that can be processed and the read request performance improves

Session Session

• Session Session refers to the Session between the Zookeeper server and the client
• ZookeeperA client connection in is one between a client and a serverTCPLong connection:
  • When the client is started, a TCP connection is established with the server. When the first connection is established, the life cycle of the client session begins
  • In this TCP long connection, the client can maintain an effective session with the server through heartbeat detection, send requests to the Zookeeper server and receive responses, and also receive watch event notifications from the server
• Client sessionSessionAs session entities, these include the following4Properties:
  • SessionID: global unique id of a Session, used to identify a Session
  • TimeOut:Session Timeout event
    • When creating a Session instance, the client sets a Session timeout period
    • When a client is disconnected due to heavy server pressure, a network fault, or a client disconnection, the Session is still valid as long as any server in the cluster can be reconnected within the TimeOut period specified by the Session TimeOut
    • TickTime: indicates the time when the next session times out
    • IsClosing: The service end closes a Session after detecting that the Session timeout expires
• The clientClientandZookeeperWhen the cluster establishes a connection, the entire sessionSessionThe state change of is shown as follows:
  
  
• If the connection between the Client and Zookeeper Server is disconnected due to TimeOut, the Client is in the CONNECTING state (2 in the figure). The system automatically attempts to connect to the Zookeeper Server
• If a Session is successfully CONNECTED to any Server in the Zookeeper cluster during the Session validity period, the Client enters the CONNECTED state, which is 3 in the figure
• Note:
  • Client If the Client loses contact with the Zookeeper Server due to network failure, the Client stays in the current state and tries to actively connect to the Zookeeper Server
  • A Client cannot declare its Session Session expired. Session Session expired is determined by the Zookeeper Server. The Client can voluntarily disable the Session

Permission control ACL

• Zookeeper uses Access Control Lists (ACLs) to Control permissions. This is similar to permission control for UNIX file systems
• ZookeeperContains the following five permissions:
  • CREATE: permission to CREATE child nodes
  • READ: Permission to READ node data and child node list
  • WRITE: permission to update node data
  • DELETE: indicates the permission to DELETE a child node
  • ADMIN: Sets the ACL permission of a node
  • Note:
    • CREATE and DELETE permissions are permissions for child nodes

They are used

• Kafka:
  • Zookeeper provides Kafka with the registration of brokers and topics and load balancing of multiple partitions
• HBase:
  • Zookeeper ensures that there is only one Master in the HBase cluster
  • Saves and provides regionServer status information
• Hadoop:
  • Zookeeper provides high availability support for NameNode

Select a Zookeeper application scenario

• Application scenarios where Zookeeper plays a significant role:
  • Coarse-grained distributed locks
  • Distributed primary selection
  • The active/standby HA switchover is performed
  • Scenarios that do not require high TPS transaction count support
    • These requirements tend to focus on big data, offline tasks and other related business areas
    • In the field of big data, segmentation of data sets is emphasized, and most of the time, these data sets are processed in parallel by task, multi-process and multi-thread
    • You can use Zookeeper to coordinate these tasks and process threads at some point
• Zookeeper uses the following application scenarios that require strict assessment of scenarios, capacity, and SLA requirements:
  • Transaction link service for transaction scenarios
  • Master business data access
  • Large-scale service discovery and server registration
  • Mass health surveillance
    • Zookeeper should be avoided in these scenarios
• ZookeeperReasons for selecting an application scenario:
  • ZookeeperWrite operations are not extensible, and horizontal scalability cannot be solved by adding nodes
    • In the project practice, you can find ways to sort out services, divide service areas vertically, and divide services into multiple Zookeeper registries. However, business is not allowed to divide governance business according to technology
    • The technical limitations of the Zookeeper registry cannot damage the connectivity of service services or hinder the development of service services
  • Long connections in Zookeeper may be intermittently disconnected. After a flash interruption occurs, the mechanism such as whether service operations need to be retried also needs to be controlled by the services captured by exceptions