Set up ZooKeeper clusters on multiple Ali cloud servers

The foreword 0.

For the convenience of testing, the computer had better be able to run multiple virtual machines, or have 2-3 cloud servers (student machine 9.9 January, you can buy a lightweight and ECS, or borrow a classmate to use)

1. Dubbo vs. SpringCloud

2. Microservice registry

Introduction: What is a registry and what are the common ones?

• Registry: service management, the core is a service registry, heartbeat mechanism dynamic maintenance;
• Service provider: reports its network information to the registry during startup
• Service consumer: report your network information to the registry when starting, and pull the provider’s network information

Why use a registry?

There are more and more micro-service applications and machines, and the caller needs to know the network address of the interface. If the network address is controlled by configuration files, the maintenance of dynamically added machines will bring great problems!

Mainstream microservice registries

• Zookeeper (used with Dubbo)
• Eureka(used in conjunction with SpringCloud)
• Consul (used in conjunction with SpringCloud)
• Etcd (used in conjunction with SpringCloud)

3. Download and install Zookeeper

3.1 Installing Zookeeper on CentOS

Note: Installation of Zookeeper depends on the JDK environment! So make sure the server is configured with the JDK environment in the virtual machine first!

• To upload the zooKeeper package to the cloud server, I put all the packages in ** /usr/local/src/software/* *

  • Tar -zxvf zookeeper-3.4.14.tar.gz Decompress the ZooKeeper package

  • Mv zookeeper-3.4.14 ZooKeeper Change the folder name to Zookeeper to facilitate memory

  • In the mkdir zookeeper directory to create the data catalogue data, and the PWD gain the data directory path * * / usr/local/SRC/software/zookeeper/data * *

  • CD/usr/local/SRC/software/zookeeper/conf to enter the conf directory

    • CFG zoo_sample. CFG Changes the zoo_sample. CFG file name to zoo.cfg

    • Vim zoo. CFG Modify the configuration file. Wq Save the configuration and exit.

      

  • Useradd zooKeeper Creates a user named zookeeper

  • Chown -r zookeeper:zookeeper zookeeper/ Grant the zooKeeper folder permission to the newly added ZooKeeper user

    

• Then we can start the ZooKeeper operation

  • Zookeeper bin directory:

    

  • / zkserver. sh start Start ZooKeeper.

    

• To verify the startup success, let’s connect to the ZooKeeper service using the client:

  • Run the netstat -tlun command to check whether the port service is enabled.

    

  • It is recommended that you turn off the firewall so that you do not need to add rules to the firewall

  • Ali Cloud security group start port

    

  • Then we use the zooKeeper client to connect zooKeeper of the server on the local computer (zooKeeper can be used after downloading and decompressing zooKeeper under Windows)

    • .\ zkcli.cmd-timeout 5000-server x.xx.xx. XXX:2181 IP address is the IP address of the server

      

      

    • Zookeeper connection completed!

3.2 ZooKeeper Data Model

4. Common Zookeeper command zkCli

• Chown -r zookeeper:zookeeper zookeeper/ Grant the zooKeeper folder permission to the ZooKeeper user

• Su zookeeper Switch to the zooKeeper user

• Run the./ zkserver. sh start command to start zooKeeper and stop zookeeper. / zkserver. sh stop command

  • If the following error information is displayed, the zooKeeper permission fails to be assigned. Switch to user root and assign the zooKeeper folder permission to user root again:

    

• Then you can start successfully:

  

• Then we can learn zkCli command:

4.1 zkCli command

ZooKeeper -server host:port cmd args stat path [watch] set path data [version] ls path [watch] delquota [-n|-b] path ls2  path [watch] setAcl path acl setquota -n|-b val path history redo cmdno printwatches on|off delete path [version] sync path listquota path rmr path get path [watch] create [-s] [-e] path data acl addauth scheme auth quit getAcl path close connect host:portCopy the code

Create and view nodes

• Create [-s] [-e] path Creates an ACL node

  • -s: creates a sequential node

  • -e: creates a temporary node

  • Data: data content of the created node

    

• Look at the node

  • Get the root node: ls /

  • Obtain the child nodes of a node :ls/PATH, eg:ls/CSP0020000000001

  • Get data of a node: get /path

    [zk: Localhost :2181(CONNECTED) 5] GET/cSP0010000000000 CSP001 --> Node value cZxid = 0x5 --> Transaction ID ctime = Wed Nov 18 16:06:18 CST 2020 --> node creation time mZxid = 0x5 --> last update transaction ID mtime = Wed Nov 18 16:06:18 CST 2020 --> last update time pZxid = 0x5 --> The transaction ID of the child node list that was last modified cversion = 0 --> version of the child node list dataVersion = 0 --> Version of the data content aclVersion = 0 --> ACL version ephemeralOwner If the current node is not a temporary node, the value of this field is 0 dataLength = 6 --> Length of data content numChildren = 0 --> Number of child nodesCopy the code

  • View the status of a node: stat /path

    [zk: Localhost :2181(CONNECTED) 4] stat/cSP0020000000001 cZxid = 0x5 --> Ctime = Wed Nov 18 16:06:18 CST 2020 --> MZxid = 0x5 --> Last update transaction ID mtime = Wed Nov 18 16:06:18 CST 2020 --> last update time pZxid = 0x5 --> The transaction ID cversion = 0 --> version of the child node list dataVersion = 0 --> Version of the data content aclVersion = 0 --> ACL version ephemeralOwner = If the current node is not a temporary node, the value of this field is 0 dataLength = 6 --> Length of data content numChildren = 0 --> Number of child nodesCopy the code

  • Gets the status of a node’s children and the current node: ls2 /path

    [zk: Localhost: 2181 (CONNECTED) 6] the create - e/csp0010000000000 csp1999 my name is CSP # add temporary child node to the node Created /csp0010000000000/csp1999 [zk: [CSP1999] # cZxid = 0x4 ctime = Wed Nov 18  16:05:27 CST 2020 mZxid = 0x4 mtime = Wed Nov 18 16:05:27 CST 2020 pZxid = 0xc cversion = 1 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 6 numChildren = 1 [zk: localhost:2181(CONNECTED) 8]Copy the code

Modify the node

Remove nodes

• delete /path [version]Non-recursive deletion
• rmr /path [version]Recursive delete

[zk: localhost:2181(CONNECTED) 15] ls / [csp0010000000000, csp0040000000003, zookeeper, csp0020000000001, Csp0030000000002] [zk: localhost:2181(CONNECTED) 16] delete /csp0030000000002 Localhost :2181(CONNECTED) 17] ls / [cSP0010000000000, CSP0040000000003, Zookeeper, cSP0020000000001] [zk: localhost:2181(CONNECTED) 18] delete /csp0010000000000 /csp0010000000000 # This node has child nodes, delete failed! [zk: CONNECTED) 19] RMR /csp0010000000000 Localhost :2181(CONNECTED) 20] ls / [cSP0040000000003, Zookeeper, cSP0020000000001]Copy the code

5. Session mechanism and Watcher mechanism in Zookeeper

5.1 Session Mechanism in Zookeeper

• For the connection between the client and the server, you can set the timeout period. If the heartbeat packet mechanism (the client pings the heartbeat packet request from the server) is used to check that the heartbeat is over, the session expires
• When a session expires, all temporary nodes created by that session are discarded

5.2 Watcher mechanism in Zookeeper

6. ACL permission control in Zookeeper

• You can set read and write permissions for nodes to ensure data security

• Permissions You can specify different permission scopes and roles

• Common Commands

  • GetAcl Obtains node permissions
  • SetAcl Sets the node permission
  • Addauth: enter the authentication and authorization information. Enter the plaintext password for registration, but in ZK, it is saved in encrypted form

  [zk: localhost:2181(CONNECTED) 3] rmr /csp0001 [zk: localhost:2181(CONNECTED) 4] create /csp0001 HelloWorld Created /csp0001 [zk: Localhost :2181(CONNECTED) 5] getAcl/cSP0001Copy the code

• **[Scheme: id:permissions]**

• Scheme Authorization Mechanism

  • worldThere’s only one ID underneath, which is thetaanyoneAll of usworld:anyone:permissions
  • authOn behalf of authentication login, the registered user must have permissionauth:user:password:permissions
  • digestPassword encryption is required to accessdigest:username:BASE64(SHA1(password)):permissions(Auth is different from Auth because auth is in clear text and digest is ciphertext.)
  • IP, such as:ip:localhost:psermissions
  • Super stands for supermanage, having all permissions;
  • Open the /bin/zkserver. sh server script file in the Zk directory and find the following line:

• Id: indicates the user that is allowed to access

• Permissions: a string of permission combinations

  Cdrwa: c --> create Creates a child node D --> delete Deletes a child node R --> Read Reads data on a node W --> Write Writes data A --> Admin Manages permissions cdrWA CDR CDWCopy the code

• Acls are used in the following scenarios:

• The development environment can be separated from the test environment by using acl. The developer has no right to operate the test node. In the production environment, the service controlling the specified IP address can access the related node

7. Three roles and their election mechanism in Zookeeper

7.1 Three roles and their functions

• Leader: Serves as the sole handler of write requests for the entire ZK cluster and is responsible for initiating and deciding votes and updating system status.
• Follower: receives client requests, processes read requests, and returns results to the client. Forward the write request to the Leader; Vote in the election of Leader.
• Observer: can be understood as a non-voting Flollower, which is designed to help followers process more read requests. If the Zookeeper cluster has a high load of read requests, or if there are many, many clients that span the machine room, you can set up some Observer servers to improve read throughput.

7.2 Three common modes for a registry

7.3 Election Mechanism of Zookeeper

• myid

  This is the unique identity of the server in the ZK cluster and is called myID. For example, if there are three ZK servers, the numbers are 1,2, and 3 respectively.

• zxid

  ReentranReadWriteLock 32Bit high low0000000000000000  0000000000000000
  
                     epoch                                 xid
  00000000000000000000000000000000   00000000000000000000000000000000The zxID is of the type Long, with the medium height32Bit means epoch, low32Bits represent xID. That is, the ZXID consists of two parts: epoch and XID. Each Leader will have a different epoch value, representing a period or era. When a new Leader is created, the epochs in the ZXids of all ZKServers will be updated. Xid is the transaction ID of ZK. Each write operation is a transaction and has an XID. Each write operation requires a proposal from the Leader, and all followers vote on whether to approve the write operation.Copy the code

• The logical clock

  The Logicalclock, Logicalclock, is an integer. The concept is called Logicalclock at election time and the value of epoch in zxid. That is, epoch and logicalClock are the same value, but different names in different cases.

• Election status of ZK

  LOOKING, election status (find the status of the Leader). LEADING, leader status. The server in this state is called the Leader. FOLLOWING, follower state, synchronization leader state. The server in this state is called Follower. OBSERVING: indicates the OBSERVING state. Synchronization: indicates the leader state. The server in this state is called an Observer.Copy the code

8. Set up the Zookeeper cluster