1. What is NameServer?

NameServer is a service and Broker manager that holds Broker state.

NameServer features:

  • NameServer maintains a long connection to each Broker server and checks for the survival of the Broker every 30 seconds.
  • If a Broker is detected, it is removed from the routing registry. However, route changes do not immediately notify message producers. – Reduces the complexity of NameServer implementation and provides a fault tolerance mechanism on the message sender to ensure high availability of message sending.
  • The high availability of NameServer itself can be achieved by having multiple NameServerr servers that do not communicate with each other, that is, the data between NameServer servers is not exactly the same at any one time

Function:

  • Message producers and message consumers provide routing information about topics
  • NameServer Stores basic information about routes
  • Manage Broker nodes, including route registration, route deletion, and so on

2. NameServer Registers routes and removes faults

2.1 What Information does NameServer Store

  • Topic Message queue routing information. Load balancing is performed according to the routing table when messages are sent.
  • Broker base information, including brokerName, owning cluster name, and primary and standby Broker addresses
  • Broker cluster information that stores the names of all brokers in the cluster
  • Broker state information. NameServer replaces this information every time it receives a heartbeat packet
  • A list of FilterServers on the Broker for class pattern message filtering

RocketMQ is subscription-based, with a Topic having multiple message queues, and a Broker creating four read queues and four write queues by default for each Topic. BrokerName consists of multiple brokers in a cluster. BrokerName consists of the same multiple brokers in a master-slave schema. BrokerId 0 for Master and greater than 0 for Slave. LastUpdateTimestamp in BrokerLivelnfo stores the last time a Broker heartbeat packet was received

2.2 Route Registration

Steps for route registration:

  • The Broker sends heartbeat packets
    • Header
      • The Broker address
      • BrokerId, 0:Master; Greater than 0: slave
      • The name of the Broker
      • The name of the cluster
      • The master address is null at the first request and is returned after the slave registers with the NameServer
    • Body
      • List of message filtering servers
      • Topic configuration
  • NameServer Heartbeat packet processing
    • Route registration requires a write lock to prevent concurrent modification of the routing table in the RoutelnfoManager. Whether the cluster to which the Broker belongs exists, or if it does not, then add the Broker name to the cluster Broker collection
    • Maintain BrokerData information.
    • If the Broker is Master and BrokerTopic configuration information changes or is first registered, the Topic routing metadata needs to be created or updated, populated with topicQueueTable, which automatically registers routing information for the default Topic. This contains routing information for mixaii.default TOPIC. When a message producer sends a TOPIC, routing information for mixAIi.default TOPIC is returned if the TOPIC is not created and BrokerConfig’s autoCreateTopicEnable is true.
    • Updated BrokerLivelnfo, live Broker information table, and BrokeLivelnfo are important basis for performing route deletion
    • Register a list of FilterServer addresses for brokers that have multiple FilterServer message filtering servers associated with one Broker

The NameServer maintains a long connection to the Broker, whose state is stored in brokerLiveTable, Each heartbeat packet received by NameServer updates the brokerLiveTable with the status information about the Broker and the routing table (topicQueueTable, brokerAddrTable, BrokerLiveTable, filterServerTable)

2.3 Route Deletion

It is for the Broker to send a heartbeat packet to NameServer every 30 seconds to maintain the state of the Broker within the NameServer.

For NameServer, the brokerLiveTable status table is scanned every 10 seconds. If the Broker’s lastUpdateTimestamp is currently more than 120S in length, the Broker is considered invalid. Remove the Broker and close the connection to it. Update the corresponding information at the same time.

Trigger route deletion:

  • NameServer Timed Scan brokerLiveTable Detects the time difference between the last heartbeat packet and the current system time. If the timestamp is greater than 120s, remove the Broker message.
  • The unregisterBroker directive is executed if the Broker is normally closed.

2.4 Route Discovery

RocketMQ route discovery is non-real-time and NameServer does not actively push it to the client. Instead, the client periodically pulls the latest route to the topic.

3. Summary

This is where the NameServer route discovery and deletion mechanism comes in. We can see that this design has a case where: NameServer needs to wait for at least 120s for the Broker to fail before it can be removed from the routing table. If, during the failure of the Broker, the routing information obtained by the message Producer based on the topic contains the Broker that has already been looked at, the message will fail to be sent. What about this situation.