Core components are started sequentially

The Partitioner is used to determine which partition each message is routed to. Returns which partition is used for int (int). The implementation class is DefaultPartitioner, which implements Partition and takes an atomic self-incremating random number,

private final AtomicInteger counter = new AtomicInteger(new Random().nextInt());

There are two situations when you select Partition

(1) If no key is specified in the message, counter+1 is used to retrieve all available partitions according to the cluster metadata information. Then the counter and the number of partitions are modulo to determine which partition to usepolling.



(2) If a key is specified, then hash the key through the utility class, and the resulting int is modulo of the number of partitions. So the same key canfixedSend a partition.

The Metadata component fetches Metadata from the broker cluster, including the status of topic-partitions, the positions of the leader and followers of each partition. New passes in two important configurations:

Ms: 100ms, metadata. Max. Age. Ms: 100ms, metadata. Max. Age. Ms: 100ms, metadata. Max. Age

As we start, we go back and call the update method. We just set the address of the manually configured broker (bootstrap.servers) to the cluster object, without actually sending a request to pull metadata. Why do you do that? Because metadata may be a lot, which topic to use when sending a message to pull the better is also a reflection of lazy loading.

RecordAccumulator – RecordAccumulator is responsible for receiving and caching messages. It packs the messages sent to each partition into batch batches. Each batch is full of messages. Or the lot has not been full, to a certain time must be sent. New passes in several core parameters:

Batch. Size: 16K buffer. Memory: Total cache size: 32M. Default BLOCK60S, maxBlockTimems compression. Type: compression type, gzip type, no compression by default. Retry. backoff.ms: Failed retry interval, default 100ms.

(1) The core append method is appended to the partition of the last batch. (2) The core append method is appended to the partition of the last batch. (3) The core append method is appended to the partition of the last batch.

The NetworkClient – the network communication component that encapsulates the network-related content. The core is the NIO, as shown by the encapsulated selector in the parameter. Core parameter

Connections. Max. Idle. MS defaults to 9 minutes. The maximum amount of idle time a network connection can take is less than 10 minutes for the server. Max. In. Flight. Requests. Per. The default 5 connection, the value of each network connection can endure producer side sent to broker the number of messages and no response. Receive. Buffer. Bytes Socket send buffer size, default is 128K Receive. Buffer

Request. Timeout. MS: Request timeout time, default 30s

Sender – The data Sender component gets data from the buffer. Is a thread that holds the network component NetworkClient and RecordAccumulator, concatenates the buffer with the network component, and passes in the parameters:

Max. In. Flight. Requests. Per. Connection this parameter specifies the producers before receiving the server response can send many messages. The higher its value, the more memory it takes up, but it also increases throughput. Setting it to 1 guarantees that messages are written to the server in the order they were sent. If you set it too much, then if some of the batch fails and retries, it's out of order. Request. Timeout. MS: The timeout time of the request is set to 30 seconds Max. Acks: 0 sent to return, no matter the success or failure; 1 Default, return after writing to leader; -1 isr is written to return. Tries the number of times you try

7. Initialize some important parameters (1) metadata

Ms: 100ms, metadata. Max. Age. Ms: 100ms, metadata. Max. Age. Ms: 100ms, metadata. Max. Age

(2) RecordAccumulator

Batch. Size: 16K buffer. Memory: Total cache size: 32M. Default BLOCK60S, maxBlockTimems compression. Type: compression type, gzip type, no compression by default. Retry. backoff.ms: Failed retry interval, default 100ms.

(3) the Sender:

Request. Timeout. MS: The timeout time of the request is set to 30 seconds Max. Default: 1M, can go up to 10M. Acks: 0 sent to return, no matter the success or failure; 1 Default, return after writing to leader; -1 isr is written to return. Tries the number of times you try

It is clear that each component is responsible for its own parameters.