Kafka as a message middleware, how to ensure the minimum consumption of messages is often considered in the interview or development; 1. Consume at least once (from producer, consumer, kafka cluster perspective)
Producers: After sending data, producers can have callback functions to check whether the Future was sent. Retries parameter Sets the number of retries.
Consumer: Disable automatic submission and enable manual submission (which may cause repeated consumption) enable.auto.mit =false;
Kafka cluster:
1) acks=all(ensure that all replicas are synchronized to the message); Min.insync.replicas >1(acks = all; Set to ensure that at least one copy is alive); Replication. factor >= 3 (at least one copy is guaranteed to keep up);
Three parameters are used together; Replic. factor = min.insync.replicas + 1 is recommended
2) unclean. Leader. Election. Enable = false (this parameter in the leader after hanging up, not from the copy can not meet the requirements of the elected leader)