We know that message queue type components are often used in big data development. Message queue middleware is mainly used to realize asynchronous message, application decoupling, traffic peak cutting and other functions. Then, Kafka and ActiveMQ are common in business. With message-oriented middleware, where is the difference between the two, today we will do a simple comparison.
1. Basic comparison between Kafka and ActiveMQ
Kakfa was originally designed to be implemented with high throughput + high performance +HA, so we can see from the comparison in the above table that Kafka’s throughput is about two orders of magnitude higher than ActiveMQ’s. The core design of Kafka is the append log file. That is, write the log file continuously to achieve the message data write. Therefore, the performance of using this disk appending write is much higher than ActiveMQ random write. We can have a more intuitive feeling through the following table.
| contrast | ActiveMQ | Kafka |
|---|---|---|
| Development of language | java | scala |
| Interface protocols | JMS specification compliance, good language support | It does not follow the standard MQ interface protocol and is relatively complex to use |
| throughput | 10000 level (Random disk read and write) | Millions (sequential disk read and write) |
| The cursor position | ActiveMQ can not read historical data | The client can manage it by itself and read it repeatedly |
| timeliness | Ms level | Ms level |
| A working model | The queue and topic | There are only topics, but each topic can have many partitions |
2. Activemq and Kafka from the perspective of consumption pattern
1, activemqActivemq uses the traditional one-to-one, one-to-many (publish/subscribe) pattern, namely queue and topic. A queue is many-to-one. Producers send messages to a queue, and consumers take one from the queue, consume one, remove one from the queue. Each message is consumed only once.
Topics are broadcast. The producer sends messages to the broker. Each message contains a topic. Consumers subscribe to topics of interest, and all consumers who subscribe to a topic receive a message.
Topics are broadcast. The producer sends messages to the broker. Each message contains a topic. Consumers subscribe to topics of interest, and all consumers who subscribe to a topic receive a message.
2, kafkaKafka is a publish/subscribe based messaging system. His concept is a little more complicated.
The Kafka Brokers in the figure above is a Kafka cluster. A Producer is a Producer that sends messages to a Kafka cluster. Before sending messages, the messages are classified into topics. When sending messages, Producer can specify partitions to send messages to different partitions of a Topic or specify balancing policies to send messages to different partitions. When no partition is specified, the default random balancing policy is used to store messages randomly in different partitions. When a consumer consumes a message, Kafka uses offset to record the current consumption location. During consumption, several different groups can consume messages under the same topic at the same time. For a group, the number of consumers should not be more than the number of partitions, because in a group, each partition can be bound to at most one consumer, that is, a consumer can consume multiple partitions, and a partition can only consume one consumer. Therefore, if the number of consumers in a group is greater than the number of partitions, no message will be sent to the excess consumers.
Summary of application scenarios
In general Kafka is particularly suited for high-throughput, high-performance scenarios such as real-time data analysis and log analysis. Activemq is written in Java, so it can be used as a JAR package in a Java project to be started and configured with code. When the concurrency is below 10,000 levels, activemQ can be used as message middleware to achieve a more concise effect.