Read this article in 4 minutes at 🎙️

This series of articles will focus on Pulsar and Kafka’s latencies, followed by articles on throughput.

The test details are covered in this article (blue, below), the test methods are covered in detail in the next article (green, below), and the test results are detailed in the previous article (red, below).

Test details:

Set the benchmark

To set up the benchmark tests, we followed the steps documented on the OpenMessaging site. After applying the Terraform configuration, You get the following set of EC2 Instances: We benchmarked using the steps provided on the OpenMessaging website. Applying the Terraform configuration yields the following set of EC2 instances:

I3.4 Xlarge instances for Pulsar/BookKeeper and Kafka Broker include two NVMe SSDS to improve performance. Both powerful virtual machines have 16 Vcpus, 122 GiB of memory, and high-performance disks.

Two SSDS is ideal for Pulsar because not only can two data streams be written, but data streams can also be parallel on disk. Kafka can also use the two SDD’s by allocating partitions between the two drives.

The Ansible Playbook for Pulsar and Kafka uses the tuned-adm command (latency performance profile) to tune low latency performance.

Tuned – reference linux.die.net/man/1/tuned adm order details…

The workload

While some workloads are included with the benchmark that are ready to run, some changes have been made to more closely match the results for Kafka in the LinkedIn Engineering blog. Defining a new workload is not difficult; simply create a YAML file with test update parameters.

If you read LinkedIn blogs, they run messages that are 100 bytes in size, because in general, if the messages are too small (much less than 100 bytes), the test comparison is not significant; All message queues are not good at handling “big messages” (much larger than 100 bytes), so a compromise size of 100 bytes was chosen, which is the size of the single message chosen for all messaging system tests.

This size makes it easier to test the performance of the messaging system itself. Regardless of the size of each message, the total number of messages used for testing is fixed. The more efficient a message system is in processing messages, the better its performance is. At the same time, the less likely it is that network or disk throughput constraints will affect test results. The performance of messaging systems for handling “big messages” is also a topic worth discussing, but we are currently only testing “small messages.”

In addition, we added a benchmark with six partitions in the test. Because we used a lot of 6 partitions in our LinkedIn tests, we added it as well.

LinkedIn blogs contain producer-only and consumer-only workloads, and the workloads we tested included both producer and consumer. There are two reasons.

First, as it stands now, benchmarks do not support producer-only or consumer-only workloads; Second, in real life, messaging systems serve both producers and consumers. We decided to test with a real-world scenario of producing and consuming messages.

In summary, the load set we used for testing is as follows:

Kafka Consumer Group and Pulsar subscriptions are similar in that they allow one or more consumers to receive all messages on a topic. When a topic is associated with multiple consumer groups/Subscription, the messaging system provides the topic with multiple copies of each message, or “fan out” messages.

Every message published on a topic is sent to all consumer groups/Subscription. If all messages are sent to the same topic and there is only one Consumer group/ Subscription, the producer rate is equal to the consumer rate.

If there are two consumer groups /subscription on a single topic, the consumer rate is twice as high as the producer rate. We tried to simplify our testing, so we used the former, where multiple consumers receive all messages on a topic.

The previous article detailed the test results of Pulsar and Kafka. The state of Fsync was a variable in the test, and the number of partitions was adjusted during the test to better compare the latency of Pulsar with Kafka.

Want to keep up to date with Pulsar’s development, user stories and hot topics? Follow the Apache Pulsar and StreamNative wechat accounts for the first time to share everything about Pulsar.

Click on thelinkSee the Original English version