Kafka was originally developed by LinkedIn and opened source in early 2011. I graduated from Apache Incubator in October 2012. Confluent was founded in 2014 by the creators of the Kafka project.

Confluent co-founder and CTO Neha Narkhede writes about the release of Apache Kafka 1.0.

Kafka was originally designed as a highly scalable messaging system; Today, it is a full-featured distributed streaming platform that supports publishing, subscribing, storing, and processing of streaming data. There are a number of companies using Apache Kafka for critical business purposes.

Neha Narkhede summarizes some of the core features that Kafka has added along the way.

While being adopted at those companies for mission-critical applications, Kafka has also matured at a steady pace, adding first replication and the ability to store keyed data indefinitely, then the Connect API for integrating Kafka easily with systems like MySQL and Elasticsearch, then the Streams API to enable native stream processing in Kafka for mission-critical real-time applications and event-driven microservices, and then this spring, exactly-once processing semantics. 

Version 1.0 offers many new features, such as a better Streams API, more operational metrics, support for Java 9, increased tolerance for hard drive failures, and more.

New features

  • [KAFKA-4602] – KIP-72 Allow putting a bound on memory consumed by Incoming requests

  • [KAFKA-4819] – Expose states of active tasks to public API

  • [KAFKA-5341] – Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics

  • [KAFKA-5651] – KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines

  • [KAFKA-5694] – Add ChangeReplicaDirRequest and DescribeReplicaDirRequest (KIP-113)

  • [KAFKA-5738] – Add cumulative count attribute for all Kafka rate metrics

  • [KAFKA-5746] – Add new metrics to support health checks

  • [KAFKA-5854] – Handle SASL authentication failures as non-retriable exceptions in clients

To improve the

  • [KAFKA-1595] – Remove deprecated and slower scala JSON parser

  • [KAFKA-2376] – Add Kafka Connect metrics

  • [KAFKA-3741] – Allow setting of default topic configs via StreamsConfig

  • [KAFKA-4108] – Improve DumpLogSegments offsets-decoder output format

  • [KAFKA-4260] – Improve documentation of configuration listeners=PLAINTEXT://0.0.0.0:9092

  • [KAFKA-4322] – StateRestoreCallback begin and end indication

  • [KAFKA-4380] – Update the usage description of clean shutdown file.

  • [KAFKA-4416] – Add a ‘–group’ option to the console consumer

  • [KAFKA-4501] – Support Java 9

  • …

Detailed features, see the release notes: https://dist.apache.org/repos/dist/release/kafka/1.0.0/RELEASE_NOTES.html.

Download: https://kafka.apache.org/downloads.