Kafka vs RabbitMQ: Architecture Decision for Event-Driven Systems

On a recent Distera Labs engagement, a logistics client needed to ingest telemetry from roughly 4,000 vehicles at a sustained 2,000 events/second, with consumers ranging from a real-time dashboard to a nightly analytics batch job.

The candidates

Kafka and RabbitMQ both entered the shortlist. RabbitMQ is a natural fit for task queues and routing-heavy workloads; Kafka is built around durable, replayable logs.

What decided it

Three requirements pushed us to Kafka:

1. Replay. The analytics team needed to reprocess the last 7 days of telemetry after fixing a bug in an aggregation job — Kafka's log retention made this a config change, not a data-recovery exercise.
2. Multiple independent consumers at different speeds. The real-time dashboard and the nightly batch job both read the same stream without coordinating with each other, which consumer groups handle naturally.
3. Partition-level ordering per vehicle. Keying by vehicle ID kept each vehicle's event order guaranteed within a partition, which the state-machine logic downstream depended on.

Where RabbitMQ would have won

If the workload had been command/task dispatch with complex routing (topic exchanges, per-consumer acknowledgment, priority queues) rather than a durable event stream, we'd have reached for RabbitMQ instead. The two aren't substitutes — the decision comes down to whether you're modeling a log or a queue.