Kafka
- Streaming
- hostedBy
- Upstash for free
- supports Cloudevents
- related: Apache Samza
From Redpanda
Go to text →
Similar
Managed
From Azure Event Hubs
Go to text →
Pros
Cons
- massive
- Event Sourcing is not a top-level architecture 1
Features
- dynamic routing
- compaction
Use Cases
- "Kafka is not an event store; rather, it is an enabler for building event stores."
Comparisons
From NATS
Go to text →
Kafka vs NATS
- partitions and topics "makes Kafka highly suited to applications where order matters; for example, state machine replication, event sourcing, log shipping, log aggregation, SEDA (staged event- driven architecture) and CEP (complex event processing)."
- Speaking of topics, the equivalent NATS subject is a lightweight construct that is created automatically based on demand (subscriptions) and is pruned automatically when the demand ceases. NATS subjects are cheap to create, which makes them great for hierarchically organised data, allowing for a fine-grained subscription model.
- (Note: NATS Streaming addresses this, but as stated earlier, it is a different product in its own right.)
- Kafka will never assign a partition to multiple consumers in the same group. So, although Kafka’s load balancing scheme is more coarse-grained than NATS’; it manages to preserve the order of records at the consumer nodes.
- kafka is lightweight
Dendrite use case
- "switching to JetStream was transformational. Performance is excellent overall. We make heavy use of the interest-based retention policies to ensure that work is cleaned up when it is complete. We take advantage of being able to manually acknowledge items to ensure they were processed successfully before cleaning up. We’ve been able to successfully manage pools of workers from NATS streams and considerably reduce the amount of complexity in our codebase in a number of places which no longer need their own persistent storage. We’ve even had it successfully running in application-embedded scenarios on iOS and Android as a part of our peer-to-peer demos."
Learning Resource
c.footnote (Private)
Footnotes
Backlinks