Accepted
2024-06-15
FizzBuzz Enterprise Edition needs to process numbers and determine their FizzBuzz classification at scale. The initial implementation used a synchronous request-response model, but as our customer base grew, we identified several limitations:
- Scalability: The monolithic architecture couldn't handle peak loads during end-of-quarter FizzBuzz processing
- Resilience: A single service failure would bring down all FizzBuzz operations
- Flexibility: Adding new divisibility checks (e.g., Bazz for 7) required modifying the core service
- Observability: It was difficult to trace a FizzBuzz determination through the system
We evaluated several architectural patterns:
- Monolithic with caching: Lower complexity but doesn't address resilience
- Microservices with REST: Better separation but tight coupling through synchronous calls
- Event-driven with message queues: Loose coupling, high scalability, but increased complexity
- Serverless functions: Good scalability but cold start latency concerns for real-time FizzBuzz
We will adopt an event-driven architecture using Apache Kafka (hosted on Upstash) as our message broker.
The architecture consists of:
- Number Ingestion Service: Accepts numbers and publishes to
number-topic - Fizz Service: Consumes numbers, determines divisibility by 3, publishes to
fizz-topic - Buzz Service: Consumes numbers, determines divisibility by 5, publishes to
buzz-topic - FizzBuzz Aggregator: Performs windowed stream join on
fizz-topicandbuzz-topic, publishes tofizzbuzz-topic - API Service: Serves FizzBuzz results to clients
Numbers are published to a single topic and consumed by both Fizz and Buzz services in parallel. This allows independent scaling and failure isolation.
The aggregator uses a 5-second tumbling window to join Fizz and Buzz results. This balances latency against the need to correlate results for the same number.
We configure Kafka consumers with enable.auto.commit=false and manually commit offsets after processing to ensure exactly-once processing of each number.
- Horizontal Scalability: Each service can be scaled independently based on load
- Fault Tolerance: Failure in one service doesn't cascade; Kafka provides durability
- Extensibility: Adding a new divisibility check (e.g., Bazz service) only requires adding a new consumer and updating the aggregator
- Observability: Event streams provide natural audit logs and debugging capabilities
- Technology Flexibility: Services can be written in different languages (TypeScript, Go)
- Increased Complexity: More services to deploy, monitor, and maintain
- Eventual Consistency: Results are not immediately available; clients must handle async nature
- Operational Overhead: Kafka requires management (mitigated by using Upstash managed service)
- Debugging Difficulty: Distributed tracing required to follow a number through the system
- Message Loss: Mitigated by Kafka's durability guarantees and proper consumer configuration
- Out-of-Order Processing: Mitigated by using message keys (number value) for partition assignment
- Windowing Edge Cases: Numbers at window boundaries may experience higher latency
Rejected because synchronous communication reintroduces tight coupling and cascading failure risks.
Rejected due to vendor lock-in concerns and cold start latency.
Rejected because Kafka provides better durability guarantees and ecosystem tooling.
- Designing Event-Driven Systems
- Kafka: The Definitive Guide
- Internal RFC: "FizzBuzz at Scale" (2024-05-01)
- 2024-06-15: Initial decision
- 2024-09-01: Updated window size from 10s to 5s based on latency requirements
- 2025-01-15: Added reference to Upstash migration