Design a Rate Limiter

1. Problem Statement

2. Architecture Discussion Map

Use this as one discussion aid, not a single correct answer. Your design should follow from the requirements, scale, and trade-offs you establish.

graph TD
    A[Client] --> B(Route53 / Global LB)
    B --> C[API Gateway Node 1]
    B --> D[API Gateway Node 2]
    C -- Local L1 Cache --> E[(Caffeine/Guava)]
    C -- Atomic check and update, script or CAS --> F[(Sharded counter store)]
    D -- Atomic check and update, script or CAS --> F

3. Key Focus Areas

• 1
  Rate Limiting Algorithms (Token Bucket vs Fixed/Sliding Window)
• 2
  Distributed State Management (Shared Counter Store and Atomic Updates)
• 3
  Race Conditions (Read-Modify-Write issues)
• 4
  Performance (Latency overhead < 5ms)
• 5
  Placement (Client vs Middleware vs API Gateway)

4. What the Interviewer Looks For

The interviewer probes these areas, and the report grades the interview against them.

• Requirements, rule model, and capacity
• Algorithm choice and semantics
• Distributed state and atomicity
• Latency, hot keys, and scalability
• Failure policy, multi-region, and operations