Rate Limiter

EvolvingSystem Designredisscalabilityinterview Created Oct 1, 2026 · Updated Oct 3, 2026

Protect a service from abuse and noisy neighbours, and keep latency predictable under load.

Pick by burst behaviour

Choose the algorithm by the burst behaviour you want, not by what is easiest to implement.

Algorithms

• Token bucket: allows bursts, refills at a steady rate.
• Sliding window: smoother limits at a higher memory cost.

const allowed = tokens > 0 // [[not a link]] inside code stays as written

Warning

A fixed window lets a client send double the limit around a window boundary.

Why a Lua script? (click to expand)

A Lua script makes the read-modify-write on Redis atomic.

Related

When sharding limiter keys across Redis nodes, see Consistent Hashing. For the idea behind it, read virtual nodes. Cost-aware limits are related to Prompt Caching. A note I have not published yet: Distributed Locks.