Rate Limiter Anti-Patterns

We centralize rate limiting at the API gateway (not inside each microservice) because per-service implementations duplicate logic, create inconsistency, and let malicious traffic penetrate deep into the system before being rejected.

Why: Teams add rate limiting where the pain is felt: inside the service that is overloaded. But this means every service implements its own version with different algorithms, different Redis keys, and different limits. A request that passes the gateway still consumes network, TLS, deserialization, and auth resources before being rejected.

We chose sliding window counter (not fixed window) because fixed window counters allow up to 2x the intended rate at window boundaries, which can overwhelm downstream services.

Why: Fixed window is the simplest algorithm: one counter per minute, reset at :00. Candidates pick it for simplicity and never consider what happens when 100 requests arrive at 0:59 and another 100 at 1:00. Both windows allow 100 each, so 200 requests pass in 2 seconds against a 100/min limit.

We always include Retry-After on every 429 (not as optional) because returning 429 without it forces clients into blind retry loops, amplifying the very overload we are trying to prevent.

Why: Developers return the 429 status code and consider their job done. But without Retry-After, every SDK, mobile app, and script retries immediately. A well-intentioned exponential backoff still starts at 1 second. Multiply that by thousands of throttled clients and the retry storm is worse than the original traffic.

We use Redis as the shared counter store (not in-memory HashMaps per server) because local counters let each server track independently, allowing N times the intended limit across N servers.

Why: In-memory counters are fast and straightforward. On a single server, they work perfectly. But behind a load balancer with 10 servers, a user hitting round-robin gets 10 separate counters. A 100 req/min limit becomes effectively 1,000 req/min because no server knows about the others.

We use Lua scripts (not distributed locks) for counter updates because Redlock or ZooKeeper adds 5-10ms of latency per request and creates a bottleneck.

Why: Candidates think: 'concurrent counter updates need a lock.' They reach for Redlock or ZooKeeper. But a distributed lock requires 3 round-trips (acquire, operate, release) at 2-3ms each. At 100K RPS, that is 100K lock acquisitions per second, and lock contention causes queuing that pushes p99 above 50ms.

We define per-endpoint limits (not a blanket global limit) because a single rate for all endpoints either throttles cheap reads too aggressively or lets expensive writes run unchecked.

Why: It is simpler to configure one rule: 100 req/min for everything. But a GET /user/profile costs 1ms while a POST /search costs 200ms and scans millions of rows. If both share the same limit, the search endpoint can be hammered 100 times/min, consuming $100 \times 200\text{ms} = 20\text{s}$ of server time per user per minute.

We chose fail-open with a local fallback (not bare fail-closed) because when the rate limiter's Redis cluster is unreachable, rejecting all requests turns a cache failure into a full API outage.

Why: Fail-closed sounds secure: if we cannot check the rate, block the request. But Redis goes down for reasons unrelated to abuse (network partition, maintenance, OOM). During those minutes, every legitimate user gets 429, and the API is effectively offline. The rate limiter, a protective system, becomes the single point of failure.

We apply service-level quotas to internal callers (not external-only limits) because a single runaway microservice can cascade-fail the entire backend when internal calls are unthrottled.

Why: Internal services are trusted. Teams assume internal calls are always well-behaved. Then a batch job spins up 100 threads and hammers the user service at 50K RPS. The user service saturates its connection pool, starts timing out, and every other service that depends on it fails too.