RateLockr

Problem

Rate limiting one server is a counter. Rate limiting several servers behind a load balancer is a race, because every instance reads the count, decides, and writes it back, and two instances reading the same number at the same moment both think there is room.

Approach

Every decision runs inside one EVALSHA call. Redis executes Lua on a single thread, so the script is the critical section and there is nothing left to synchronise. No distributed locks, and no compare-and-swap retry loop to tune.

Three algorithms, each a self-contained Lua script with a TypeScript wrapper.

Token bucket keeps tokens and last_refill in a hash and refills lazily from elapsed milliseconds, clamped with math.max(delta, 0) so a backwards clock cannot mint free capacity. Its TTL comes from the bucket’s own shape, ceil((capacity / refill_rate) * 2) seconds, so idle keys evict themselves instead of accumulating.

Sliding window log uses a sorted set with a UUID member scored by arrival time. Expired entries get pruned with ZREMRANGEBYSCORE before the count is taken, which removes the boundary artifact where a fixed window lets through double its limit across a window edge.

Fixed window is a single integer key, the cheapest of the three. Denied requests decrement back, so a rejected burst does not inflate the counter against the next legitimate caller.

All three wrappers fail open, and the useful part is how they say so. On a Redis error the wrapper returns allowed: true with remaining: -1, and that -1 is a signal rather than a placeholder. The caller knows the decision was made without Redis and can apply its own controls. A rate limiter that takes the whole service down when its datastore blinks has traded one outage for a worse one.

Telemetry started as wildcard SCAN over the keyspace, which is O(N) and returns a different answer depending on when you ask. I replaced it with per-second counter keys on a 120-second TTL, and a read path that asks for exactly the thirty seconds it wants: sixty GETs batched into a single pipeline round trip, zero-padded so the chart always has thirty points whether traffic arrived or not.

Then the dashboard broke under its own instrumentation. The traffic simulator dispatches a refetch-stats event per request so the chart can update without the simulator knowing the chart exists, and a twenty-request burst turned that into dozens of concurrent stats calls that timed the API out. The fix is a leading-edge throttle at 750ms with a trailing fetch so the final state still lands, plus a guard that drops a fetch if one is already on the wire.

Results

Eighty-one commits between May and July 2026. Unit tests per algorithm, an integration test for concurrent access and one for eviction. Prometheus metrics at /api/metrics cover evaluation count by algorithm and result, evaluation latency, Redis errors and active rule count.

There is no load-test figure here, and that is deliberate. I had one, I could not produce the script that generated it, so it came off the site rather than stay as a number I could not show you.