Rate Limiting Agent Calls

Where we left off

Lesson 4's INCR was atomic; Lesson 5's EXPIRE gave a key a lifespan. Put together, they're the entire mechanism behind rate limiting an agent's calls, no separate library needed.

The fixed-window counter

def is_allowed(r, user_id: str, limit: int, window_seconds: int) -> bool:
    key = f"ratelimit:{user_id}"
    count = r.incr(key)
    if count == 1:
        r.expire(key, window_seconds)
    return count <= limit

This is a fixed-window limiter, the simplest realistic version of the idea: INCR a per-user counter on every call; the first call in a fresh window also sets a TTL equal to the window's length, so the counter resets itself once the window passes, no cleanup job needed. If the counter is over limit, reject the call.

Why the count == 1 check matters

Only the call that creates the key should set its expiry, a call that finds the key already there is inside an existing window and shouldn't reset its clock. Without this check, a burst of calls late in a window would each push the expiry further out, and the window would never actually close, "one over limit" would let the count climb forever.

What "fixed-window" trades away

A true token bucket lets a limit refill smoothly and allows a small burst at a window boundary to even out; a fixed window can allow up to 2 x limit calls across two adjacent windows if they cluster right at the boundary (a burst just before the window resets, another just after). For per-user, per-minute limits on agent calls, that's usually an acceptable simplification. This is the tradeoff, spelled out honestly, not hidden behind a library name.

Checking without incrementing

r.ttl(f"ratelimit:{user_id}")

Useful for telling a caller how long until they can try again, without counting the check itself as a call.