Get CV
Back to business cases

Redis

Session keys that never expired

Redis filled with laravel_session blobs. Horizon lost reserved jobs to allkeys-lru.

Horizon started dropping reserved jobs on a quiet Tuesday. Redis INFO said used_memory_human 1.9G, maxmemory 2G, evicted_keys climbing. SCAN on laravel_session:* returned 1.79 million keys. expires on the whole db was 12. The queue keys sat next to session blobs under allkeys-lru. A listing lock and a reserved job left together.

SESSION_DRIVER was redis because Sanctum cookie auth needs a session on the API host. Anonymous catalog hits still created a session. The cookie lifetime in config was 120 minutes. The Redis key had no TTL. A crawler that never came back left a hash forever.

The problem was session keys without EXPIRE sharing LRU with Horizon. Quiet traffic still filled Redis. Jobs vanished. I needed TTL on write, no session on public catalog routes, and noeviction so a full Redis errors instead of dropping a reserved job.

A month of catalog traffic. Almost every key is a session that nobody will read.
A month of catalog traffic. Almost every key is a session that nobody will read.

What Redis is for here

Queue, locks, the dirty id set, and sessions that belong to a signed-in user. HTML stays in Memcached. Anonymous requests do not StartSession on the public catalog routes. The session key gets EXPIRE 7200 on write. maxmemory-policy is noeviction. If Redis is full, a web worker gets an error. A reserved job does not vanish.

HTML eviction and job eviction are different failures. They do not share a policy.
HTML eviction and job eviction are different failures. They do not share a policy.
  • auth_token stays httpOnly on the cookie. Redis stores the session id, not the token in JSON.
  • FLUSHDB is not the fix. The next crawler fills the same keys without TTL.
  • Horizon prefix and session prefix stay on one instance. The prefix does not save you from LRU.

used_memory fell to the size of the queue plus a few thousand live sessions. Failed jobs stopped appearing with empty payloads. The interesting number was expires, not used_memory, the first time I opened INFO.

What I took from this

INFO expires is the first number I read now. used_memory without TTL is a time bomb next to Horizon.

allkeys-lru treats a reserved job like a session blob. noeviction makes fullness a web error, which I can see, instead of a silent lost job.

Anonymous catalog hits do not need a session. Sanctum still has one after login. That split is cheaper than 1.79 million hashes.

Back to business cases