01
Caching
Expiry, stampedes, and what the cache was doing while it lied.
3Every caching incident p99 has published — what broke, what the operator was told, and the fix that held.
02
Incidents
3- Cache hit rate fell to 34% and stayed there. Stopping the rollout 18 minutes in changed nothing.caching · cascading-failure · sharding · control-plane · load-sheddinghard
- 80% of reads hit one post ID. Redis pins at 100% and every key's p99 goes 1ms → 300ms. Then the TTL expires.redis · caching · hot-key · thundering-herd · postgreshard
- 98% cache hit rate → 40%, every 60 seconds, like clockworkredis · caching · postgresmedium
→