TRICKYwalkthrough
Hot Key Problem
Consistent hashing distributes keys evenly, but not access evenly. A viral tweet cached under one key receives 80% of all reads.
Virtual nodes do not help: they balance key count, not request volume. The fix: hot key splitting.
“That key maps to one primary node, which gets hammered while 99 others sit idle.”
Detect hot keys via per-node CPU divergence (one node at 90% while peers are at 30%) and split into 10 sub-keys: cache_key:0 through cache_key:9. The client appends , spreading reads across up to 10 nodes.
Writes update all 10 (10x write amplification, acceptable for read-heavy keys). Discord uses this pattern for their largest servers.
Alternative: read replicas on 2 additional nodes, simpler but limited to 3x fan-out. Trade-off: splitting requires the client to know which keys are hot via metadata sync.
Related concepts