LRU is harder to beat than the KV-cache papers suggest
Recent analysis shows that the Least Recently Used (LRU) caching algorithm remains more competitive than recent key‑value cache research indicates. While newer KV‑cache methods claim superior hit rates and lower latency, empirical tests reveal LRU’s simplicity and low overhead still deliver comparable performance across typical workloads. The findings suggest that claims of easy replacement for LRU may be overstated.