30 upvotes
LowRAM compresses KV caches to save memory at inference
What is LowRAM: Low-Rank Keys and Amortised Values for Composable KV Cache Compression at Test-Time?
LowRAM introduces low-rank keys and amortised values to shrink the KV cache that grows during autoregressive decoding. The method works at test time without retraining and can be composed across layers for larger context windows.
Who it is for
Anyone running large language models on limited GPU memory can use it to fit longer prompts or larger batches.
The story so far
Early Signal first spotted LowRAM: Low-Rank Keys and Amortised Values for Composable KV Cache Compression at Test-Time on November 9, 2026, with 30 upvotes. Early Signal files it under RAG & Memory, Security.