Janas: Run LLMs larger than RAM by streaming experts from SSD
Janas is a new framework that allows inference with language models exceeding a machine’s RAM by dynamically loading model components from solid‑state drives. The system streams required expert modules on demand, leveraging SSD throughput to keep latency low while reducing memory footprints. It aims to make very large models accessible on commodity hardware without extensive hardware upgrades.