▲ 388 today1k starsC++HN ×1New · 25d old
SSD streaming lets laptops run 700B MoE models
What is inferssd?
InferSSD streams mixture-of-experts weights directly from an SSD so the model’s active parts fit in RAM or VRAM. It works with standard GGUF files and provides a browser chat UI and an OpenAI-compatible API.
Who it is for
Developers who need large language models but lack high-end GPUs can now run them locally.
Try it in 60 seconds
You need: A computer with an SSD and macOS, Linux (CUDA) or Windows; GPU optional for CUDA acceleration.
No one-line install. The README has the setup steps.
Apache-2.0 licenseUpdated todayC++85 open issues
New projects are not vetted. Skim the code, and run it in a fresh folder or a container before giving it your files or keys.
The story so far
Early Signal first spotted wici-ai/inferssd on November 12, 2026, when it had 1k GitHub stars. It now has 1k. On its busiest day it gained 388 stars. The project is 25 days old and written mainly in C++. It is published under the Apache-2.0 license and has 100 forks. It has been discussed in 1 Hacker News thread, the top one with 1 points. Early Signal files it under LLMs, Inference.
Questions about inferssd
- Is inferssd free and open source?
- Yes. wici-ai/inferssd is open source under the Apache-2.0 license, and the code is on GitHub at https://github.com/wici-ai/inferssd.
- What do I need to run inferssd?
- A computer with an SSD and macOS, Linux (CUDA) or Windows; GPU optional for CUDA acceleration.