AdvertiseWatchSubscribe
Last scan 55 h agoMonday, November 16, 2026

▲ 388 today1k starsC++HN ×1New · 25d old

SSD streaming lets laptops run 700B MoE models

LLMsInference

What is inferssd?

InferSSD streams mixture-of-experts weights directly from an SSD so the model’s active parts fit in RAM or VRAM. It works with standard GGUF files and provides a browser chat UI and an OpenAI-compatible API.

Who it is for

Developers who need large language models but lack high-end GPUs can now run them locally.

Try it in 60 seconds

You need: A computer with an SSD and macOS, Linux (CUDA) or Windows; GPU optional for CUDA acceleration.

No one-line install. The README has the setup steps.

Apache-2.0 licenseUpdated todayC++85 open issues

New projects are not vetted. Skim the code, and run it in a fresh folder or a container before giving it your files or keys.

The morning email · free

Get tomorrow's biggest AI launches before everyone else.

One short email each morning: what launched, why it matters, how to try it. Five minutes, no spam, unsubscribe any time.

The story so far

Early Signal first spotted wici-ai/inferssd on November 12, 2026, when it had 1k GitHub stars. It now has 1k. On its busiest day it gained 388 stars. The project is 25 days old and written mainly in C++. It is published under the Apache-2.0 license and has 100 forks. It has been discussed in 1 Hacker News thread, the top one with 1 points. Early Signal files it under LLMs, Inference.

Questions about inferssd

Is inferssd free and open source?
Yes. wici-ai/inferssd is open source under the Apache-2.0 license, and the code is on GitHub at https://github.com/wici-ai/inferssd.
What do I need to run inferssd?
A computer with an SSD and macOS, Linux (CUDA) or Windows; GPU optional for CUDA acceleration.

Get the signal first.

Join the list for the morning email: the day's biggest AI launches, explained in five minutes. Unsubscribe any time.

Prefer a reader? Follow the RSS feed.