207 likes
Tiny 300M LLM runs at 466 tokens/s on iPad
What is MNN-Pocket-0.3B?
MNN-Pocket-0.3B is a 300-million-parameter language model tuned for instructions and built to run on-device with the MNN inference engine. The INT4 version fits in 173 MB and can generate about 466 tokens per second on an iPad Pro M5.
Who it is for
Developers who need a responsive LLM on mobile or edge hardware can embed it for real-time chat or text generation without cloud latency.
How to try it
Open the demo on the Hugging Face model page.
The story so far
Early Signal first spotted taobao-mnn/MNN-Pocket-0.3B on November 13, 2026, with 207 likes. The project is 54 days old. Early Signal files it under Inference, Video, LLMs.
Questions about MNN-Pocket-0.3B
- How do I try MNN-Pocket-0.3B?
- Open the demo on the Hugging Face model page.