129 likesNew · 1d old
Flash-Quantized 763B MoE runs on a 12GB GPU
What is DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF?
The repo offers three non-uniform GGUF quantizations of DeepSeek-V4.1-Flash-1017, a 763-billion-parameter mixture-of-experts model. By assigning a different low-bit format to each weight tensor, the model fits into a 12 GB VRAM card and can be used with standard llama.cpp tools.
Who it is for
Developers who want to experiment with state-of-the-art MoE LLMs without a multi-GPU server can now load the model on a single consumer GPU.