2.4k likes
Qwen3.8-27B Gets Tiny, Fast GGUF Quantizations for Multimodal AI
What is Qwen3.8-27B-GSQ-RCO-GGUF?
Four non-uniform GGUF-quantized versions of the 27-billion-parameter Qwen3.8 model are released, each sized to fit a chosen memory budget. The GSQ-RCO pipeline mixes different bit-widths per tensor and bundles the vision projector, so the model can process image-text prompts and runs unchanged in llama.cpp, Ollama or LM Studio.
Who it is for
If you want to run a large multimodal LLM on a laptop or edge device without a big accuracy hit, you can download these GGUF files and use them with existing inference tools.