3 upvotes
Study finds Hundreds of Quantized LLMs on HuggingFace Broken
What is Broken on Arrival: Non-Functional Quantized LLM Artifacts in Public Model Registries and How to Catch Them?
Researchers downloaded every publicly listed GGUF-format model and ran them to see if they actually work. Over half of the 305 code-capable quantized LLM files failed basic functionality tests.
Who it is for
If you pull a quantized model for local inference, you could waste time and compute on a file that won’t run, so check the paper’s detection script before you download.