123 likesNew · 5d old
Open-source 35B diffusion model with 128K context
What is Clover-35B-A3B-Preview?
Clover-35B-A3B-Preview is a 35-billion-parameter mixture-of-experts language model that uses block-diffusion decoding to generate text. It supports prompts up to 128 K tokens while only activating about 3 B parameters per token.
Who it is for
Developers and researchers who need very long-context generation or want to explore diffusion-style decoding should give it a try.