19 upvotes
Sparse Autoencoder Features Shift with Prompt Tweaks, Study Finds
What is Representation Redistribution in Sparse Autoencoders?
The paper shows that when you slightly rephrase a prompt, the sparse autoencoder that breaks down a language model's hidden states can move activity to different features. This means a change you see in one feature may not repeat on other, seemingly equivalent prompts.
Who it is for
Interpretability researchers and prompt engineers should treat SAE-based explanations as fragile and test them across multiple prompt variants.
How to try it
Open the paper and accompanying code at https://huggingface.co/papers/2610.21240