Anthropic Fellows paper argues neurons may be a stronger basis for interpretability

3 people · 3 posts · first post Tue 22:45 UTC · latest 1h ago

A paper from January's Anthropic Fellows cohort finds transcoders mostly pick up neuron features and that low-magnitude neuron activations are hard to interpret, suggesting neurons may be a stronger basis for interpretability than commonly assumed. Tim Hua shared it, calling it a banger; Roon commented that neural nets are already clean.

All posts · 2, most important first

1. @tim_hua_ research · Tue 22:45 UTC
   Banger: Transcoders mostly just pick up neuron features; low magnitude neuron activations are hard to interpret.
   There has been much discussion recently about how interpretable neurons in LLMs are. Today we're announcing our paper from January's cohort of Anthropic Fellows, where we show neurons may be a stronger basis for interpretability than commonly assumed. 🧵 (1/6) open on X
   2 likes · 0 reposts
2. @tszzl lab · Wed 00:09 UTC
   neural nets are already clean
   There has been much discussion recently about how interpretable neurons in LLMs are. Today we're announcing our paper from January's cohort of Anthropic Fellows, where we show neurons may be a stronger basis for interpretability than commonly assumed. 🧵 (1/6) open on X
   89 likes · 2 reposts

Reposted by @andrewcurran_.