Goodfire Researcher Says Neural Networks Are Already Clean
Goodfire researcher Nick Cammarata argues that large neural networks are far more understandable than previously believed, sparking an intense debate over the future of AI safety and interpretability.
Interpretability researcher Nick Cammarata has published a new essay titled "Neural Networks Are Already Clean," contending that artificial intelligence models are much easier to comprehend internally than the field currently assumes. Writing for the AI interpretability startup Goodfire, Cammarata refutes the popular superposition hypothesis—a concept he helped establish alongside Chris Olah in their influential paper "Zoom In." Superposition posited that networks squeeze multiple unrelated concepts into single neurons due to capacity constraints, making them tangled and difficult to decipher without complex extra models like sparse autoencoders.
Cammarata argues that famous historical counterexamples were simply misunderstood due to weak analysis tools and outdated architectures. For instance, a famous neuron in the InceptionV1 image model that previously appeared to activate for both cat faces and car fronts is actually a simple V-shape detector. In an automated interpretability project at OpenAI in 2023, Cammarata noted that only about 1.7 percent of neurons were classified as understandable. However, using modern automated agents to analyze models at scale, Goodfire found dramatically different results. In the language model Qwen3-32B, roughly 90 percent of neuron poles had a single primary function, about 7 percent had two related roles, and only 2 to 4 percent remained unexplained. Goodfire reports similar findings in models containing up to trillions of parameters.
The findings have sharply divided experts over what this means for AI alignment and safety. OpenAI employee Roon praised the essay, suggesting it could represent a landmark moment in scientific history that makes AI steering an ordinary scientific task. Conversely, Neel Nanda, who leads the interpretability team at Google DeepMind, remains skeptical. Nanda, whose team deprioritized sparse autoencoders after his perspective shifted in mid-2024, contends that language models contain far more facts than physical neurons and that superposition remains a reality. While Nanda acknowledges that direct neuron inspection is a pragmatic diagnostic tool, he cautions against declaring interpretability solved.
This is our own summary of reporting by The Decoder
