XLNCXLNC Book a 20-minute scoping call
Menu

Blog

A neuron with one meaning still needs a network

Interpretability looks inside. Measurement checks what comes out. You need both.

By · · Measurement practice

In short

  • A neuron label like "V-shape" is a hypothesis about what the neuron means.
  • Construct validity research from 1955 and 1959 tests a label two ways: what it should relate to, and what it should not.
  • Cammarata already runs tests of both kinds. A full matrix adds at least one method from outside the labeling model family.
  • The same question applies to LLM judges: is one score measuring one thing?

What would it take to trust the label on a neuron?

On October 15, Nick Cammarata of Goodfire published a long essay, "Neural Networks Are Already Clean." He argues that much of what looked like hopeless tangle inside neural networks came from weak tools and ordinary, fixable problems (Cammarata, 2026). The Economist covered it on October 24 (The Economist, 2026).

His best-known example is an image-model neuron that seemed to fire for cat faces and car fronts. He now writes: "it's not a cat and car neuron, it's a V-shape neuron." It fires for pointed things and goes negative for round ones.

He also returned to GPT-2 XL, from OpenAI's 2023 automated interpretability project (Bills et al., 2023). Back then, "fewer than 2% of its neurons came out clean." With 500 new neuron studies, "about 70% look like roughly one thing." He notes the scoring "isn't identical to the original project."

Neel Nanda of Google DeepMind worries it will be hard to guarantee a complete picture. The Economist adds that it is too early to know whether the method scales to frontier models (The Economist, 2026).

This is good news, and it deserves credit. It also raises an old question. "One main meaning" is a label. How do we know what the label means?

He is already testing his labels

Cammarata does not ask readers to take his labels on faith. "The worry that you can explain anything is fair," he writes, "but a good explanation is specific enough to lose."

He backs that with a test. For 183 neurons, an agent read only each neuron's explanation and wrote new examples that should fire it, plus near misses that should not. The new examples fired their neurons strongly 76% of the time. The near misses fired them 11% of the time. Texts about the most distant concept the agent could invent fired them about 1% of the time (Cammarata, 2026).

For single neurons he also checks steering, what feeds the neuron, what it feeds and what drives it negative. "Any of these could break the explanation," he writes, "yet they keep pointing to the same place."

He is candid about limits: "I trust these only up to a point."

Psychologists call this testing construct validity.

A label is a hypothesis

In 1955, Lee Cronbach and Paul Meehl argued that a measure means what its pattern of relations says it means. You state how the construct should relate to other things you can observe, then you check. They called that web of predictions a nomological network (Cronbach & Meehl, 1955).

The network does two jobs. It says what the construct is related to. It also says what it is not related to.

Labels can pass checks and still mean nothing. Andrew Maul wrote deliberately meaningless survey items. They still looked fine on reliability, factor analysis and correlations (Maul, 2017). The checks were real. The meaning was not.

So "70% have one main meaning" is the claim a network would test. It is not yet the conclusion.

Two checks from 1959

Donald Campbell and Donald Fiske then added a practical tool, the multitrait-multimethod matrix (Campbell & Fiske, 1959). Measure several traits by several methods and read two patterns.

  • Convergent evidence. Different methods aimed at the same trait should agree.
  • Discriminant evidence. The same method aimed at different traits should not agree just because it is the same method.

Figure 1 marks those regions on a published matrix.

Four regions of a multitrait-multimethod matrix, the correlations printed in Campbell and Fiske 1959, Table 2, page 86. Courtesy, Honesty, Poise, and School Drive, measured by peer ratings and an association test. Dark cells in parentheses are the reliability diagonal. Blue italic cells are the validity diagonal, the same trait by both methods. Tan cells are heterotrait-monomethod, different traits by one method. Pale cells are heterotrait-heteromethod, different traits by two methods. In the same-method blocks, cells above the diagonal repeat the printed lower triangle. The two-method block is the printed rectangle. The pale .16 and the blue .16 are different cells.
Figure 1. Four regions of a multitrait-multimethod matrix, using the correlations printed in Campbell and Fiske (1959), Table 2, page 86. Courtesy, Honesty, Poise, and School Drive, measured by peer ratings and an association test, from Kelley's study of school children. Parentheses mark the reliability diagonal. Blue cells are the validity diagonal: the same trait by both methods. Tan cells are heterotrait-monomethod: different traits, one method. Pale cells are heterotrait-heteromethod: different traits, two methods. In the same-method blocks, cells above the diagonal repeat the printed lower triangle. The two-method block is the printed rectangle. The pale .16 and the blue .16 are different cells. On a narrow screen, scroll the figure sideways to read every column.

The second check is the one people skip. If one family of agents writes the labels, the test examples and the grades, some agreement can be the method agreeing with itself. Correlated errors across large language models are documented at scale (Kim et al., 2025).

Here is how the matrix maps onto a neuron.

Multitrait-multimethod ideaIn interpretability
TraitsCandidate meanings: "V-shape," "pointed object," "car front"
MethodsTop activations, steering, ablation, upstream and downstream circuits, held-out behavior
Convergent evidenceSteering, circuits and new examples all point to "V-shape"
Discriminant evidenceNear misses and round shapes stay quiet; the "V-shape" explanation beats "cats and cars" on the same tests
Method varianceLabels and tests written by agents from one model family

Cammarata's near-miss test is discriminant evidence for one method. A full matrix would cross several candidate meanings with several methods, including at least one method run by a different model family or by people.

The V-shape neuron already has the start of a network. It should fire for pool tables and bullet trains. It should go negative for golf balls and clocks. It feeds neurons for flames. Cammarata reports all three (Cammarata, 2026).

Relations the essay reports for the V-shape neuron, with no coefficients. It fires for pool tables and bullet trains, goes negative for golf balls and clocks, and feeds a flame neuron.
Figure 2. Relations the essay reports for the V-shape neuron. It fires for pool tables and bullet trains, goes negative for golf balls and clocks, and feeds a flame neuron. A diagram of those reports, with no coefficients.

The next step is to write that network down before the data come in, then count hits and misses. A meaning that survives both earns its name.

Inside and outside

Interpretability asks what is inside a model. Measurement asks how much of a property the system shows, on a scale someone else can check. A perfect map of the inside still does not set a pass line.

Luca Mari, Mark Wilson and Andrew Maul offer a frame for the outside question, the Hexagon framework (Mari, Wilson & Maul, 2023; Wilson & Mari, 2026). It separates six things:

  • the property you mean
  • what the instrument actually picks up
  • a local reference
  • the local value you read
  • a shared public reference
  • the public value you report

It also names the operations that link them. Metrological traceability is the documented chain through those links, with an uncertainty at each one.

The Hexagon pairs naturally with a construct map (Wilson, 2023). You order the levels of a property before you collect data, then test whether the data keep that order. We call those ordered levels waypoints. For reasoning complexity, we use the stages of the Model of Hierarchical Complexity as those waypoints.

That is a nomological network with a ruler attached.

We hold our own work to the same standard. Today the chain from our judge scores is unbroken to our keyed reference set of record. The link to the shared ruler is the one we are still closing. A stopping rule that asks only which side of a cut a text is on is in build (hypothesis testing mode).

The same question about judges

A neuron with one main meaning and an LLM judge with one score face the same question. Is this one thing, or two things averaged together?

For judges, fit checks ask whether a judge measures one thing. When we checked one judge, Jev, on 350 safety conversations, its bias questions split away from everything else (my post on Jev). One score was carrying two rulers.

It is not rare. In our Oct 25 calibration of 15 LLM judges, 14 failed our fit checks at one or more complexity levels, often on small samples.

Checking outputs against a scale found that. Looking inside might tell us why.

What we would like to see

Three steps would turn "one main meaning" into evidence of meaning:

  • Write each label's network in advance: what it should relate to and what it should not.
  • Fill a multitrait-multimethod matrix for a sample of neurons, with at least one method from outside the labeling model family.
  • Report how often labels hold on both convergent and discriminant evidence, with an uncertainty.

Cammarata has done much of the hard part.

If you build or buy LLM judges, how do you check what your scores mean? Tell me in the comments.

I'll also cover judge checks in a free Maven Lightning Lesson, Beyond Pass Rates: AI Evaluation That Moves the Business, on Thursday, January 7, 2027, 9:00 AM ET. Save a seat on Maven.

References

Tell us which number you do not trust.

You will get a short note from Matt when a measurement slot opens. Nothing else, no drip sequence.

We store what you send with the few providers our Privacy Policy names, and use it to reply to you. We do not sell it. Email matt@xlnc.co and we delete it. Read our Privacy Policy.