Inside a second-argmax network

Given ten real numbers, return the position of the second largest.

ARC trained a recurrent network to do this with sixteen ReLU neurons and 432 weights. Its example_2nd_argmax() checkpoint achieves about 95.3% accuracy on standard Gaussian inputs. It has no biases:

ht+1 = ReLU(W ht + uxt) h0 = 0 s = V h10

The largest score selects an input position, indexed from zero. The pseudocode follows how all sixteen neurons and 432 weights produce those scores. The experiments below isolate how a candidate write promotes an input and how a false record produces a rank error.

How it finds the runner-up

As inputs arrive, the network builds evidence for their positions. Delayed writes strengthen candidates, and recurrent activity carries older evidence forward. Learned comparisons usually lower the score at the maximum’s position, allowing a strong runner-up to win.

When a new maximum arrives, those comparisons can release the old maximum from suppression, exposing it as the new runner-up. A record response also penalizes the new maximum’s own position. Learned thresholds and limited write supply can cause these signals to favor the wrong rank.

Functional routes through the circuit: maximum-like history drives comparisons, delayed signals feed position carriers, and both routes contribute to the output. Carrier history also feeds back into comparison thresholds.
Arrows summarize connections between neuron groups; the wiring diagram below shows individual weights.

First nine inputs: [1,0,0,0,0,0,0,0.5,0]. Position 0 holds the maximum, 1; position 7 holds the runner-up, 0.5.

Final input

Score 0 − score 7

With a final input of 2, all four comparisons close, removing their suppression of position 0. Trace and control.
The sixteen neurons in native order at two consecutive updates. Dark dashed red paths show candidate neuron 5 inhibiting all eight carriers; faint paths show each neuron's two largest incoming recurrent weights.
Darker dashed paths show candidate 5 inhibiting the eight carriers. Faint paths show the two largest recurrent inputs to each neuron. Solid positive, dashed negative.

A write is also feedback

Neurons 1, 6, 7 and 8 compare each arriving input with learned thresholds. Their shared activity and their differences help determine which inputs become candidates and how positions are scored. They draw on neuron 2, which carries an approximation to the maximum, and neuron 4, which responds by varying amounts when an input crosses a learned record threshold.

Neuron 5 forms a candidate for the preceding input. Its amplitude is proportional to max(0, τ − x), where τ is the stored cutoff and x the arriving input. Lowering the cutoff reduces or closes this candidate.

The candidate writes one update later by inhibiting eight carriers. Their learned readout can turn this reduction in activity into a relative score advantage for the tagged position. In the write example below, the timing is:

read x₇ → read x₈: form candidate for position 7
        → read x₉: spend candidate

Normalize each carrier by the cost of a candidate write. Its update is then:

available = relu(background)
spent = min(available, old_candidate)
carrier_next = available - spent

Limited supply can clip a write, so more candidate activity need not produce a proportionally stronger positional signal. Carrier activity also feeds back into comparison thresholds and an admission base, c, a history term used to form future cutoffs. A write changes both positional evidence and the conditions for later writes.

The sixteen-state pseudocode expands these steps into the complete update, including the learned filters and output contributions. Its coordinate map connects each variable to physical neuron activity.

One write, three consequences

To isolate a write, compare keeping a positive candidate with erasing it while holding the other fifteen old activities and the receiving input fixed. If carriers 0, 14 and 15 have nonnegative preactivations at the larger candidate endpoint, the trained weights guarantee three changes: the tagged position gains against all nine rivals, the admission base c rises, and the cutoff τ falls. This holds even if any of the other thirteen receiving ReLUs changes state.

Here the write arrives at the tenth update and promotes input position 7. The displayed changes in the base and cutoff are computed from the state after that update.

100%

Change in score 7 − score j

Admission base Δc Candidate cutoff Δτ

Changes relative to erasing old neuron 5, for [0.8,0,0,0,0,0,0,0.2,0,1]. All three carrier reserves hold. Exact bounds.

The write promotes position 7 even though its value, 0.2, is below the runner-up, 0.8 at position 0. A write can therefore strengthen the wrong candidate. Explaining which inputs the network admits requires understanding the thresholds that produced the write.

A false record leaves the maximum ahead

After inputs [1,0], the learned record threshold is about 0.979. An offer near 511/512 activates record neuron 4 even though it is below the existing maximum, 1.

For the full family [1,0,z,0,0,0,0,0,0,0], with z within 1/16384 of 511/512, the network selects maximum position 0 instead of runner-up position 2. At the following update, remove only the old record signal’s contribution to the comparison differences. This repairs the entire family. Removing its common-query contribution leaves the error.

Offer type
0.99804688

Score 2 − score 0

The source is removed once, while reading input 3. Every subsequent update follows the trained recurrence. Exact family and control.

For the true-record control, set z = 513/512. Position 0 is now the correct runner-up. Removing the contrast source narrows its margin over position 2 without changing the winner. The source strengthens the correct margin here even though it causes the error above. To use this cut as a repair, we would need to check how it behaves on other histories.

Evidence and limits

The reproduction kit verifies the two conditional claims above with rational arithmetic. It also reconstructs all 432 source coefficients from the circuit program and compares its scores with the original recurrence at every update. Run the checks.

A larger intervention study first failed an audit of its event labels. After those definitions were corrected, a separate cohort passed the technical audit but missed a coverage target. The archived summary records both outcomes; reproducing that study requires its separate implementation and reports.

Several groups of learned weights still appear in the program as filters whose effects depend on input history. We still need to explain what these filters accomplish, why other rank errors occur, and how the circuit determines overall accuracy.