Inside a second-argmax network

Technical note

Checkpoint

ARC’s example_2nd_argmax() loads one_layer_16_hidden_94_acc_2nd_argmax.pth. The function zoo_2nd_argmax(16,10) uses a different checkpoint with the same architecture. The checkpoint studied here has SHA-256:

498bd9a87c469e18e824a78e55c22843c7595776f1bfc2dfe07c36e5616218bb

model.json contains the 16 input, 256 recurrent and 160 readout weights. Rows of W are receiving neurons. Updates read one old state; prediction takes the first maximum of the ten final scores.

Direct evaluation gives 95,288 correct predictions out of 100,000 independent standard-Gaussian sequences: 95.288%. The evaluation record records seed 0, NumPy PCG64, CPU float64 and the generated input hash. These samples measure how often the network succeeds. Deriving that rate from its mechanism remains an open question.

Successful record switch

Both runs start from [1,0,0,0,0,0,0,0.5,0]. Final input 0.25 selects position 7; final input 2 selects position 0. The trace retains all sixteen final activities and all ten scores. Its comparison contribution to the displayed margin is:

comparison_margin = sum((V[0,i]-V[7,i])*h[i]
                        for i in (1,6,7,8))

That contribution is −18.266 for input 0.25 and zero for input 2, which closes all four comparisons. Other neurons contribute −6.648 and +1.363 respectively. Restoring only the lower-input comparison activities in the record run changes its total margin from +1.363 to −16.903 and selects 7 again. This control isolates how the comparisons affect the final readout. We have not shown that any input sequence could produce its mixed state, and this fixed numerical example leaves open how widely the rank switch occurs.

Sixteen stored coordinates

The pseudocode and its NumPy implementation store:

c, H, L, tau, w16, w87, a, b,
z0, z3, z10, z11, z12, z13, z14, z15

These variables express the same sixteen physical activities in different coordinates, so the program carries sixteen values from one update to the next. Let P=(0,3,10,11,12,13,14,15) and Q=all rows except 4,5,9. Positive scales normalize carry, record, query, candidate, relay and carrier activity:

M = -u4 / W42
G = W5 · relu(u); G9 = W9 · relu(u)
cost[k] = -G * W[k,5]                 k in P
h2=M*m; h4=u4*d; h7=abs(u7)*q
h5=G*a; h9=G9*b; h[P]=cost*z

For i=1,6,8, set g_i=q-h_i/abs(u_i). Widths are w16=g6-g1 and w87=-g8, with centre v=(g1+g6)/2. Native rows define the mixed fields:

T4 = -W4 · h / u4
R7 = W7 · h / abs(u7)
H = (T4+R7)/2; L = R7-T4
tau = W5 · h / abs(u5)
K = W[5,Q] · u[Q]
c = -(W[5,Q] @ W[Q,:]) · h / K

Solving four linear equations recovers m,d,q,v. The learned thresholds then produce d_next=relu(x-T4) and q_next=relu(R7-x). Those thresholds can differ from the true ranked input values.

Carrier background combines transport, current input, carry, query, comparison differences, relay 9 and record refill from 4. Old candidate 5 subtracts from that background before rectification. The resulting carrier flow updates c, H and L; a value/query packet and admission level form the next cutoff, as expanded in pseudocode block 3.

Physical rowsRetained operations
0, 3, 10–15Full carrier transport, controller/input drive, record refill and old-candidate expenditure.
2, 4Carry drive and record hinge.
1, 6, 7, 8Threshold gaps, signed clips, common query and comparison differences.
5, 9Previous-cutoff candidate and budgeted relay with deficit/self inhibition.
All readout columnsCarry, record, query, candidate, relay, carrier and separate signed-contrast terms.

Every table in circuit.json is derived from the trained weights. The checker uses these tables to reconstruct all source coefficients and compare complete histories. The three contrast clips stay separate because merging them into one cap loses the effect of their unequal widths.

Supplied-write law

Compare two nonnegative old states differing only at physical neuron 5 by G*δa, with δa>0, and keep the receiving input fixed. Nonnegative larger-endpoint preactivations at rows 0, 14 and 15 keep those receivers on throughout the segment. Every other ReLU secant is in [0,1]:

v[i] = G * W[i,5]
delta_h_next[i] = secant[i] * v[i] * delta_a
secant[0] = secant[14] = secant[15] = 1

For every response row, the rational verifier sums fixed terms and the negative/positive parts of the thirteen free terms. With the pinned weights, all nine score7-score[j] lower bounds and the c lower bound are positive; the tau upper bound is negative.

The bounds retain all sixteen receiving paths and describe the immediate effect of changing neuron 5. Determining whether the candidate has the right rank, whether its position wins, or whether the gain survives later updates requires further checks. The demo starts from a state produced by a real input prefix and erases neuron 5; we have not shown that another prefix could produce the erased state.

False-record family

For [1,0,z,0,0,0,0,0,0,0] with z∈[511/512−1/16384,511/512+1/16384], target 2 loses to maximum 0. At zero-based input 3, split the old record source for i∈{1,6,7,8}:

common[i] = abs(u[i]) * W[7,4] / abs(u[7])
contrast[i] = W[i,4] - common[i]

Other entries are zero. Remove a component by subtracting component[i]*old_h4 from the receiving preactivation, once, then run the native suffix. Removing contrast selects 2 throughout the interval; removing common query still selects 0.

The proof propagates affine states over the entire interval, checks receiving signs at both endpoints and checks all nine rival margins. It recomputes the gates separately for the fixed true-record control, z=513/512. There, the contrast cut reduces the correct 0-versus-2 margin while leaving position 0 ahead. Extending the cut beyond the proved family would require checking its effects on other input histories.

Reproduction and arithmetic

Use Python 3.12 or newer. Node is needed only for the browser checks.

python3.12 -m venv .venv
. .venv/bin/activate
python -m pip install -r reproduce/requirements.txt
python reproduce/check_circuit.py
python reproduce/proofs.py --check results/proofs.json
python reproduce/check_trace.py
python reproduce/evaluate_accuracy.py --check results/accuracy.json
node --test reproduce/browser.spec.mjs

Python’s standard library is sufficient to verify the two exact claims. NumPy runs the circuit program, and Node checks the browser recurrence against saved examples. The results are recorded as proofs, fixed examples and circuit checks.

The exact proofs treat the stored binary weights as rational numbers. The implementation checks allow a declared absolute tolerance for floating-point error. Recovering the fields can leave tiny positive decoded activities where a physical gate is closed, so event labels use the signs of newly generated preactivations, without an epsilon. The rational arguments establish results in exact arithmetic; floating-point comparisons check agreement within the stated tolerance.

Recorded intervention study

A larger study tested nine intervention profiles: both signs at inputs 0, 8 and 9, one positive lift to physical neuron 14 after six inputs, and both signs of a persistent W[5,5] edit. The evidence summary preserves its outcomes below. Reproducing the study requires the underlying reports and full verifier, which are outside this repository.

Amended cohortResult
Parents / arms8,192 / 73,728
Point-choice disagreements0
Certified / unknown / false70,791 / 2,937 / 0
All-six-input certification, negative-second errors40/58; the 70% minimum required 41.
Other coverage promises31 passed.

The original cohort failed its event audit because three features were read from numerically recovered activities. Before running a separate amended cohort, those events were redefined using signs of generated source preactivations. The amended cohort passed the technical audit but missed a coverage promise. Because the definitions changed between cohorts, we could not complete the planned two-cohort confirmation under a fixed protocol. The original cohort’s failed result still stands.

A certificate tells us whether a bound settles a particular intervention. An “unknown” result means the bounds did not settle the outcome; it says nothing by itself about which computations we understand. Matching a point prediction also leaves its causal explanation open. The exact negative-order families explain specific cases, while classifying arbitrary sampled errors with a negative runner-up remains an open problem. ARC also asks for an accuracy estimate derived from the mechanism; that remains to be done.