Pre-reg: released MolmoAct2 in sim, off-contract _convmap (20 seeds, parallel)

2026-08-12 17:3xZ — owner prio 17:13:24Z: “Could we also try running the released checkpoint directly”, with an attached box-side note on molmoact2 unit contracts (committed copy: fontaine/notes/molmoact2-unit-contracts-box-note.md). Exploratory rough-numbers pass, not a registered claim.

Plain words. The released checkpoint speaks a different unit language than our simulator: its normalization table assumes joint angles in the older community convention, while our sim reports controller-native v3 values that sit below the release table’s floor. Fed raw, the model would be effectively blind and the score would measure the unit clash, not the policy. So we translate at the boundary — convert the sim’s state into the model’s units on the way in, and the model’s actions back into sim units on the way out — and clearly label the result as an off-contract read.

Design (case 3 of the box note)

• Checkpoint: ~/marius-convert-gate/converted/molmoact2_so100_101_release (already converted; backbone allenai/MolmoAct2-SO100_101).
• Shim: exact per-joint affine, both directions — state-in (v3 → model units before its q01/q99 table normalization), action-out (model output → v3 before the controller). No re-training, no table edits.
• Arms: same 20 seeds (sim100 list 0–19), fixed post-flip sim, parallel driver workers=8 — paired vs the step-500 and step-2000 corrected arms (parallel-path rough rows, per the failed-oracle rule).
• Label: _convmap, off-contract — never pooled with ftrig contract reads; interpreted as a lower bound (the release trained on a mixture of conventions through one table; outputs are mixture-blurred even under a perfect shim).

Gates / tripwires (mandatory, pre-GPU)

1. Print the release box from its norm_stats; verify the mapped reachable set A⁻¹(release box) covers the sim task workspace — the clamp travels with the model. Fail → report, don’t run.
2. First-action-vs-current-state check (the note’s unit-bug detector; the release contract read had first_mae 18.0 vs state-copy 2.5). A correct shim collapses this to ~state-copy scale. If it does not, STOP — do not spend the GPU on a mismatched map.
3. GPU gate: ≤0.5 GPU-h (one 20-seed parallel arm ≈ 0.1; debug budget included).

Reads

• success, mean/median progress_final_cm, knock-aways ≥1 cm, per-seed paired deltas vs step-2000 corrected and step-500 arms; videos + rows to fontaine-reports under /ftrig_eval20_flip_parallel/release_convmap/.
• Cross-check bank (the box asked): does our sim calibration imply the same lift +180° / elbow +90° old-convention map that fit_convention_map snapped? Disagreement → flag in-channel; one of the two sides has a sign/offset wrong.

Results (2026-08-12 17:5xZ, same session)

Headline: the released checkpoint is INERT in our sim — progress_final exactly 0.00 on all 20 seeds, the boat never touched. Not frozen: the arm moves smoothly and repeatably (swings down-left to a consistent off-task region near the table edge, wrist camera ends staring at darkness, every seed) — coordinated in-workspace motion that entirely ignores the scene’s objects. Zero knock-aways, zero approaches.

Paired (release − step-2000): +0.46 cm CI95 [−0.01, +1.11], 4/2/14 — entirely an artifact of step-2000’s two knock-aways; the release’s zero is inertness, not competence. Paired (release − step-500): −0.02 [−0.26, +0.24], noise. Off-contract lower-bound read, as registered.

The shim (tripwires did real work)

The gated fit_convention_map on rig-table → release-table gave lift +180 only. Tripwire (a) then failed three joints, and tripwire (b) — first-action-vs-state, vs the ftrig contract anchor run through the same metric — arbitrated:

• elbow_flex +90 override: identity left 56% of the rig range below the release floor (+90: 10%); confirmed empirically (elbow first-action delta 9.2° vs anchor 13.2°). The gate had missed it by 2.2° of midpoint pad — a near-tie the coverage instrument caught.
• wrist_roll −90 override: the smoking gun was the clamp signature — first-action delta 34.5° under identity ≈ exactly the gap from sim home (77.6°) to the release ceiling (43.5°); with −90 the home maps to −12.4°, dead center of the release box, and the delta collapsed to 0.97°.
• Final map (lift +180, elbow +90, wrist_roll −90): arm-joint mean first-action delta 2.98° vs contract anchor 6.31° — the shim collapses the unit-bug detector below state-copy scale, exactly the note’s prediction for a correct map.
• Residual caveat: wrist_flex/wrist_roll coverage stays ~53–61% uncovered under ANY discrete offset — a span mismatch (the release corpus’s wrist workspace is much narrower than our rig’s), so the clamp bites during dynamic wrist motion. Part of why this read is a lower bound.

Cross-check bank (for the box)

Our seam fit vs the box’s curated-panel snaps: lift +180 AGREE (gated fit, no override needed). elbow +90 AGREE — but only via the coverage+first-action override; the stats-side midpoint gate alone picks identity for our rig table (midpoint 27.6° sits 2.2° inside the padded box). If the box’s estimator ran on a dataset shaped like our rig table it would call elbow in-convention too — the gate’s midpoint displacement rule under-translates near-tie joints; coverage fraction + a first-action probe disambiguates. wrist_roll: −90 for our rig (sign resolved empirically), consistent with the panel’s ±90 wrap family. No sign/offset contradiction between the two instruments once the near-tie is arbitrated.

Interpretation

The unit contract is now demonstrably NOT the blocker — state is visible, actions land in-workspace, first-action continuity is better than the fine-tuned checkpoint’s. What remains is everything else: scene appearance, camera geometry, task grounding. The release moves through its own prior’s motion distribution, blind to our boat. This cleanly brackets the ftrig fine-tune’s contribution: 2000 (or 500) steps of rig data buy scene-directed reaching (approaches, knock-aways, one +1.59 best) from a base that, unit-corrected, does nothing task-relevant in this scene.

Artifacts: rows + 20 videos + chart (/ftrig_eval20_flip_parallel/release_convmap/), instrument sim/convmap.py + --convmap-seam-stats on the parallel driver, tripwires fontaine/scripts/convmap_tripwires.py, oracles tests/test_sim_convmap.py. GPU spend ≈0.19 GPU-h (tripwires ~0.08 + run 0.09 + debug margin) of the ≤0.5 gate.

Amendment 1: official-map rerun (registered 2026-08-12 18:5xZ, pre-launch)

Why. Owner 18:19:08Z caught a real discrepancy: the official LeRobot v3.0→v2.1 SO-100/101 conversion (irenegracekp/molmoact2-so101 inference.py: offsets 0,90,90,0,0,0, signs 1,-1,1,1,1,1) sign-flips shoulder_lift — model = 90 − arm. Our fitted map used (+1,+180) on lift: the mirror (−1,+90) qualified in the fit and covers the release box better (7.5% vs 27.9% uncovered) but lost to the pre-registered MIRROR_MARGIN=0.25 rule by 20.4 pt. A wrong lift sign direction-inverts decoded lift motion — consistent with the filmed swing-down-and-park — and the first-action detector is sign-blind at rest (any bijection preserves action≈state at the start pose). The INERT 0.00×20 headline above is therefore SUSPECT on lift until this rerun re-dispositions it.

Arms (owner-confirmed 18:34:34Z + 18:36:29Z): same 20 seeds, fixed post-flip sim, workers=8, ≤0.4 GPU-h total.

• Arm A (primary): the snippet map EXACTLY — signs 1,-1,1,1,1,1, offsets 0,90,90,0,0,0 (wrist_roll identity, per the snippet).
• Arm B (secondary, owner-confirmed): snippet + wrist_roll −90 — our empirically-resolved wrist arm (identity clamps sim wrist home 77.6° above the release ceiling 43.5°; −90 may absorb a rig-specific zero). Arm A runs first.

Instrument delta. --convmap-override extended to carry sign (JOINT=[SIGN,]OFFSET, e.g. shoulder_lift=-1,90); bare form unchanged (+1). Oracles in tests/test_sim_convmap.py. New --rows-jsonl per-episode stream on the parallel driver feeds per-episode in-channel updates (owner ask 18:34Z; completion order under workers=8).

Procedure. Tripwires (a)+(b) under the official map first (3-seed first-action probe; b is sign-blind at rest — recorded, not decisive); then arm A, then arm B, per-episode Discord posts as rows land. Reads: same as the parent (success, progress_final, knock-aways, paired vs the existing release_convmap rows and the step-500/step-2000 arms). The INERT claim gets explicitly re-dispositioned either way — correction posted if lift sign changes the behavior, confirmation if not.

Amendment 1 results (2026-08-12 19:0xZ, same session)

Re-disposition: the INERT characterization is PARTIALLY OVERTURNED. Under the official lift sign the release is not inert — it reaches down over the table toward the boat and makes contact on 2 of 20 seeds — but it remains task-incapable: 0/20 pickups on both arms, median progress exactly 0.00, mean ≈ 0. The parent’s flat 0.00×20 was an artifact of the wrong lift sign parking the arm off-task; the parent’s conclusion (units are not the main blocker; scene/task grounding is) stands.

Tripwires first (both arms): lift mirror (−1,+90) covers the release box at 7.5% uncovered vs the old +180’s 27.9%; first-action arm mean 2.62° (arm B) vs contract anchor 6.31° — the detector is sign-blind at rest, so it can’t distinguish +180 from the mirror, but nothing flags. Arm A’s wrist_roll-identity leg read 34.0° on wrist_roll (the known clamp signature); run anyway per the owner’s call.

• Arm A seed 6: directed reach to 1.4 cm from the boat (wrist camera ends looking straight at it), retreats without closing; progress_final +4.61. Seed 16: sweeps through the boat, knock-away −5.26 — the boat was touched, which never happened in 20× parent episodes.
• A vs B (wrist_roll identity vs −90): NULL — paired −0.02 CI95 [−0.75, +0.66], 11/20 exact ties. 20 seeds cannot separate the wrist arms behaviorally; the official identity stays canonical (external documentation), arm B’s cleaner first-action (0.47° vs 34.0°) noted as mechanism, not outcome.
• Paired vs the ftrig arms: A−step2000 +0.35 [−0.55, +1.29], B−step2000 +0.37 [−0.27, +1.08]; A−step500 −0.13 [−0.89, +0.62] — all CI-include-0. The earlier bracket claim softens accordingly: the unit-corrected base shows occasional scene-directed reaching (1–2/20 approaches vs the rig fine-tune’s 7/20), so 500–2000 ft steps buy more frequent engagement, not engagement from zero.
• Instrument lesson (for the box): the pre-registered MIRROR_MARGIN=0.25 rule rejected a real, externally documented mirror that also won on coverage (20.4 pt margin loss). Mirror adjudication should carry a coverage tiebreak or an external-doc override; our sign-carrying --convmap-override is the escape hatch.

Canonical shim going forward: the official snippet map exactly — signs 1,-1,1,1,1,1, offsets 0,90,90,0,0,0.

Artifacts: rows + 20 videos per arm on the reports Space (arm A, arm B). GPU spend ≈0.25 GPU-h (2× tripwires ~0.07 + 2×20-seed arms 0.18) of the ≤0.4 gate.

Amendment 2: seed-6 30-second extension (registered 2026-08-12 19:53Z, pre-launch)

Owner call 19:25Z (watched the arm-A seed-6 video: “a clear grab and lift — can we rerun just seed 6 with a longer time horizon; the idea was 30 seconds”): the eval-20 protocol above ran --replans 15 × 30 ticks = 15 s of sim time, half the sim100 protocol’s 30 replans = 30 s. The owner’s read is right — the seed-6 reach was cut off at half the intended budget.

• Arm: arm A verbatim (release checkpoint, official snippet map exactly), seed 6 only, same euler-10 decode, same seam stats — the only change is --episode-seconds 30 (900 ticks, 30 replans). Runs on the parallel driver at workers=1 (same code path as the parent rows).
• Instrument: new --episode-seconds flag (commit c26a99e) states the episode budget in TIME and derives the replan count from the resolved chunk horizon — a fixed replan count quietly scales the budget with chunk length.
• Read (record-only, n=1): does the seed-6 approach continue to contact/grasp given the second 15 s, or does it hold/retreat? progress_final + the video are the artifacts; no statistical claim from one episode. The first 15 s are NOT expected to replay the parent row bit-exactly at the replan boundary (the 15-replan episode ended; this one replans through tick 450), but ticks 0–449 follow the identical policy path and should match closely.
• Gate: ≤0.05 GPU-h (one 900-tick episode, ~2× a parent episode’s wall).

Amendment 2 results (2026-08-12 20:0xZ, same session)

GRAB CONFIRMED. With the full 30 s the seed-6 episode is a grasp and carry: the wrist camera shows the boat held in the gripper jaws for ~4 s while the arm swings across the table; benchy→disk reaches 1.04 cm at t=15.9 s (the boat carried nearly onto the disk), the gripper never opens, and the arm swings back and sets the boat down at its spawn distance (final 10.14 cm, upright 0.82, z 4.7 mm — back on the table, slightly tipped). No success trigger. The release’s failure mode on this seed is place/release, not grasp — a material sharpening of the amendment-1 read, where the 15 s budget cut the episode off mid-carry and logged it as “directed reach, retreats without closing” (+4.61).

Trajectory (benchy→disk cm at 1 s intervals): flat 10.07 → 13 s; 12.36 at 14 s (pre-grab nudge); 3.81 at 15 s; 1.27 at 16 s; 5.84 at 17 s; oscillates while carried; set down 10.56 → parks 10.14 from 24 s. Video: rollout_seed006.mp4 (curl-verified 200); rows.json.

Comparability note: the first 15 s are not bit-identical to the parent arm-A row (parent decoded batched at workers=8; this run is batch-1 at workers=1 — the pinned parallel bf16 drift), but the approach shape matches; the parent episode ended mid-carry. Protocol note for future eval-20s: the eval-20 runs above gave 15 s (--replans 15 × 1 s molmoact2 chunks) vs the sim100 protocol’s 30 s — decide the budget in seconds via --episode-seconds and state it in the pre-reg. Spend ~0.02 GPU-h of the ≤0.05 gate.

Amendment 3: 100-episode arm-A eval at 30 s (registered 2026-08-12 20:1xZ, pre-launch)

Owner call 20:16:53Z (“I want to evaluate the release checkpoint with the correct mapping (arm-A) on 100 episodes in parallel”):

• Arm: release checkpoint + the canonical shim (official snippet map exactly: signs 1,-1,1,1,1,1, offsets 0,90,90,0,0,0), arm-A settings verbatim (euler-10, bf16, seam stats = the ftrig rig-recomputed table). Seeds 0–99, parallel driver workers=8, --episode-seconds 30 (900 ticks — the fixed budget; the 20-seed runs above were 15 s).
• Reads (frozen): mean/median progress_final_cm; engagement split (progress_cm > 1 cm count); knock-aways (progress_final ≤ −1 cm); successes; per-seed chart. Grab/carry identification is qualitative from videos (record-only) — the seed-6 lesson is that distance rows under-report grabs (a carried boat set back at spawn reads 0.00).
• Comparisons: within-run + descriptive vs the 20-seed 15 s arm-A rows (record-only: same driver/workers but doubled budget and different batch composition ⇒ bf16-drift class, no pooled claims).
• Gate ≤ 1.5 GPU-h (~50–60 min at the officialmap pace scaled 2×); babysit entry at launch; per-episode --rows-jsonl stream with in-channel updates every ~10 episodes.

Amendment 3 results (2026-08-12 21:1xZ, same session)

9/100 SUCCESSES — the first sim successes this task has ever recorded (all prior evals, ours and the release’s: 0/500, 0/100, 0/20). Success = the sim’s physics criterion (boat resting upright ON the disk, within disk radius, at disk height, still, not held). Seeds 42, 47, 55, 56, 57, 73, 85, 92, 93; final XY center distances 2.1–4.0 cm with base z 12–30 mm and upright 0.97–1.00.

• The time budget was the whole story: every success tick (480–886) lands PAST tick 450, where the old 15 s eval-20 budget ended — under the previous protocol this run scores 0/100. The owner’s 19:25Z catch (fixed as --episode-seconds) is what unlocked the read.
• Distribution: 9 successes · 27 knock-aways (≤ −1 cm, worst −10.9 at seed 28) · 64 quiet; mean progress_final −0.27 cm, median 0.00; 0 reset strikes (validity green); 3 episodes end with the boat tipped.
• Seed 6 repeats its carry-and-return signature (min 1.4 cm, boat set back down near spawn) — grasp events still under-report in distance rows; video remains the grab detector.
• Disposition: the INERT claim is FULLY OVERTURNED. The release checkpoint is task-capable in our sim at ~9% success once BOTH confounds are removed: the shoulder_lift sign (amendment 1) × the halved time budget (amendment 2/3). The remaining ~91% split into knock-aways and non-engagement — a competence gap, not a harness artifact.

Artifacts: rows + chart + the 9 success videos (+ seed 48 carry-return, seed 28 worst knock-away) on the reports Space (rows, curl-verified). Spend ~0.86 GPU-h of the ≤1.5 gate; 51.4 min wall at workers=8, mean batch 7.5. Results in-channel 21:15Z.