Grasp-SFT bootstrap — the chain, end to end (probe read banked)

2026-08-15, drafted ~09:2xZ during the stage-C ride (pre-reg: grasp-SFT bootstrap); probe section finalized ~13:4xZ, forward pointers ~16:0xZ. Status: the stage-C run was killed at step 2040 on the owner’s order (10:10Z) and the formal stage-D exam is suspended; in its place a two-arm probe of the step-2000 checkpoint ran and is now fully banked — see the probe section below. Headline: 9 → 28/100 on the unseen holdout, ≈3.1× the released base we warm-started from (the owner-agreed primary anchor), with no memorization signature on the training band. A data-pipeline bug found the same morning (corrupt q01/q99 quantile rows — see the ledger) caps this checkpoint’s ceiling; the corrected-table retrain is pre-registered (owner-gated) on our first-class stack (bijou.train) per the owner’s standing decision.

Plain words. Our previous reinforcement-learning experiment ended with an unusual verdict: the robot was too clumsy for rewards to teach it anything. So we went back a step and did what you’d do with a clumsy student — showed it worked examples. This page is the record of that chain: (A) we wrote a scripted “expert” that can solve the task by cheating (it reads the simulator’s exact object positions), and tuned it until it succeeded on held-out scenarios it had never seen; (B) we let that expert perform the task hundreds of times overnight and kept the 313 successful attempts as demonstrations; (C) we fine-tuned the robot’s neural policy on those demonstrations; and (D) the tuned policy sat a 100-scenario exam it had never seen. The result: it now completes the task 28 times in 100 where the model we started from managed 9 — and, reassuringly, it is not just replaying the demonstrations from memory: it does no better (actually slightly worse) on the exact scenarios it studied than on brand-new ones. A separate bug we caught the same morning means the model was trained with a mis-calibrated sense of one wrist joint’s range, so 28 is probably an undercount of what the demonstrations can teach; the corrected re-run is queued behind the owner’s GPU time.

Stage A — a scripted expert that earns its gate

The expert drives the arm from privileged simulator state (true object pose), so its only job is physical competence: grasp the boat, carry it, place it on the disk. Getting there surfaced eight distinct mechanisms, each one a real property of the rig the learned policies also face — the full list is documented in sim/scripted_expert.py, but three carry the story:

• The torque wall is real. Unregularized IK picks straight-arm poses whose gravity moment saturates the sysid’d shoulder servo (force pinned at its 3.478 limit) — the arm literally cannot hold the pose it chose. A nullspace posture pull toward the low-torque basin fixed the solve; the same static-torque wall bounds any learned policy’s low-forward grasps.
• Carry height was capped by the same wall — until the traverse became a pure pan arc: the pan joint’s axis is vertical, so swinging the lifted posture costs no gravity torque and the carry height survives the trip. This was the breakthrough that took the expert from 0 to 10/16 successes in one change.
• Grasping fails by jamming, not by missing. The dominant failure is the moving-jaw shell landing on the boat’s deck and pressing 20–40 N — contact, not gravity (static load is 0.13 of the limit). Physical jam detection with a retreat-and-retry on the mirrored wrist-roll branch recovers most of these.

The registered gate (≥14/20 held seeds) failed first time, 11/20. One amendment was allowed and spent: A1 registered a robustness pass (lower place-droop, re-grasp recovery, a jam-flip budget of 3), in-channel with an objection window, and a fresh held set — seeds 1040–1059, never touched during tuning. It read 15/20 PASS, and the 75% fresh vs 80% burned spread says the pass generalized rather than overfit. Stage A closed: sim hosts the grasp; the old F-physics reading was an expert-coverage gap, not a physics gap.

Stage B — 313 demonstrations, and a small-sample lesson

Collection ran as a detached unit against a 4-hour wall: production visual config, successes only, demo seeds ascending from 1000 (the 100 eval seeds 0–99 can never appear in training data by construction). It banked 313 kept of 486 attempted (64%) — gate ≥300 GREEN, ~4.0 GPU-h.

The chart is also a statistics lesson we’ve now paid for twice: the n=20 gate reads suggested 75–80% expert success, and the first hours of collection seemed to underperform it. A CPU-side n=200 measurement (seeds 1078–1277) put the true rate at 62.5% — the gate reads were ordinary small-sample optimism (a 75% read at n=20 has a CI stretching well below 60%). The §8 record in the pre-reg prices this; the collection itself was never touched mid-run.

Stage C — SFT on the demos, riding the rig-ft r1 recipe

Stage C fine-tunes the MolmoAct2 action expert only — the flow-matching denoising loss is the sole training objective; the VLM trunk (the autoregressive part) is completely frozen (--ft_vlm=false --ft_embedding=none). “AR” in this arm’s name refers to the model family / decode path, not the loss. Recipe: LR 5e-5 on the action-expert params, global batch 64, 3000 steps ≈ 3.5 epochs over the 313 episodes / 54,101 frames — deliberately verbatim-class on the rig-ft r1 recipe that worked before, with a mechanical diff receipt in the launcher header: only the data mixture, run names, and step count differ. (Owner decision 10:07Z 2026-08-15, registered mid-ride: this is the last run on their train_lerobot.py — all subsequent training goes through bijou.train / the first-class stack.)

The curve cleared its one registered reference — materially below the warm-start loss by ~570 steps — with room: 0.464 at step 20 to 0.038 by step ~1000. Note the curves are not comparable in absolute terms (different data, different starting distance from the target behavior); the reference is shape and stability, and both show the same clean settle with no instability at this LR.

The step-2000 probe — what the checkpoint actually learned

The owner’s 10:10Z re-steer replaced the formal exam with a sharper question: does the tuned policy generalize, or does it replay its demonstrations? The probe ran the step-2000 checkpoint (converted from the killed run’s last banked save) under the frozen eval protocol on two arms — the 100 unseen exam seeds, and the first 100 seeds of the demo-collection band, which the banked collection state splits into 64 spawns the policy actually trained on (the scripted expert succeeded there, so those episodes are in the SFT set) and 36 spawns the expert failed on — a free, same-distribution holdout.

Two reads, both banked in reports/analysis__grasp_sft_step2000_probe.json:

• The causal read of the SFT is 9 → 28 ≈ 3.1× on truly unseen seeds. The right comparator is the released checkpoint we warm-started from, which scored 9/100 on the same scenarios under its own intact normalization (owner-corrected framing, agreed 12:0xZ). The ~1–2/100 ftrig4k/W0 rows are different-lineage context, not the baseline. And since the step-2000 checkpoint both trained and served under the corrupt quantile table, 3.1× is a floor on what the demonstration data is worth.
• No memorization signature — if anything the sign is inverted. The policy is worst on the exact spawns it saw demonstrations for (14%) and best on seeds it never saw (28%). At these sample sizes the inversion is ~2 standard errors — suggestive, not proven — but the memorization signature (trained ≫ unseen) is decisively absent, and the sim’s determinism makes this a strong test: a trajectory-replaying policy would ace its own training spawns. One candidate mechanism for the inversion: “kept” spawns are the ones the scripted expert could solve, and scripted-expert-friendly need not be policy-friendly — the bands aren’t difficulty-matched.

Probe ledger: ~3.4 GPU-h vs the 4.0 gate, 0 reset strikes across all 200 episodes, 200 videos banked under outputs/sim/grasp_sft/step2000_probe/.

Stage D — the exam (SUSPENDED by the 10:10Z re-steer)

At the stage-C endpoint the checkpoint is converted (two-hop, carrying the demo-set-recomputed normalization — the same identity frame it trained in, no shim anywhere in the chain) and evaluated on the frozen 100 seeds, sequential driver, euler-10. The decision surface was frozen in the pre-reg before stage A ran:

Context anchors (record-only, not gates): ftrig4k read ~1/100 successes with +0.08 cm mean progress on this exact protocol; the stage-1 W0 arm read 2/100. Those are what “before the bootstrap” looks like. The formal exam never ran: the 10:10Z owner re-steer killed stage C at step 2040 and replaced the exam with the two-arm probe above. And the table’s consequence column has since been re-based: GRPO-R2 Amendment A2 (registered 14:4xZ) moves the R2 activation bar to the token-SFT arm’s discrete-head unseen count — a flow-head sim100 read no longer triggers the GRPO registration, because token-GRPO trains the discrete head that this stage-C run never touched. The ≥20 / 5–19 / <5 surface survives as the frozen verdict on the data for whichever corrected-table flow checkpoint banks next; it just isn’t the R2 trigger anymore.

Where this goes next — three owner decisions pending

Everything below is pre-registered and launch-ready; nothing runs until the owner picks and frees the GPU (theirs since 13:35Z).

1. Retrain arm — corrected-table retrain: continue-from-2k (proposed primary 13:33Z: warm in features, ~2.9 GPU-h, likely better endpoint, muddier attribution) vs from-base (clean table-fix pricing, per the posted draft). Either way the read is against the same comparators: 9/100 base primary, 28/100 corrupt-table floor.
2. Route for the next SFT GPU-hours — A: flow retrain (~5.5 GPU-h, prices the table fix); B: token-SFT arm (~7–8 GPU-h, unlocks token-GRPO via A2); C: one --objective joint run (both heads, confounded read). A and B share the corrected base and don’t block each other.
3. Composition — the --image-augment sim2real flag (landed, oracle-pinned, p=0 bitwise-identical) composes with whichever arm runs: on the retrain directly (one run, two changes, confounds the vs-28 comparison) or as a clean follow-up A/B.

Ledger

• Chain spend, final: ~0.9 (A) + ~4.0 (B) + ~2.7 (C, killed at step 2040) + ~3.4 (probe) ≈ 11 GPU-h vs the pre-registered ≤13 gate.
• Banked artifacts: reports/analysis__grasp_sft_stageA_gate.json, ..._a1.json (+ 20 gate videos under outputs/sim/grasp_sft/stageA_gate_a1/), reports/curve__grasp_sft_stageb_collect.json, reports/curve__grasp_sft_stagec_ar_loss.json, reports/analysis__grasp_sft_step2000_probe.json (+ 200 probe videos), and the step-2000 weights-only delta on fontaine-checkpoints/molmoact2_grasp_sft_stagec_ar_step2000.
• The quantile class bug (lerobot per-episode quantile aggregation: q01/q99 = weighted mean of per-episode quantiles; wrist_roll’s true ±157° box banked as [35.5, 94.4], ~19% of frames clamped) is fixed in collect_demos.rewrite_quantile_stats() with an oracle; the dataset’s stats.json is corrected and re-uploaded. This checkpoint remains trained-on-corrupt — the corrected-table retrain pre-reg (owner-gated) prices the difference.
• Charts regenerate from banked JSONs only: fontaine/scripts/grasp_sft_chain_charts.py.

Addendum 2026-08-16 — route C taken; first verdict is in

The owner picked route C at 00:18Z (GPU freed): one --objective joint run merging the A+B pre-regs, per the registered amendment — L_flow + 1.0·L_CE, flow head insulated (flow grads into the trunk ≡ 0, so the merge preserves both parents’ semantics), from-base with the corrected norm table, made to fit in VRAM by the new --offload-optim (AdamW moments in host RAM, bitwise-exact oracle).

The run completed clean at 06:51Z — 2000 steps, ~5.7 GPU-h, flow loss_action 0.0245, CE loss_aux 0.155 from 4.33. Weights banked: fontaine-checkpoints/molmoact2_grasp_sft_joint_corrected_step2000.

Leg 1 of the endpoint probes (flow head, unseen seeds 0–99, euler-10) landed 08:21Z: 44/100 successes — against the base’s 9/100 and the corrupt-table stage-C AE’s 28/100. The A §5 verdict surface fires TABLE_FIX_POSITIVE outright (44 > 31, the conservative 28+3 clause), so the pre-registered overlap band at 29–31 never comes into play: the corrected lineage becomes the SFT artifact. What the +16 doesn’t yet separate is table fix vs the joint-CE trunk (the confound the amendment accepted going in); the remaining legs (flow-train memorization read, token-unseen vs the R2 bar ≥20, token-base anchor) close out by ~12:3xZ and the consolidated chart-led report follows.