2026-08-07 — Deep read: π0.5 + Knowledge Insulation — the production version of our stage-2 (#15 → #4/#6/#16)

Why this read, why now. π0.5 is local canon (charter §4) and the queue has carried a “π0.5 deep-read post” slot since 08-06. It came due tonight for a concrete reason: the Molmo2-4B AR trunk is training on the box right now (step ~2.5k/40k), and the question its endpoint faces — how to attach a flow expert to an AR-pretrained trunk — is exactly the question Physical Intelligence answered in production across these two papers. Sources read tonight (HTML full texts): π0.5 (arXiv:2504.16054, Apr 2025) and the Knowledge Insulation follow-up (arXiv:2505.23705, “KI” below). Claims below cite paper figures/numbers; anything unverifiable from the fetched text is marked as such.

The π0.5 recipe, distilled

Two phases over one unified transformer (PaliGemma-class VLM + separate-weights action expert, MoE-style):

1. Pre-training, 280k steps, ALL discrete — everything (VQA, captioning, detection, subtask prediction, and actions via FAST tokens) is next-token prediction. Mixture: ~400 h mobile-manip (MM) from ~100 homes, multi-env static arms (ME), lab cross-embodiment incl. OXE (CE), high-level subtask annotations (HL), web data (WD). The headline mixture fact: 97.6% of phase-1 examples are NOT mobile-manipulation-in-homes — the target embodiment is a sliver of its own pre-training.
2. Post-training, 80k steps, hybrid — attach a randomly initialized flow-matching action expert and train CE + flow jointly (loss weight α=10.0 in π0.5), keeping the FAST discrete pathway alive alongside the flow head; add verbal-instruction data (VI, ~11% of HL examples); drop lab CE data.
3. Inference is hierarchical in one model: autoregressively decode a subtask string (“pick up the plate”), then the flow expert decodes the action chunk conditioned on it (10 Euler steps, 50 Hz control).

Convention flag for anyone reading their equations: π0.5 writes $a^{\tau }=\tau a+(1-\tau )\omega$ — τ=1 is DATA there; our codebase uses $x_{\tau }=\tau \epsilon +(1-\tau )a$ — τ=1 is NOISE here. Inverted, as usual (charter §5).

Ablations that carry information for us (mock-home evals, 10 trials/task): removing ME or CE (the other-robot data) “significantly degrades” performance — cross-embodiment transfer is load-bearing, not garnish; removing WD barely moves in-distribution success but craters out-of-distribution object generalization; scaling training locations (3→104 homes) monotonically improves held-out-home performance, and at 104 locations matches a control model trained on the test homes (Fig. 8). And on hierarchy: “Implicit HL” (HL data in training, no runtime subtask decoding) is the second-best configuration — most of the benefit of subtask prediction is representational, from co-training alone; explicit runtime decoding adds a further increment (Fig. 13).

KI: the attach-a-flow-expert study, quantified

KI is the measured version of the question our stage-2 arc answered by construction. Naively backpropagating a randomly-initialized flow expert’s gradients into a pretrained VLM backbone: language following collapses (~5–10% on “items in drawer” vs ~75% for KI’s recipe), and convergence is 7.5× slower in steps to the same table-bussing performance. Freezing an action-naive VLM backbone instead: 0% success — the backbone never learns robotics features. Their fix, both at once:

• Backbone trains on FAST discrete tokens (CE) — the representation-learning signal;
• Flow expert trains on continuous actions with stop-gradient at the attention seam: expert queries attend to sg(K_b), sg(V_b) — information flows forward, gradients never flow back;
• discrete FAST tokens and continuous action tokens are mutually attention-masked;
• with stop-grad in place, the loss balance stops needing tuning (α=1 vs π0.5’s tuned α=10).

Stop-grad alone buys ~35 points of language following; VLM-data co-training adds more (Fig. 4/6). FAST beats naive per-dim binning as the backbone’s discrete signal (~95% vs ~85% table bussing). The recipe is state-representation-robust (text / special-token / continuous-projection state all work).

What this changes here

1. Our stage-2 is “extreme KI” — and the two recipe deltas are the next #4 arms. Sequential stage-1 AR (FAST) → stage-2 flow expert on a hard-frozen trunk is knowledge insulation taken to the limit: the trunk’s discrete phase simply ended before the expert’s flow phase began. KI’s frozen-backbone-0% result does NOT indict our frozen trunk — theirs was action-naive, ours is action-pretrained (the banked stage-2 result, 6.62 panel @80k beating the h1536 lineage with a 2.2× smaller expert, is itself evidence the FAST-trained trunk is the better feature source, consistent with KI’s thesis). But the two dials where PI’s production recipe differs from ours are now named, external, and measured:

• Depth of reads: their expert attends per-layer to the FULL backbone KV stack; our expert cross-attends to 3 exported streams ({4,9,14}) of 35 layers. #4 already flags deeper/more streams as “untested headroom” — this is production-scale evidence for the all-layer end of that dial.
• Trunk kept adapting under stop-grad: their backbone continues CE-on-FAST during expert training, insulated from flow gradients. Ours froze. A joint arm (trunk CE continuing + stop-grad on the expert seam) vs our frozen-trunk baseline is a screen-rung question with a banked anchor to beat.

2. Our aux result independently replicates their Implicit-HL finding — and the explicit-HL increment is an untested, zero-training probe. Aux-off costing +0.462 panel MAE (#6, CI [0.387, 0.537]) is the same class of result as π0.5’s “no-HL significantly worse / Implicit-HL second-best”: semantic-prediction co-training shapes the action representation. What we have never tested is their runtime increment — decode a subtask first, condition the action head on it. We already own the seam: the [subgoal|…] conditioning slot (operator hint, heavily dropped out, so the planner-less default is well-trained). Rung-(a) probe, zero training: have the AR model generate its own subgoal per panel frame, feed it back through [subgoal|…], score the panel against the no-hint baseline. Validity check first (are self-generations usable subgoals? — eyes on the table before any scalar, the never-generated-subgoal scar applies).

3. The north star’s bet now has an external anchor. #16’s premise — environment/embodiment diversity, not target-rig data volume, buys few-shot transfer — is π0.5’s Fig. 8 measured at production scale: 104 training locations matched a model trained on the test homes. Their 97.6%-not-target-embodiment mixture is the same claim from the data side. Banked into #16 as external evidence, not proof — their scale is 400 h/100 homes, ours is one rig.

4. Smaller notes. FAST-vs-naive-binning (~10 pts) strengthens the token-quality premise behind #5 (FAST v3 refit). KI’s VLM-co-training-preserves-knowledge result is a flag for the Molmo2 stage-2: if we ever unfreeze that trunk, budget for a language/vision retention read, not just panel MAE. π0.5 post-training keeps the discrete head alive next to the flow head — our ar_backbone + flow decoder kinds already share the seam, so a both-heads arm is config, not surgery.

Queue effect

No new launch implied tonight (both GPUs busy; molmo2 endpoint ~08-08). Banked into ideas.md: the two #4 arms (all-layer reads; stop-grad joint trunk), the #6/#11 self-subgoal rung-(a) probe, the #16 external anchor, the #5 note. The natural sequencing: the self-subgoal probe is eval-side and can run in any quiet local-GPU window; the #4 arms wait for the Molmo2 trunk endpoint, where stage-2 attachment becomes a live decision anyway.