HF evacuation audit: the schema-2 fleet is fully recoverable — zero weight gaps
2026-08-19 11:1xZ (audit run 11:08–11:16Z). Closes queue item
hf-evacuation-audit-v2-fleet (standing rule: upload valuable
checkpoints; follows v1-fleet-upgrade retiring all six
local schema-1 originals in b1d1b27). Method: local sha256sum
against fontaine-checkpoints LFS sha256s (343 files enumerated),
plus two live converter re-runs to prove the extraction claims
bitwise. CPU + network only, 0 GPU-h.
Plain words. This morning we deleted the last old-format copies of
every model we care about, keeping only the new-format (_v2)
directories on the local disk. This audit asks the scary question:
if this machine died tonight, could every one of those six models be
rebuilt? The answer is yes, verified — not by trusting notes, but by
checksum. For each model, either its source is already banked
bit-for-bit in our HuggingFace vault, or it can be regenerated by a
frozen converter script from a public artifact — and we re-ran the
two regenerations that had never been proven, confirming the rebuilt
weights are byte-identical to what’s on disk. Nothing needed
uploading; the vault was already sufficient.
The rule being audited
A banked source + a committed deterministic converter counts as a
recovery path (bijou.convert_v1, bijou.convert_legacy, and
bijou.convert_molmoact2 are frozen readers: same source →
content-identical output). The audit therefore accepts three path
shapes: the _v2 itself on HF, a v1/legacy source on HF, or a
converter chain rooted in a recoverable artifact.
The mapping
Local _v2 (size) | Recovery path | Verification |
|---|---|---|
converted/er_60k_step_060000_vla_v2 (9.1G) | HF fontaine_molmo2_er_60k_ddp4/step_060000/ (complete format-3 trainer ckpt) → convert_legacy | backbone + expert + prompt all sha256 bitwise = HF LFS; banked config has aux_loss_weight: 0.5, and the committed pre-rename fallback reproduces narration_weight 0.5 (this morning’s regeneration fix survives recovery) |
finetune/fontaine_grasp_sft_joint_corrected/step_002000_v2 (12.2G) | HF molmoact2_grasp_sft_joint_corrected_step2000/ (schema-1 v1 dir, schema_version: 1 confirmed) → convert_v1 | flow_decoder sha256 bitwise = HF LFS (d9be44ce…); v1 backbone + metadata + train_log all banked |
converted/molmoact2_base_corrected_stats_v0_vla_v2 (19G) | public allenai/MolmoAct2-SO100_101 → convert_molmoact2 --family molmoact2_flow + corrected stats (committed build_corrected_norm_stats.py, donor = the public repo’s own norm_stats.json) | live re-run 11:14Z: re-extracted flow_decoder sha256 4517d649… bitwise equal to the on-disk expert |
converted/…_vla_jointsurface_v2 (19G) | same source, --family molmoact2_joint (the release trained both heads) | same expert inode as above (hard-link count 3) — covered by the same re-run |
converted/molmoact2_grasp_sft_stagec_ar_step2000_vla_v2 (19G) | public base + HF molmoact2_grasp_sft_stagec_ar_step2000/model-delta.safetensors overlay (README on HF documents it; upload oracle was green at f51eebc) → stage-C export → convert_molmoact2 | delta sha256 bitwise: local staging = HF LFS (98f32f4d…); live re-run 11:16Z: re-extracted expert sha256 d840174d… bitwise equal to the on-disk expert |
converted/…_corrected_v1_v2 (21G) | same chain + corrected stats table | its flow_decoder hashed bitwise equal to the stage-C expert (d840174d…) — it is a metadata-only variant (substituted stats, stats_note recorded), so the same re-run covers it |
The two legacy expert-source dirs
converted/molmoact2_base_corrected_stats_v0/ and
converted/molmoact2_grasp_sft_stagec_ar_step2000/ (format-3
bijou_config + expert.safetensors, 2.2G each) were flagged for
inclusion before any retirement decision. Verdict: both experts are
now proven byte-derivable from recoverable artifacts (the two live
re-runs above are exactly their contents), so the dirs hold no unique
bytes. Safe to retire whenever disk pressure demands; no upload
needed first.
Caveat worth stating
Three of the six paths root in the public allenai/ MolmoAct2-SO100_101 repo (base trunk + expert; the stage-C delta
overlays onto its shards). That is an external dependency: if allenai
ever pulled the repo, the base trunk would survive only on this box
(local snapshot, 21G). The standing dedup rule (deltas vs released,
never re-uploading public bytes) accepts this trade; flagged here so
the acceptance is a recorded decision, not an oversight.
Net result: zero gaps, zero uploads required. The pristine 17.7G
trunk part-files inside the _v2 dirs never need banking — they are
converter extractions, which is the point of the conversion-first
design.