GPT-OSS-Lite Pretraining Workflow

GPT-OSS-Lite Pretraining Workflow A workflow diagram generated by Archify. 01 / Micro-batch 02 / Optimizer clock EX / Recovery / exit Fetch Micro-batch · 8 × 4096 windows · Micro-batch Fetch Micro-batch 8 × 4096 windows Forward + CE · (CE + .01×aux) / 4 · Micro-batch Forward + CE (CE + .01×aux) / 4 Finite Loss? · before backward · Micro-batch Finite Loss? before backward Backward · micro_step += 1 · Micro-batch Backward micro_step += 1 Periodic Save · every 2000 updates · Optimizer clock Periodic Save every 2000 updates Cadence / Stop · step < 61000? · Optimizer clock Cadence / Stop step < 61000? Optimizer Commit · clip · AdamW · LR · step++ · Optimizer clock Optimizer Commit clip · AdamW · LR · step++ 4 Micros Done? · micro_step % 4 == 0 · Optimizer clock 4 Micros Done? micro_step % 4 == 0 Final Save + RNG · no sampler position · Recovery / exit Final Save + RNG no sampler position Discard / Restore · clear grads + micro count · Recovery / exit Discard / Restore clear grads + micro count Abort · 5 failures, no checkpoint · Recovery / exit Abort 5 failures, no checkpoint yes 61000 reached yes scaled loss no: next micro 5 + no checkpoint no 2000 due skip / restore latest continue continue Legend Agent logic Policy Context / trace

Two clocks

  • • 4 finite micro-batches per commit: 131072 configured tokens/update.
  • • CE is computed in 8192-token chunks before finite-loss testing.

Persistence limits

  • • Periodic: model, optimizer, scheduler, metadata. Final adds RNG separately.
  • • Latest-complete checks model/optimizer/meta, not scheduler or RNG.
  • • Rollback restores training state, not RNG or loader position.

Source / configured baseline

  • • Revision 3c9675deaf16
  • • configs/pretrain_a100_502m.yaml
  • • ModelConfig defaults · no CLI overrides