Round 3 (s03) build: diagnosis, data, notebooks, checks
Built 2026-10-04 (WAT) from eval/results/round3-plan-from-s02.md (commit 360a9cb). Scripts:
scripts/build_round3.py (data), scripts/make_notebooks.py (training notebook), scripts/make_eval_notebook.py
(eval notebook), scripts/dryrun_s03.py (CPU dry run). Reports: data/built/reports/r3_*.json.
Diagnosis from the s02 samples (Oviezibe/ren-evals/s02-vs-base/)
| symptom | finding | fix |
|---|---|---|
| HumanEval+ 0.706 vs base 0.804 (23 lost / 7 gained) | None of the 23 losses is a format failure. Every one has no prose before the code, no think leak, the right signature, a closed fence and no truncation. They are careless implementations: 18 AssertionError (edge cases, < vs <=), 3 timeouts (naive recursion), 1 ValueError, 1 MemoryError. Likely cause: the code targets come from another model (OpenCodeInstruct = Qwen2.5-Coder-32B), and SFT on them plus ~70% non-code data shifted the base's own coding behaviour. |
On-policy self-distillation (notebook phase 0). The base model (adapter off) answers 1,600 unseen OpenCodeInstruct prompts in exactly the training rendering. Their unit tests run in the sandbox, and only passing answers are trained, normalised to one ```python block. There is no teacher fallback. Those rows get weight 0.24, plus 0.10 of v1's execution-verified code_only rows. Prose+code groups stay at 0. 96 held-out pool problems are scored again after training (code_check.json). |
| Agent format 0.88 vs 0.96 | Qwen/Qwen3.5-9B ships no generation_config, so HF generate() stopped only at `< |
endoftext |
| Interruption stop 10/14 | All 4 failures were a correct "Stopped..." followed by a made-up turn. Cut at the turn end: 14/14 (base 12/14). | The same stop fix. Also 447 extra tool-aware stop records (weight 0.03). |
Caveat: s03-vs-base numbers are not directly comparable with s02-vs-base. The stop rule changes the base and the adapter runs alike, so the comparison stays paired.
New data: revise finished work, then resume
datagen/progdata/interruption_r3.py builds 900 records. Every step is executed in the bwrap sandbox in transcript order,
and the final file state is checked.
revise_part(410): a 2–3 part terminal request. Mid-way through the last part the user corrects part 1, which is already finished. The agent redoes part 1, verifies it, says "Part 1 is fixed... Back to part N", and resumes without redoing finished steps. Checks: the corrected part-1 checks and the other parts' checks pass; the original part-1 checks fail.migrate_csv(249): a;→,migration file by file. The user reports that line ~N (up to ~1600) of an already-migrated file split a quoted"Last; First". The agent compares the original and migrated line, then fixes the line in place or redoes the file quote-aware. It greps every migrated file, then resumes with the quote-aware command. Check: sha256 of every expected file.migrate_rename(241): anopen_db→connect_dbrename. The substring sed also hitopen_db_v2at a named line of a finished module. The agent fixes that line, verifies with grep pluspython3 -c 'import ...', and resumes with a word-bounded sed.
These records carry train_turns="all_after_interrupt". ren.formats therefore trains every assistant turn after
the interrupt (acknowledge+fix, verify, resume, final), not just the first and last.
ren-data-v2 (private: Oviezibe/ren-data-v2 @ 558471996f58845bfa548ecdb6344a98faa1b863)
22,065 records = ren-data-v1 (18,039) + the round-3 packs. Admission rules for new records:
- Dropped if they hit a heldout_v1 prompt or user key, or share a 13-gram with HumanEval+ or held-out MMLU-Pro.
- Programmatic records dropped if their user turns duplicate v1.
- Dropped if the upstream id is already in v1.
| new group | records | notes |
|---|---|---|
| open:general/code_selfdistill | 1,820 | OCI shards 5-7 (not read before). Upstream tests all pass and judge ≥ 4; reference re-executed. 1,426 stub / 474 task. Prompt pool only (weight 0); the notebook generates the targets. |
| programmatic:terminal_first_action_r3 | 581 | 1,150 picked; 532 dup-v1 and 37 eval-key drops. Filtered as full trajectories, cut at package time. |
| programmatic:terminal_task_r3 | 278 | fresh executed tasks (seed 20261201) |
| programmatic:interruption_stop_r3 | 447 | 25% are explicit go-ahead contrast records |
| programmatic:interruption_revise_r3 | 900 | above |
Licence bases: no Llama-generated, OpenAI, Anthropic, eval-only or mock rows. The phase-0 rows are Qwen3.5-9B
(apache-2.0) outputs on cc-by-4.0 prompts. Round-3 seeds 20261201-4 are in heldout.TRAIN_SEEDS_R3.
Overlap (r3_overlap_report.json, the eval's own check over all v2 shards plus templates): disjoint. 0 overlapping
items in format, decide, terminal and interruption; 0 new HumanEval+ drops; 0 MMLU-Pro drops.
Mix (ROUND3_MIX) and the CPU dry run on real v2
The dry run uses a mock tokenizer at 3.6 chars/token. Phase 0 ran on CPU with the reference answers: 1,597/1,600 passed in the sandbox.
| area | groups (weight) | dry-run share |
|---|---|---|
| coding | selfdistill:code 0.24, code_only 0.10 | 33.9% |
| agent | first_action_r3 0.05, first_action_r2 0.01, terminal r3/r2/v0 0.03/0.03/0.02, open terminal 0.03, tool_use 0.03, swe 0.02 | 22.2% |
| interruption | revise_r3 0.09, interrupt_r2 0.06, stop_r3 0.03, interruption_dialogue 0.02, open_interruption 0.02 | 22.0% |
| format | template 0.07, web 0.02, safety 0.04 | 13.0% |
| decide | typed_decision 0.09 | 9.0% |
The dry run produced 5,984 rows and 2.41M tokens, which is 184 optimizer steps. Problems: none. That includes every contract check: strict first-action rows are exactly one call; stop rows have no tool call; revise rows have one call per tool turn and resume targets; phase-0 targets are a single block; held-out problems are not trained.
Training setup
- Start point: continue from the s02 adapter (
INIT_ADAPTER = Oviezibe/ren-s02 @ 51de9ea, r=16/alpha=16, loaded into the same LoRA with a missing/unexpected-key check). - Why not from the base: it keeps the interruption and Decide gains, needs half the tokens, and leaves quota for a retry. Phase 0 still samples the base (adapter off), so the code targets are the base's own style.
- Settings: lr 3e-5 (s02: 5e-5), budget 2.4M tokens,
max_hours=4.5. - From-base fallback: set
INIT_ADAPTER=None, budget 4.9M and lr 5e-5.
Kaggle
| item | path / setting |
|---|---|
| Training notebooks | /workspace/kaggle_upload/ren_s03_smoke.ipynb, then ren_s03_full.ipynb (from notebooks/kaggle_s03_round3_sft.ipynb) |
| Adapter repos (private) | smoke → Oviezibe/ren-s03-smoke; full → Oviezibe/ren-s03 |
| Eval notebooks | /workspace/kaggle_upload/ren_eval_s03_smoke.ipynb, then ren_eval_s03_full.ipynb |
| Eval results | Oviezibe/ren-evals/s03-vs-base/ (/smoke in smoke mode) |
Session estimate: install and model about 25 min, phase 0 ≤ 60 min (cap), training 3.1-3.8 h at 214-175 tok/s, final evals about 30 min. Total about 5-6 h.
Quota:
- Kaggle's weekly GPU quota resets Saturday 00:00 UTC = Sat 01:00 WAT.
- Used since the Sat Oct 3 reset: about 12-13 h (Oct 3 pilot evals, s02 smoke, train and eval), so about 17 h remain until Sat Oct 10 01:00 WAT.
- Planned: s03 smoke ~1 h + full ~5.5-6 h + eval smoke ~0.5 h + eval full ~1.5-2 h ≈ 9 h, leaving about 8 h of margin.
- Not enough margin for a from-base rerun plus eval (~10 h) on top.
Risks
INIT_ADAPTERloading (key names when the s02 state dict goes into a fresh PEFT model) is tested only on CPU logic. The notebook stops on missing or unexpected LoRA keys, so a mismatch fails fast in the smoke run.- Phase-0 time and yield are estimates. If base pass rate × generated count falls short, selfdistill underfills its
0.24 share and the remainder goes to the other groups (logged in
selfdistill_stats.json). - The s03 eval numbers are not comparable with the s02 report (see the stop fix above).
open:general/safety(Nemotron safety rows) is Mixtral-generated: kept as before, flagged.- Smoke/full eval upload copies for s02 and the pilot are no longer rewritten by default (
--s02-uploads,--pilot-uploads).