[yliu382]

Summary

Adds examples/cobol-cpt-grpo, a production-shaped example that adapts Qwen/Qwen2.5-Coder-32B to COBOL on 2 x 8 H200 with RDMA. It has seven stages, each with its own tau.yaml:

• 01-collect, 02-data-prep, 03-benchmarks, 05-rl-prompts: CPU Jobs that collect the corpus, prepare it, build the benchmarks and the RL prompts.
• 04-cpt: continued pretraining as a RayJob. A Ray Train TorchTrainer with 16 workers runs HF Trainer with DeepSpeed ZeRO-3.
• 06-grpo: GRPO in one RayJob. A trl vllm-serve actor (TP=8) runs on one worker and a TorchTrainer with 8 workers on the other. The reward is a GnuCOBOL compile reward plus a structural reward.
• 07-eval: a RayJob on one 8-GPU worker. It runs five suites with vLLM and compiles every generated program with cobc (free format, then fixed format).

Data collection defaults to SOURCE_LICENSES=permissive, which takes only MIT, Apache-2.0, BSD, CC-BY-4.0 and US-government repositories. Copyleft or unlicensed sources need SOURCE_LICENSES=all. Gated Hugging Face datasets are opt-in through an hf-token Secret. The README lists every source and its license.

The training image, workspace, PVC, output root and topology are placeholders, listed in the README's "Substitute placeholders" table. No image is published. The Dockerfile and pinned image-requirements.txt are included so users can build their own.

The example is also listed in examples/README.md.

Related issue

None.

Validation

python3 scripts/check-license-headers.py                         passed (887 files)
cd core && go test -run TestShippedExampleConfigsWarnNothing ./runconfig   ok
tau run --config <each of the 9 tau.yaml> --dry-run=client       all render
                                                                 (against a cluster profile copy)
06-grpo: pytest test_reward.py                                   94 passed
06-grpo: pytest test_driver.py                                   passed (local Ray)
04-cpt:  pytest test_continued_pretrain.py test_driver.py        passed
07-eval: pytest test_driver.py                                   passed
ruff check (SDK rules)                                           clean

The pipeline was run end to end on 2 x 8 H200 with the same code, launched with tau run. Results are in the README:

• The published GRPO checkpoint scores 64/220 compile on the 220 benchmark tasks, and the 07-eval RayJob reproduces every metric exactly.
• A full rerun with CPT, GRPO and eval all launched as RayJobs scores 57/220, with the RayJob CPT checkpoint at 53/220.
• The Job-based CPT checkpoint scores 58/220 on the same eval harness, so the gap comes from CPT. The README says the cause is still being investigated.

The real-cluster e2e suite was not run.

Compatibility and operational impact

None. This adds a new example folder and one index row; no APIs, CLI, charts or CRDs change.

The CPU stages use python:3.11.17-bookworm and the training image builds from vllm/vllm-openai:v0.28.0, both from Docker Hub and pinned by digest. The README documents these as exceptions to the MCR base-image preference.

Checklist

• The change is focused and does not include unrelated edits.
• Tests cover the changed behavior, or I explained why tests are not needed.
• Documentation and examples are updated when public behavior changes.
• Generated files are updated from their source and included in this pull request. (Not applicable: no generated files.)
• New dependencies are publicly available and their license and provenance were reviewed.
• The diff contains no secrets, private endpoints, customer data, or Microsoft-internal information.
• Fork-based CI can validate the change without repository secrets, or maintainer-only validation is clearly identified. (GPU runs are maintainer-only, as described above.)

🤖 Generated with Claude Code

[Kevin Cho (chokevin)]

Blocking issues found in the closeout review. Please address these before merging.

[Kevin Cho (chokevin)]

This version is affected by GHSA-vxj4-q4g9-crj7 and is currently failing the dependency-review check. Please update virtualenv to 21.14.7 or newer before merging.

[yliu382]

Bumped to 21.14.7 in 9f4264a. A second advisory, GHSA-c978-59m8-c77v (published 2026-10-21, first patched in 21.14.9), then failed dependency review again, so 7fecbf0 pins virtualenv==21.14.9. Its requirements are identical to 21.14.7's, so the other pins are unchanged.

[Kevin Cho (chokevin)]

This makes the stage succeed even when every required source fails, and the same pattern appears in collect-rl-data.sh. Please track failed required steps and exit nonzero, and fail if collection produces no usable records; otherwise a partial or empty corpus can be published as a successful stage.

[yliu382]

Fixed in 9f4264a. Both collect-cobol-data.sh and collect-rl-data.sh now record every required step that fails and, at the end, exit 1 with the list of failed steps. They also exit 1 if collection produced no COBOL source files.

This surfaced a real problem on the rerun: the X-COBOL and NIST download URLs had gone stale, and the stage failed instead of publishing a partial corpus. 037e258 moves both to current URLs with sha256 checks.

[Kevin Cho (chokevin)]

The membership check is case-insensitive, but this split is case-sensitive. For inputs containing Procedure Division or other casing, the split returns the entire program and leaks the reference procedure body into the prompt. Please find the boundary case-insensitively and slice the original text; the copybook builder has the same issue.

[yliu382]

Fixed in 9f4264a. text_before_procedure_division() finds the header with a case-insensitive regex and slices the original text. Both the paragraph-completion builder and the copybook builder use it. 03-benchmarks/test_build_benchmarks.py covers PROCEDURE DIVISION, Procedure Division, and procedure division, and checks that the reference body never appears in the prompt.

[Kevin Cho (chokevin)]

Stage 03 also generates copybook_usage.jsonl, but this evaluator never loads or scores it. Please either evaluate that suite here or remove it from the generated benchmark set and the claim that the pipeline evaluates all five suites.

[yliu382]

Fixed in 9f4264a. copybook_usage is now a required suite: it is loaded, checked in preflight, and scored with BLEU and edit distance. The README lists what each of the five suites reports. The rerun scored 76 copybook tasks.

Related follow-up in cdb4c8b: 2 copybook prompts and 1 paragraph prompt from the new corpus are longer than the 32768-token context. The eval now sets those aside before generation, lists them under excluded_over_context in eval-summary.json, and fails if any suite is left empty.

[Kevin Cho (chokevin)]

Stage 05 records whether each program requires free or fixed-format compilation, but that field is dropped here and preflight later always uses cobc -free. This silently removes every fixed-only training example. Please preserve compile_fmt through loading and use it during preflight/reward compilation, or normalize the records before writing them.

[yliu382]

Fixed in 9f4264a. Prompt loading keeps compile_fmt from each stage 05 record. Preflight compiles each prompt in its own format, and reward_fn receives compile_fmt as a dataset column, so each completion is compiled with -free or -fixed to match its prompt. test_reward.py covers preflight, reward compilation, and records without the column, which default to free.

On the rerun, preflight kept 1,762 prompts: 918 free and 844 fixed. Previously every fixed-only record was dropped.

[Kevin Cho (chokevin)]

Missing benchmark artifacts are treated as empty suites, so evaluation can write a successful summary without meaningful results. The compiler path similarly counts a missing cobc as skipped samples. Please preflight the required nonempty benchmark files and cobc --version, and fail the task when either contract is unmet.

[yliu382]

Fixed in 9f4264a. Before loading the model, main() runs cobc --version and checks that all five suite files exist and are not empty. If either check fails, it fails with the list of problems. If cobc disappears during the run, compilation raises instead of counting the sample as skipped. test_run_eval.py covers these cases, including that the model is never loaded when preflight fails.

cdb4c8b adds one more preflight check: generation prompts that cannot fit in max_model_len with their output budget are set aside and recorded in eval-summary.json. The run fails if a suite is left empty. That's 3 of 448 tasks on the new corpus.

[Pengfei Ni (feiskyer)]

The GRPO stage can reuse held-out evaluation inputs and stop only one distributed rank.

[Pengfei Ni (feiskyer)]

Stage 03 builds evaluation tasks from val.jsonl, but this stage rereads the complete raw source index without excluding that partition. A held-out program passing the compile/body/token filters can therefore supply a GRPO training prompt, including the procedure prefix used by the evaluation tasks. Build RL prompts from the training partition or exclude validation source identities/content hashes, and assert the sets are disjoint.

[yliu382]

Fixed in 9f4264a and 0677658. Stage 05 now reads val.jsonl from 02-data-prep and drops any source program that shares 8 consecutive normalized lines with a validation program. Normalization ignores case, whitespace, and blank lines. The check runs on both the raw text and the text after EXEC/COPY stripping. I used content windows rather than source identities because the raw index contains renamed copies of the same program under different paths. After filtering, assert_disjoint_from_val fails the stage if any emitted prompt still overlaps.

On the rerun with a fresh corpus, 1,285 candidate programs were excluded and the 2,057 emitted prompts passed the assertion. The first attempt hit the assertion on 4 prompts that only overlapped after COPY stripping, which is why 0677658 also checks the stripped text. The README marks the earlier GRPO results as partly contaminated.

[Pengfei Ni (feiskyer)]

Only rank 0 installs this callback. When either alarm fires, the pinned Transformers/TRL loop reads should_training_stop locally without broadcasting it: rank 0 leaves training while the other workers enter their next distributed operation, hanging or failing the multi-GPU run. Synchronize the alarm decision on every rank before any rank leaves the loop; keep metrics publishing rank-zero-only.

[yliu382]

Fixed in 9f4264a. TauGRPOMetricsCallback is installed on every rank, and each rank evaluates the alarms. The stop decision is all-reduced with MAX, so if any rank raises an alarm, every rank sets should_training_stop at the same step. Only rank 0 publishes metrics.

06-grpo/test_stop_sync.py starts two gloo ranks, raises the KL alarm only on rank 1, and checks that both ranks stop at the same step. The 2-node GRPO rerun completed all 162 steps.

[Pengfei Ni (feiskyer)]

For a multi-paragraph program whose selected paragraph calls a later paragraph, stage 03 discards that later definition. This compiles only the prefix plus generated paragraph, so even the exact reference completion fails with an unresolved PERFORM target. The paragraph and overall compile scores then include failures caused by missing benchmark context. Preserve the untouched suffix in each task and append it for compilation while scoring only the generated paragraph; add a regression where the reference completion must compile.

[yliu382]

Thanks, confirmed. Paragraph tasks dropped every later paragraph, so a reference that PERFORMs one couldn't compile; stop_at was recorded but unused. Fixed in 85bd4bf:

• Stage 03 now stores the rest of the procedure as suffix.
• The eval cuts each paragraph completion at the next paragraph header and compiles prompt + completion + suffix.

Each compile suite now also reports how many tasks compile with the reference (reference_compiled) and with an empty generation (empty_compiled, 114a7a0). The paragraph reference count went from 16/61 to 30/61. An empty generation compiles on 31/61 paragraph, 34/50 program and 68/100 structural tasks, more than any model, so the README results (f64b18b) now lead with BLEU, edit distance and the structural checks.

[Kevin Cho (chokevin)]

Please remove examples/cobol-cpt-grpo/03-benchmarks/__pycache__/build_benchmarks.cpython-312.pyc before merging. This is generated Python bytecode and should not be committed; keep the source and tests, but remove the cache artifact.

[yliu382]

Kevin Cho (@chokevin) Removed the bytecode file in 2300002, and __pycache__/ and *.pyc are now in the root .gitignore (d2cc73e) so it cannot be committed again.

[Kevin Cho (chokevin)]

Verified the requested cleanup: the committed Python bytecode has been removed, and .gitignore now excludes pycache/ and *.pyc. The only changes since the reviewed revision are this cleanup and the ignore rules. No further blocking requests. Reviewed with GitHub Copilot assistance.

[Pengfei Ni (feiskyer)]

LGTM