prepare-training-corpuslisted
Install: claude install-skill bastos/skills
# Prepare Training Corpus
Make the corpus reproducible before any expensive labeling or training call.
## Define the contract
Write down:
- the population and approved data sources;
- the unit of an example and the group that must never cross splits;
- target split sizes and distributions;
- input, output, abstention, and identifier schemas;
- required positive, negative, ambiguous, and boundary cases;
- leakage rules for prompts, labels, canonical IDs, and teacher-only metadata.
Separate deterministic scenario generation from teacher labeling. Materialize and validate the entire candidate pool without model calls first.
## Materialize deterministically
1. Pin source snapshots and generator revision.
2. Set and record the random seed.
3. Assign splits by stable group identity, not by individual row.
4. Write a selection manifest before calling a teacher or evaluating a model.
5. Keep opaque request-local identifiers in model-visible inputs when canonical IDs would leak answers.
6. Store teacher/reference fields outside the model prompt.
Do not silently repair generated or teacher-produced rows. Preserve raw failures and derive validated outputs separately.
## Validate and freeze
Use `scripts/build_corpus_manifest.py` for JSONL split invariants:
```sh
python scripts/build_corpus_manifest.py \
--split train=path/train.jsonl \
--split validation=path/valid.jsonl \
--split test=path/test.jsonl \
--id-field id --group-field deckGroupID \
--output path/corpus-m