forecasting-talent-validationlisted
Install: claude install-skill copyleftdev/superforecasting-skills-pack
# Validating Forecaster Skill on Held-Out Outcomes
Apply the technique with explicit assumptions and evidence limits. Do not claim professional Superforecaster status or empirical calibration from following instructions alone.
## Workflow
1. Define the cohort, question universe, lead times, scoring convention and inclusion rules before ranking.
2. Verify prospective timestamps and resolutions. Audit missing questions, selective participation and model contamination.
3. Separate selection data from a later validation block; keep related outcomes together.
4. Rank or weight candidates using only the selection block, then measure frozen performance on validation questions.
5. Compare matched benchmarks, coverage, uncertainty and domain stability. Interpret a top rank as relative to the evaluated cohort.
6. Report demonstrated performance, untested transfer claims and whether evidence is sufficient; never confer Good Judgment's professional designation.
## Detailed resources
- Read [method](references/method.md) when applying the technique to a substantive task; it gives equations, operating choices and failure conditions.
- Read [worked examples](references/examples.md) for a comparable case or to verify calculations.
- Re