adjudication-sheets
SolidBuild human adjudication / hand-labeling sheets from LLM-pipeline data without evidence truncation. Use when: (1) preparing a CSV/Excel sheet for a human to rule on cases an LLM classifier or rater panel judged, (2) a labeler reports "there is no information to label from" or cells look empty in Excel, (3) excerpt columns cluster at one exact length (e.g. all 1,500 chars — a hard truncation cap). Covers: full rating-basis recovery, Excel 32,767-char cell cap, multi-line CSV mangling, ruling dropdowns, companion text files.
Install
Quality Score: 90/100
Skill Content
Details
- Author
- kennethkhoocy
- Repository
- kennethkhoocy/applied-micro-skills
- Created
- 6 days ago
- Last Updated
- 5 days ago
- Language
- Python
- License
- MIT
Similar Skills
Semantically similar based on skill content — not just same category
llm-as-judge-scorer
Design a reliable LLM-as-judge metric — a calibrated rubric, a clear scoring scale, and bias controls — and validate it against human labels before trusting it. Use when grading open-ended LLM output (summaries, answers, tone) that exact-match can't score.
judge-calibration-auditor
Iterate-stage skill: analyzes human labels vs LLM-judge verdicts and turns every disagreement into a classified calibration signal with a stated correction — never auto-resolved toward the judge. Use when a judge and humans diverge — 'our judge disagrees with human reviewers', 'human labels vs judge verdicts, what's drifting', 'the judge scores everything 4', 'disagreement analysis' — or when /pm routes such a request here. Do NOT use to design the judge (llm-as-judge-designer), to build the eval (eval-engine), to produce the human labels themselves, or for judge-bias knowledge questions.
llm-as-judge-designer
Iterate-stage skill: turns existing eval criteria into an LLM judge prompt with anchors and few-shot calibration cases — every rubric point carrying both a pass and a fail exemplar. Use when criteria exist and the judge needs building — 'write the judge prompt for these criteria', 'turn this rubric into an LLM judge', 'judge prompt plus calibration cases' — or when /pm routes such a request here. Do NOT use to build the whole eval from a spec (eval-engine), to audit an existing judge against human labels (judge-calibration-auditor), to execute scoring over outputs, or for judge-reliability knowledge questions.