model-council-votinglisted

# Model Council Voting: Panels of Language Models as Independent Coders ## Instructions A "council" runs the same labeling, scoring, or term-discovery task through several language models independently and reads their (dis)agreement as data. It sits **on top of** the single-model codebook-and-validation workflow in `text-classification` — build and validate the codebook there first, then escalate to a council only when one model is not enough. Companion skills: `topic-modeling` (an independent, non-LLM method for cross-checking what a council finds), `llm-calibration-logprobs` (per-item confidence from one model's token probabilities, a different signal than cross-model agreement), and `methods-reporting` (the standards the write-up must meet). ### 1. When a Council Beats a Single Model - Use a council when the labeling decision is **contested or ambiguous** — fuzzy category boundaries, stance or frame coding, constructs where reasonable coders disagree. The disagreement rate is itself a measurable property of the task, not noise to be averaged away. - Use a council for **corpus-driven discovery** where the output set is not fixed in advance — e.g., which identity terms a corpus foregrounds. A discovery script of this kind runs a zero-shot extraction prompt over sampled text windows; requiring several model *families* to independently surface the same term (§4) is what separates a real corpus signal from one model's idiosyncrasy. - Use a council for **robustness reporting