conjoint-diagnosticslisted

Reviews an existing conjoint study for threats to inference and returns prioritized findings across five areas — design integrity (attributes, profile restrictions, task count and satisficing, randomization, power), estimation (estimand clarity, reference levels, subgroups, clustered standard errors, multiple testing), measurement error, external validity and behavioral benchmarking, and interpretation, including the guard-rail against reading an AMCE as a majority preference. Use when the user asks whether a conjoint design or analysis holds up, has referee comments on a conjoint, or wants a second opinion on estimation and interpretation choices. Building a design from scratch goes to conjoint-design, reshaping the data to conjoint-cleaning.
scdenney/open-science-skills · ★ 57 · Testing & QA · score 74

Install: claude install-skill scdenney/open-science-skills

# Conjoint Experiment Diagnostics ## Instructions Work through each section below for the conjoint study under review. Assess whether the study addresses each item adequately, partially, or not at all. Flag items that pose threats to inference and prioritize recommendations by severity. Branch on input: - **If a paper or manuscript is provided**, proceed through Sections 1–5 sequentially and produce a verdict per section. - **If analysis code or data is provided**, verify the actual implementation rather than just what the paper claims: (a) confirm clustering specification, (b) confirm the estimand matches the reported quantity, (c) if IRR is unmeasured, compute within-respondent task-pair agreement as a function of attribute-level differences (Clayton et al. 2023 §3.3 method 2 / `projoint`). For neighboring concerns, invoke sibling skills: `conjoint-design` (design choices), `conjoint-cleaning` (Qualtrics exports → long format), `hypothesis-building` (linking estimands to "If-Then" predictions), `methods-reporting` (full JARS/DA-RT compliance and replication archive), `cross-national-design` (multi-country / multilingual conjoints). --- ## 1. Design Diagnostics ### 1.1 Attribute and Level Selection - Are attributes conceptually distinct and non-overlapping? - Are levels realistic and mutually exclusive within each attribute? - Is the number of attributes justified? (Bansak et al. 2021 PSRM "Beyond the Breaking Point": response quality is generally robust even at 35 fi