ab-testing-whether-skills-still-triggerlisted
Install: claude install-skill sblattj/agent-chandlery
## More triggers
skill-trigger-ab (old name), synth only for these skills, mined positives too thin, rename arm keeps each skill's state.
# ab-testing-whether-skills-still-trigger
Proves a skill-visibility change (name-only, hidden, a reworded description) didn't break which skill Claude picks.
## Do this
Scripts live in `scripts/` (shared helper `lib/ccpaths.py`). Full flag docs: `scripts/README.md`, or `-h` on any script. Self-tests (no network): `cd scripts && python3 -m unittest discover -s tests`.
```bash
P=${SKILL_DIR:-~/.claude/skills/ab-testing-whether-skills-still-trigger}/scripts; W=${AB_WORK:-~/skill-ab}/ab-$(date +%m%d); mkdir -p $W
python3 $P/mine.py ~/.claude/projects --all-projects --out $W/candidates.jsonl --days 30 --max-negatives 150 # interactive sessions only
python3 $P/filter.py $W/candidates.jsonl --out $W/cases.jsonl --model claude-haiku-5-5 # Haiku 5.5 judge (--model required, bare haiku rejected) keeps standalone prompts
python3 $P/make_arms.py --skills-dir ~/.claude/skills --out $W/arms --base-settings ~/.claude/settings.json \
--arm current --arm after --set 'after:some-skill=name-only' # first --arm = baseline; ARM:GLOB=STATE, in order
python3 $P/run.py --cases $W/cases.jsonl --arms $W/arms --out $W/results.jsonl --reps 2 --workers 6 --model sonnet # resumable; tail -f $W/results.jsonl.progress
python3 $P/report.py $W/results.jsonl # Wilson 95% CI + paired McNemar vs the first arm
```
Optional: `sy