active-learninglisted
Install: claude install-skill aicodedecode/awesome-muse-skills
## Overview
Active learning is the discipline of choosing which unlabeled examples to label
next, so each annotation buys maximum model improvement. Instead of labeling a
random sample, you train on a small seed set, ask the model which examples it's
most uncertain about (or which would most change its beliefs), label those,
retrain, and repeat. In the right conditions this cuts labeling cost by 30–70% for
the same accuracy — the savings are largest when labels are expensive (experts,
preference judgments) and the data pool is large and redundant.
The classic loop is simple; the engineering is not. You need an acquisition
function (uncertainty, diversity, or expected model change), a batch strategy
(labeling one example at a time is optimal but impractical), and a retraining
cadence that keeps the loop moving. Modern practice combines uncertainty sampling
with diversity constraints to avoid labeling 500 near-identical confusing
examples, and uses model-based or embedding-based proxies when retraining a giant
model every round is too costly.
Active learning is a bet that not all labels are equal. The skill is in collecting
the evidence — a random-sampling baseline — that tells you whether the bet is
paying off.
## When to use
- Labeling budget is the bottleneck and each label costs real money or expert
time.
- The unlabeled pool is large and redundant (web crawls, logs, user queries).
- Building a classifier or preference model where the decision boundary region is
sm