machine-learninglisted

Builds, ships and operates predictive models - framing the problem, constructing features and splits, training reproducibly, serving, and knowing when to retrain. Use when deciding whether something is an ML problem, defining a target and the baseline it must beat, designing splits for time-ordered or grouped data, choosing a model family and a tuning budget, reviewing training code, or diagnosing a model that validated well and disappoints in production - train/serve skew, drift, retraining triggers. Covers leakage prevention, imbalance, encoding and missingness. Not for whether an improvement is real (statistics), stack selection or LLM, RAG and agent design (ai-engineering), or pipelines and warehouses (data-engineering). Ships an offline AST auditor (scripts/ml_check.py).
anton-winter-arch/dotagents · ★ 1 · AI & Automation · score 77

Install: claude install-skill anton-winter-arch/dotagents

# machine-learning **A validation score is a claim about a construction, not a measurement of the world.** It is only worth what the construction is worth, and most of the time the construction is where the mistake is. A scaler fitted before the split, a random split of dated rows, a target column that survived into the feature matrix, a hyperparameter search that quietly consumed the test set: each of these produces a number that is both excellent and meaningless, and none of them looks wrong in the code. So the order of work is fixed. Decide what is being predicted and what beats doing nothing, construct the data so that no future or test information can reach training, then train, and only then care which algorithm won. Algorithm selection is the most documented and least decisive part of this, and it is the part practitioners spend most of their time on. Four questions answered before any model is fitted: 1. **What decision does this change?** If no decision changes, stop. 2. **What is the baseline?** Majority class, last observed value, the rule the business already uses. Beat it or the model is not worth operating. 3. **Is the information available at prediction time?** A feature that only exists after the event is not a feature, it is the answer. 4. **What would make this wrong in production?** Answer it now, in writing, so the monitoring has something to watch. ## First: which situation is this? **Framing a new problem.** Work `references/framing.md` be