debugging-a-bad-fine-tunelisted

Diagnose a fine-tuned model that behaves badly at inference: output that never stops or repeats forever, replies that ignore the training data, garbled or nonsense tokens, a model that answers as the wrong persona, an adapter that will not load against its base, or quality that collapsed after quantisation. Works backwards from the symptom to the cause, most often a chat template mismatch between training and inference, a missing end-of-sequence token, a tokenizer mismatch, or overfitting on too small a dataset. Use when a trained model is loaded and producing wrong output. Not for models that fail to load at all for environment reasons, not for a crash while the model is being trained, and not for deciding whether the model is good enough, which is evaluating-a-tuned-model.
ErtasAI/open-model-skills · ★ 2 · AI & Automation · score 76

Install: claude install-skill ErtasAI/open-model-skills

# Debugging a bad fine-tune Use this when a trained model loads and runs but the output is wrong: it never stops, it repeats, it ignores what it was trained on, it comes out garbled, it answers as the wrong model, an adapter refuses to load, or it got worse after quantisation. This works backwards from the symptom you can actually observe to the specific cause, rather than starting from the training pipeline and hoping to spot the bug by inspection. It is not for a model that fails to load for environment reasons (wrong runtime, missing dependency, out of memory), not for a crash during training itself, and not for deciding whether a working fine-tune is actually better than the base model. That last question is **evaluating-a-tuned-model**. ## Start here If `BUNDLE-REPORT.md` exists in the project root, read its **Shape**, **Base model** and **Defects** sections before doing anything else. Several of the causes below are things that skill already found and named: a missing `generation_config.json`, a chat template that only exists in `chat_template.jinja`, a default system prompt that still claims the base model's identity, an adapter with no recorded base model. If the Defects section already names one of these, that is very likely your answer, not a separate thing to re-discover. If `BUNDLE-REPORT.md` does not exist, that is not an error. Work directly from the bundle: identify the shape (GGUF, adapter, or merged safetensors, the same three shapes **inspecting-a-model