← ClaudeAtlas

tts-evallisted

Use when the user asks to evaluate, bench, or compare a TTS model or engine for the shows — renders a fixed corpus through render.py's engine registry on the candidate engine and on the production engine as a re-rendered control, measures a fixed metric set, appends a dated ledger entry and writes one self-contained HTML report with side-by-side players. Not a show; nothing here ships audio.
schmug/clodcast · ★ 0 · AI & Automation · score 69
Install: claude install-skill schmug/clodcast
# TTS eval bench One command that says whether a new model is better than what ships, on the same lines and the same metrics every time, so the next evaluation costs an adapter and a run rather than a day. ```bash python3 skills/tts-eval/bench.py --engine breeze ``` That renders the corpus below on `breeze` **and** on the production engine, analyzes every take, and prints the ledger and report paths on its last two lines. Expect ten to fifteen minutes on an M4 Max for two engines (Breeze renders at under realtime); the analysis deps are checked before anything renders. ## How to evaluate a new model 1. Register it in `render.ENGINES` (`skills/daily-podcast/render.py`) with its adapter in `_ENGINE_GENERATORS` — capabilities, take ceiling, minimum mlx-audio, license. That is the registry's job and the whole of the model-specific work; the design is in `docs/superpowers/specs/2026-09-04-tts-engine-registry-design.md`. 2. Run the bench with `--engine <name>`. The `--engine` choices are read from `ENGINES` at parse time, so nothing in this skill changes. 3. Open the report and listen. The numbers rank; the ear decides. Then, if the answer is yes, switching a show is a one-line manifest change made deliberately (the Surface Tension case is #203). ## What it renders, and how The bench renders **through** `render.validate_manifest` and `render.render_segments` with the engine named on the manifest — the registry's own adapter, never a re-implementation of the TTS call. That is