harness-verify

# harness-verify — headless A/B turn replay Replays a recorded ModMixer chat up to a chosen point, then runs the agent's **next turn** headlessly against a chosen model — once per variant, repeated N times — and reports what the agent *did* (asked the user vs. fired a side-effecting tool like `run_test_cycle`). This is how you get runtime evidence that a prompt/tool/hint change actually moves model behavior, especially for non-frontier models (Kimi etc.) where "it's in the system prompt" is not the same as "the model obeys it". It runs the **real** system prompt, the **real** tool set, and the **real** model + credentials — but in an isolated session with side-effecting tools swapped for recording stubs, so nothing edits files, builds, launches RimWorld, or touches the user's conversation store. ## When to use - You changed how the agent should behave (a system-prompt rule, a tool description, a tool-result hint, a model) and want to see the effect. - You want an A/B: same history + model, with vs. without your change. - The surface is the agent's decision, and driving the GUI by hand is slow, costs RimWorld launches, and isn't repeatable. Not for: pure mechanical changes with no behavioral question (a typecheck is enough), or anything where you'd need many full agentic turns with real file mutation (this stubs the side-effecting tools). ## Run it ```powershell # Always dry-run first — proves the pipeline (Electron boot, model # resolution, session reconstruction, ...