← ClaudeAtlas

kitaru-replay-experimentlisted

Run and interpret one safe, bounded Kitaru replay comparison against an accepted cohort and exact evaluators. Use when a user wants to replay a cohort, test or compare a model, prompt, system prompt, parameter, agent version, or tool policy, supervise an experiment run, determine whether one candidate helped, or ask for one bounded change worth testing.
zenml-io/kitaru-skills · ★ 3 · AI & Automation · score 69
Install: claude install-skill zenml-io/kitaru-skills
# Kitaru replay experiment Test one candidate condition against known cases and explain whether the available evidence improved, regressed, traded off, or stayed inconclusive. Do not make the deployment decision. ## Core contract - Start from an accepted behavior, exact cohort version, exact evaluator versions and parameters, and one candidate change. Suggest one bounded candidate only when asked. - Replay starts a fresh agent task from each historical session's stored top-level inputs after applying the override. It does not restore an arbitrary checkpoint, conversation, process memory, adapter instance state, filesystem, or external world state. - Resolve adapter support and its construction path before asking to run the experiment. A shared replay schema does not prove that an adapter supports a requested override or tool source. - Require an explicit tool policy for every tool-using run. Omission resolves to live passthrough on the current server and is unsafe as an implicit default. - Carry exact IDs, versions, evaluator parameters, run-spec evidence, tool policy, failures, and missing results forward. - Explain remote writes, model and worker compute, cost uncertainty, and possible live effects before execution. One approval after this explanation covers experiment creation and the run start; any tool path with external effects needs separate approval. - Use established Kitaru product terms only. Do not coin labels for summaries or steps, such as “run card,” “result