autoresearch

# Autoresearch: Autonomous Experiment Loop An autonomous optimization loop where Claude edits code, runs a benchmark, measures a metric, and keeps improvements or reverts — repeating forever until stopped. ## Core Concept The loop is simple: **edit → commit → run → measure → keep or discard → repeat**. - **Primary metric is king.** Lower (or higher, depending on direction) is better. Improved → keep the commit. Equal or worse → `git revert`. - **State survives context resets** via `autoresearch.jsonl` (append-only log) and `autoresearch.md` (living session document). - **Domain-agnostic.** Works for any measurable target: test speed, bundle size, LLM training loss, Lighthouse scores, build times, etc. - **Be careful not to overfit to the benchmarks and do not cheat on the benchmarks.** Optimize the real workload, not the measurement harness. ## Setup Phase When the user triggers autoresearch, gather the following (ask if not provided). If `$ARGUMENTS` is provided, use it as the optimization goal: 1. **Goal** — what to optimize (e.g., "reduce unit test runtime") — use `$ARGUMENTS` if provided 2. **Command** — the benchmark to run (e.g., `pnpm test`, `uv run train.py`) 3. **Primary metric** — name, unit, and direction (`lower` or `higher` is better) 4. **Secondary metrics** — optional additional metrics to track for tradeoff monitoring (e.g., memory, compile time) 5. **Files in scope** — which files can be modified 6. **Constraints** — time budget, off-limits files, corr...