← ClaudeAtlas

autoresearchlisted

Autonomous goal-directed iteration loop that continuously improves prompts, templates, configs, or code. Two evaluation modes — deterministic (eval.py with proxy heuristics) or AI judge (LLM rubric scoring). Uses four-way separation in both modes. Inspired by Karpathy's autoresearch.
naveedharri/benai-skills · ★ 46 · AI & Automation · score 71
Install: claude install-skill naveedharri/benai-skills
# AutoResearch — Autonomous Optimization Loop USE WHEN the user runs `/autoresearch`, says "autoresearch", "optimize this prompt", "improve this overnight", "run an optimization loop", "iterate on this", "auto-improve", or wants to autonomously refine a file against measurable criteria. --- ## Architecture: Separation of Roles This skill supports two mutually exclusive evaluation modes. Both maintain four-way separation. ### Deterministic Mode (default) — Four-Way Separation | Role | Who | Knows Eval Code? | Knows Prompt History? | |------|-----|-----------------|----------------------| | **Main Agent** | You (optimizer) | **NO** — reads metric number only | Yes — reads logs, plans changes | | **Eval Agent** | `autoresearch-eval-agent` sub-agent | Yes — writes eval.py | No | | **Test Runner** | `autoresearch-test-runner` sub-agent | **NO** — fresh context | **NO** | | **Judge Script** | `eval.py` (deterministic Python) | IS the eval | No | Metric: `pass_rate`. Best for tasks with clear, mechanical quality checks (word count, format, keywords, structure). ### AI Judge Mode (opt-in) — Four-Way Separation | Role | Who | Knows Rubric? | Knows Prompt History? | |------|-----|---------------|----------------------| | **Main Agent** | You (optimizer) | Criteria names only | Yes — reads logs, plans changes | | **Eval Agent** | `autoresearch-eval-agent` sub-agent | Yes — writes rubric.md | No | | **Test Runner** | `autoresearch-test-runner` sub-agent | **NO** — fresh context