coral-debug

Featured

Verify and debug changes to CORAL itself — smallest reproduce loop per area (grader / daemon / CLI / hooks / manager / workspace / hub / template / config / web), where to look when something breaks (hung graders, agent restart loops, stalled agents, missing heartbeat actions, corrupted shared state, broken worktree symlinks, grader import errors, wrong-task resume), how to inspect a live or finished run under `.coral/public/`, and the canonical lint/test commands. Use when editing code under `coral/` or chasing a CORAL bug, NOT when adding a new task or extending the framework.

Code & Development 910 stars 120 forks Updated today Apache-2.0

Install

View on GitHub

Quality Score: 93/100

Stars 20%
99
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# CORAL debug & change-verification workflows This skill is for people (and Claude Code) hacking on the CORAL package itself, not for users authoring a task. For authoring guides see the siblings: - `coral-new-task` — creating a new `examples/<task>/` (seed + task.yaml + grader) - `coral-extend` — extending CORAL itself (new runtime, new CLI command, new bundled skill, ...) ## Reproduce loops Pick the smallest one that exercises your change. | Editing... | Fastest reproduce | |---|---| | `coral/grader/{task_grader,loader,subprocess_grader}.py` or a builtin grader | `uv run coral validate examples/circle_packing` (or any example with a packaged grader) | | `coral/grader/daemon.py` (parallel grading, queueing, worktree isolation) | `uv run pytest tests/test_grader_daemon.py -v` | | `coral/grader/{base,protocol}.py` | `uv run pytest tests/test_grader.py tests/test_subprocess_grader.py -v` | | `coral/cli/*.py` (argparse, dispatch, output formatting) | Run the command directly: `uv run coral log --help`, `uv run coral status`, etc. against a `latest` run under `examples/<task>/results/` | | `coral/hooks/post_commit.py` (`submit_eval`) | `uv run pytest tests/test_hooks.py -v`, then end-to-end via `coral start -c task.yaml agents.count=1` and watch `.coral/public/attempts/` | | `coral/agent/manager.py`, `state.py`, `exit_classifier.py` | `uv run pytest tests/test_manager_reliability.py tests/test_manager_seen_attempts.py -v` | | `coral/agent/builtin/*.py` (a runtime) | `uv run p...

Details

Author
Human-Agent-Society
Repository
Human-Agent-Society/CORAL
Created
5 months ago
Last Updated
today
Language
Python
License
Apache-2.0

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Featured

running-coral-experiments

Run and manage CORAL experiments from the operator side — launch agents with `coral start` (dotlist overrides, model/count, tmux vs local), monitor with `coral status` / `coral log` / `coral show` / the web dashboard, and drive the loop with `coral resume` (inject instructions, fork from an attempt), `coral heartbeat` (tune reflection cadence), and `coral stop`. Use whenever the user wants to start a CORAL run, check on agents, read scores/leaderboard, steer or resume a run, diagnose agents that keep restarting or fail every eval, scale to more agents or islands, or stop a run. Deep references for steering/heartbeat tuning and scaling/troubleshooting live alongside this skill.

910 Updated today
Human-Agent-Society
AI & Automation Featured

creating-a-coral-task

Author a new CORAL task — the three pieces that must line up (`task.yaml`, `seed/`, a packaged `grader/`), the `coral init` → `coral validate` → smoke-test loop, and how to pick a grader pattern (stdout float, test pass-rate, ratio-vs-baseline, multi-metric, or an LLM rubric judge). Use whenever the user wants to create a CORAL task, write or wire a grader, port a benchmark into CORAL, score open-ended outputs (reports/memos) with a judge, or debug a grader that crashes on the seed / ranks the leaderboard backwards / leaks the answer key. Deep references for the TaskGrader API, grader patterns, rubric judges, and the full task.yaml schema live alongside this skill.

910 Updated today
Human-Agent-Society
AI & Automation Featured

coral-new-task

End-to-end recipe for adding a new task under `examples/` — the three pieces that have to line up (`task.yaml`, `seed/`, and `grader/`), what to put in each, the `TaskGrader` API surface, the `coral validate` → smoke-test loop, and the common mistakes (repo_path pointing at the wrong dir, score direction backwards, hidden answer keys leaking into seed/, grader writing to codebase_path which the daemon force-removes, private-vs-public confusion, missing `run()` signature). Use whenever the user wants to add a new CORAL task or port an existing benchmark into CORAL.

910 Updated today
Human-Agent-Society