coral-new-task

# Creating a new CORAL task A CORAL task is **three things** that must line up: ``` examples/<task>/ ├── task.yaml # config: name, description, grader entrypoint, agent count ├── seed/ # starter code agents see when they begin (the repo_path) │ └── solution.py └── grader/ # standalone Python package ├── pyproject.toml └── src/<task>_grader/ ├── __init__.py └── grader.py # class Grader(TaskGrader): ... ``` The packaged form is the only supported form. The package gives the grader its own venv and ships everything the eval needs — grader code, helper modules, and hidden data (see "Hidden data" below). ## Reference implementations Look at these before writing anything new — copy the closest one and edit: | Reference | When to copy it | |---|---| | [examples/erdos/](examples/erdos/) | Minimal packaged grader, single grader file, numpy-only deps | | [examples/dna_design/](examples/dna_design/) | Packaged grader with bundled data files (`importlib.resources`) and `[ml]` optional-deps for heavy libs | | [examples/swebench-verified/](examples/swebench-verified/) | Tiered eval (different instance counts per tier), private answer keys, harbor integration | | [examples/circle_packing/](examples/circle_packing/) | Smallest packaged task end-to-end — single solution file, single grader file | | [examples/mnist/](examples/mnist/) | Packaged grader with a hidden answer key (note: secret data belongs under `grader.private` in a `taskda...