loopx-benchmark

Featured

Use when a LoopX-managed goal runs, tracks, scores, or analyzes a benchmark experiment through benchmark-toolkit, including experiment-board rows, solver arms, integrity qualification, matched comparisons, or case insights. Do not use for casual benchmark discussion, ordinary software microbenchmarks, or eval mentions without LoopX experiment state.

AI & Automation 6,002 stars 582 forks Updated today Apache-2.0

Install

View on GitHub

Quality Score: 93/100

Stars 20%
100
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# LoopX Benchmark Workflow Use this skill for a LoopX-managed benchmark experiment. The builtin `benchmark-toolkit` capability owns provider-neutral experiment state and integrity boundaries. This packaged skill is its task-triggered Agent playbook. The capability is catalog-ready without a per-Goal enable switch. Installing this skill does not grant runner, shell, network, credential, private-evidence, or Goal mutation authority. Respect the selected todo's required capabilities, any external provider binding, host permissions, and user gates. ## Capability surface - `loopx capability show benchmark-toolkit --format json` — catalog entry with usage hints, role boundaries, and the post-run case-insight template. - `loopx benchmark --help` — subcommands (experiment-board-show, experiment-board-upsert, source-revision-fence, integrity-qualification, classify-artifacts). ## Share a study through the public-safe contract When another benchmark developer needs portable study data, use the capability's typed study flow rather than sharing a runner-specific ledger or raw evidence: 1. Validate `benchmark_study_manifest_v0` with `benchmark study-validate`. 2. Wrap one allowlisted manifest, experiment-board row, redacted insight, or runtime observation with `benchmark upload-envelope`. 3. Run `benchmark upload-local` without `--execute` first, then explicitly execute against a caller-selected local JSONL store. 4. Verify the record/digest/revision binding with `bench...

Details

Author
loopx-project
Repository
loopx-project/loopx
Created
3 months ago
Last Updated
today
Language
Python
License
Apache-2.0

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Solid

skill-benchmark

Use when the user runs /skill-benchmark to score agent skills via LLM judges with baseline comparison, regression detection, and trend analysis, or to compare candidate models on a shared task set in a ranked table with per-model spend tracking. Not for release gating — use skill-benchmark-gate.

38 Updated yesterday
OutlineDriven
AI & Automation Listed

benchmark-curator

Design, curate, version, and quality-control benchmark or evaluation corpora for Agent Skills and quality workflows, including task taxonomies, discovery/forced/negative controls, adversarial and regression cases, difficulty strata, holdout isolation, provenance, duplication, contamination risk, coverage balance, and immutable benchmark hashes. Use when the user asks to build or maintain an eval dataset, golden set, benchmark suite, regression corpus, holdout, challenge set, or representative test cases. Do not use to execute the model experiment or claim lift (use skill-evaluator), to design the grading rubric (use rubric-designer), to edit the candidate skill (use skill-creator), or to treat leaked/known cases as a clean holdout.

1 Updated today
CometWeb-io
Web & Frontend Solid

benchmark

Local-only regression / benchmark skill for ui-clone-skills maintainers. Drives the standard ui-reverse-engineering pipeline against the canonical reference site (https://realfood.gov) and records AE/SSIM, iteration count, gate fail counts, and outcome to benchmark/history.csv so prompt / sub-doc / model-version drift surfaces as a trend. Trigger phrases: "run benchmark" / "regression benchmark" / "benchmark clone". The Makefile no longer has a `benchmark` target — setup is inline bash in this skill (Step 1 below). Internal: NOT registered in `.claude-plugin/plugin.json` `skills`. Not part of the public 3-skill marketplace surface. Maintainer tooling only.

13 Updated yesterday
voidmatcha