run-the-conditions-the-source-ran

Solid

Use at study design, before any experiment of your own is costed, on reproduction and method-evaluation tasks. Covers enumerating the systems, scenarios, stress sweeps and case studies the source names, running each one by name, measuring the preconditions the method declares it needs, and what to do when one of them fails.

AI & Automation 804 stars 25 forks Updated today NOASSERTION

Install

View on GitHub

Quality Score: 82/100

Stars 20%
97
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# The source's own systems, scenarios and stress tests are your experiment list A paper's results are attached to specific things: named systems, named scenarios, a named molecule, a named problem, a named noise axis, and the conditions the method says it needs. A checklist for reproducing that paper is written from those names. A better-designed experiment on different conditions answers a question nobody asked. ## The failure this prevents A run reproducing a feature-selection method replaced the paper's robustness experiment — degradation under falling signal-to-noise, reduced library size and increased dropout, against two named baselines — with its own structured confounder and batch-geometry design. Better science, in the abstract, and the reviewer wrote: *"it does not perform or report any simulations varying SNR, library size, or dropout, nor explicitly compare performance degradation curves versus Laplacian Score or MCFS."* That requirement scored **5**. A plain agent that simply ran the paper's sweep scored **65** on the same requirement. The same shape recurs, and it is never laziness — it is always a substitution made for a good local reason: * A chemistry run judged that saliency maps are not interpretable enough to be worth computing, and argued the point instead of computing them on the paper's own molecule: 5 against 70. * A climate run carried two of three named SSP scenarios and dropped the third; the requirement that names it scored 12 against 38...

Details

Author
tangxiangru
Repository
tangxiangru/AutoR
Created
5 months ago
Last Updated
today
Language
Python
License
NOASSERTION

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category