reproducibility-third-party-verification

Featured

(Proposal, unverified) Attempt to verify a paper's reported results by actually executing its released code/scripts against its own reported configuration — the only SOP in this package whose action type is code execution rather than text reading/judgment. Use this after unit-classification has extracted the paper's reported configuration/hyperparameters as classified units; "not_attempted" is a correct, common output when the paper's own reporting is too incomplete to run, not a failure of this SOP.

AI & Automation 392 stars 34 forks Updated 1 weeks ago Apache-2.0

Install

View on GitHub

Quality Score: 94/100

Stars 20%
86
Recency 20%
90
Frontmatter 20%
70
Documentation 15%
66
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Reproducibility Third-Party Verification (Proposal) Actually runs code to check reported results against the paper's own extracted configuration — unique action type (execution) in this package. Fills the evidence-verification × engineering-metadata gap in the evaluative-stance × content-layer matrix. ## Execution Subagent — spawned via spawn-agent skill. ## Dependency: unit-classification, not raw full_text (graph correction L20) This SOP needs the paper's reported configuration already pulled out in structured form before attempting to verify it — hence its input is `classified_units`, not `full_text` directly. An earlier graph draft had this SOP depending on nothing upstream, which meant it had no defined way to get the structured claims it needs to check. ## Proposal Status — Read Before Modifying No primary-source precedent, no inter-rater-reliability baseline. Keep "(Proposal, unverified)" in the description until real usage validates the method. Given the code-execution action type, treat any scope expansion here with more caution than the other 3 proposal SOPs. <!-- BEGIN available-tables (generated) --> ## Available SOPs | SOP | When to use | | --- | --- | | spawn-agent | Spawn a customized CC subagent with full MCP tool access. | <!-- END available-tables (generated) -->

Details

Author
yogsoth-ai
Repository
yogsoth-ai/de-anthropocentric-research-engine
Created
6 months ago
Last Updated
1 weeks ago
Language
HTML
License
Apache-2.0

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Listed

reproducibility-third-party-verification

(Proposal, unverified) Attempt to verify a paper's reported results by actually executing its released code/scripts against its own reported configuration — the only SOP in this package whose action type is code execution rather than text reading/judgment. Use this after unit-classification has extracted the paper's reported configuration/hyperparameters as classified units; "not_attempted" is a correct, common output when the paper's own reporting is too incomplete to run, not a failure of this SOP.

1 Updated 1 weeks ago
yogsoth-ai
AI & Automation Featured

third-pass-deep-read

Keshav's third pass — the heaviest of the three, a full sentence-by-sentence re-read including proofs/derivations, attempting a virtual re-implementation of the paper to surface implicit assumptions and concrete improvement points. Use this after second-pass-grasp, as the terminal step of the Keshav three-pass method, whenever genuine mastery of a paper (not just a summary) is needed. This is not a skippable recap — treat "nothing new to add" as suspicious, not a default outcome.

392 Updated 1 weeks ago
yogsoth-ai
AI & Automation Listed

third-pass-deep-read

Keshav's third pass — the heaviest of the three, a full sentence-by-sentence re-read including proofs/derivations, attempting a virtual re-implementation of the paper to surface implicit assumptions and concrete improvement points. Use this after second-pass-grasp, as the terminal step of the Keshav three-pass method, whenever genuine mastery of a paper (not just a summary) is needed. This is not a skippable recap — treat "nothing new to add" as suspicious, not a default outcome.

1 Updated 1 weeks ago
yogsoth-ai