scoring-checks

Featured

Add a new deterministic scoring check in src/scoring/checks/ that evaluates config quality. Follows the Check[] return pattern, uses point constants from src/scoring/constants.ts, and integrates via filterChecksForTarget() in src/scoring/index.ts. Use when user says 'add scoring check', 'new check', 'modify scoring criteria', or works in src/scoring/checks/. Do NOT use for display changes or refactoring scoring logic.

AI & Automation 1,223 stars 116 forks Updated yesterday MIT

Install

View on GitHub

Quality Score: 93/100

Stars 20%
100
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Adding a Scoring Check Add a new deterministic check that evaluates a single aspect of AI agent config quality. All checks must be filesystem-based with no network calls or LLM inference. ## Critical - **Check must be deterministic**: Same filesystem state → same result every time. No randomness, no external APIs. - **Point values come from constants.ts**: Every `earnedPoints` and `maxPoints` must reference `POINTS_*` from `src/scoring/constants.ts`. Do NOT hardcode numbers. - **Always return `Check[]` array**: Export a function `check<Category>(dir: string): Check[]` where category is one of: `existence`, `quality`, `grounding`, `accuracy`, `freshness`, `bonus`. - **Every check must have**: `id` (kebab-case, unique), `name`, `category`, `maxPoints`, `earnedPoints`, `passed`, `detail`, and optional `suggestion`/`fix`. - **Fix object fields**: `action` (string describing what to do), `data` (context for the fix), `instruction` (user-facing guidance). - **Register in src/scoring/index.ts**: Add the import and spread the result into the `allChecks` array in `computeLocalScore()`. - **Target filtering**: If the check is platform-specific (Claude-only, Cursor-only, etc.), add its ID to the appropriate `*_ONLY_CHECKS` set in `constants.ts`. ## Instructions ### Step 1: Define point constants in src/scoring/constants.ts Verify before proceeding: Is your check measurable with a numeric point value? Add constants below the appropriate category section (existence, quality, grou...

Details

Author
caliber-ai-org
Repository
caliber-ai-org/ai-setup
Created
4 months ago
Last Updated
yesterday
Language
TypeScript
License
MIT

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category