← ClaudeAtlas

promptfoo-redteam-supply-chainlisted

Secure the LLM supply chain and detect model drift with promptfoo. Use when establishing behavioral baselines, setting up CI/CD pre-deployment security gates, detecting model drift via ASR thresholds, running vendor acceptance tests, comparing fine-tuned vs base models, integrating static (ModelAudit) + dynamic (red team) gates, or alerting on security regressions over time. Covers OWASP LLM03 supply chain vulnerabilities.
SoluDevTech/ai-driven · ★ 3 · AI & Automation · score 57
Install: claude install-skill SoluDevTech/ai-driven
# LLM Supply Chain Security and Drift Detection LLM supply chains break the traditional "hash and verify" model — a model can pass every static check and still behave dangerously; an API can change behavior overnight. OWASP LLM03 covers this. Use two complementary approaches: **static** (ModelAudit for code-execution risks) + **dynamic** (red team baselines + drift detection for behavioral risks). Establish a baseline, schedule daily `redteam eval`, track ASR, and alert on regressions. ## Use this skill when - Establishing a security baseline for production LLM deployments - Setting up CI/CD pre-deployment security gates (static + dynamic) - Detecting model drift via ASR thresholds over time - Running vendor acceptance tests before enabling a new model/provider - Comparing fine-tuned vs base models for safety regression - Integrating static (ModelAudit) + dynamic (red team) gates in CI - Alerting on security regressions via Slack/email - Scheduling daily drift checks and weekly test regeneration ## Do not use this skill when - Running a one-off red team assessment → use `promptfoo-redteam-llm` or the relevant domain skill - Assessing a foundation model in isolation (no CI/CD) → use `promptfoo-redteam-foundation-models` - Testing guardrails → use `promptfoo-redteam-guardrails` - Testing RAG, agents, MCP, or multi-input apps → use the corresponding skill ## 🛡️ Edge cases (mandatory handling) - **Static-only assessment** — a model that passes static analysis might still be