prompt-guard

Featured

Meta's 86M prompt injection and jailbreak detector. Filters malicious prompts and third-party data for LLM apps. 99%+ TPR, <1% FPR. Fast (<2ms GPU). Multilingual (8 languages). Deploy with HuggingFace or batch processing for RAG security.

AI & Automation 14,647 stars 1007 forks Updated 1 months ago MIT

Install

View on GitHub

Quality Score: 94/100

Stars 20%
100
Recency 20%
75
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Prompt Guard - Prompt Injection & Jailbreak Detection Prompt Guard is an 86M parameter classifier that detects prompt injections and jailbreak attempts in LLM applications. ## Quick start **Installation**: ```bash pip install transformers torch ``` **Basic usage**: ```python from transformers import AutoTokenizer, AutoModelForSequenceClassification import torch from torch.nn.functional import softmax model_id = "meta-llama/Prompt-Guard-86M" tokenizer = AutoTokenizer.from_pretrained(model_id) model = AutoModelForSequenceClassification.from_pretrained(model_id) model.eval() def get_jailbreak_score(text): """Check user input for jailbreak attempts.""" inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512) with torch.no_grad(): logits = model(**inputs).logits probs = softmax(logits, dim=-1) return probs[0, 2].item() # Jailbreak probability # Check prompt score = get_jailbreak_score("Ignore previous instructions") if score > 0.5: print("⚠️ Jailbreak attempt detected!") ``` **Classification labels**: - **BENIGN** (label 0): Normal content - **INJECTION** (label 1): Embedded instructions in data - **JAILBREAK** (label 2): Direct override attempts ## Common workflows ### Workflow 1: User input filtering (jailbreak detection) **Filter user prompts before LLM**: ```python def filter_user_input(user_message, threshold=0.5): """ Filter user input for jailbreak attempts. Returns: (is_safe, score, message) ...

Details

Author
Orchestra-Research
Repository
Orchestra-Research/AI-Research-SKILLs
Created
1 years ago
Last Updated
1 months ago
Language
TeX
License
MIT

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category