ai-model-extractionlisted

# Model extraction & data inference ## When it applies You have query access to an ML/LLM endpoint and want to show it leaks the model itself, its training data, or confidential context — IP theft or privacy impact, not just a bad answer. ## Why it works Query access is more powerful than it looks. Outputs (labels, probabilities, embeddings, generations) carry information about the model and its data. Enough targeted queries reconstruct a functional copy, reveal whether a record was in training, or regurgitate memorized secrets. ## Method 1. **Model stealing**: query systematically (esp. if confidence scores/logits are returned) to train a surrogate that mimics the target — proves the model can be cloned via the API. 2. **Membership inference**: compare model behaviour (confidence, loss) on candidate records to infer whether a specific record was in the training set (privacy impact). 3. **Training-data / secret extraction (LLM)**: prompt for memorized data — PII, keys, or the system prompt/hidden context (overlaps `ai-prompt-injection`); look for verbatim regurgitation. 4. **Embedding inversion**: if an embeddings API is exposed, reconstruct approximate input text from vectors. 5. **Cost/DoS angle**: unbounded/unthrottled querying is itself a finding (LLM10). ## Gotchas - Tie it to impact: a stolen surrogate, a confirmed membership leak, or verbatim secret output — not "it answered a lot". - Respect scope/RoE — extraction requires many queries; get authorizati