ideogram-incident-runbook

# Ideogram Incident Runbook ## Overview Rapid incident response for Ideogram API outages, auth failures, rate limiting, and degraded generation quality. Covers triage, immediate remediation, fallback activation, and postmortem process. ## Severity Levels | Level | Definition | Response Time | Example | |-------|------------|---------------|---------| | P1 | API unreachable or all requests failing | < 15 min | 401 on valid key, 500 on all requests | | P2 | Degraded quality or performance | < 1 hour | P95 latency > 30s, high 429 rate | | P3 | Minor impact, workaround exists | < 4 hours | Occasional safety rejections, slow downloads | | P4 | No user impact | Next business day | Monitoring gaps, stale cache | ## Quick Triage (Run These First) ```bash set -euo pipefail echo "=== IDEOGRAM TRIAGE ===" # 1. Test API connectivity and auth echo -n "API status: " curl -s -o /dev/null -w "%{http_code}" \ -X POST https://api.ideogram.ai/generate \ -H "Api-Key: $IDEOGRAM_API_KEY" \ -H "Content-Type: application/json" \ -d '{"image_request":{"prompt":"triage test","model":"V_2_TURBO","magic_prompt_option":"OFF"}}' echo "" # 2. Test V3 endpoint echo -n "V3 status: " curl -s -o /dev/null -w "%{http_code}" \ -X POST https://api.ideogram.ai/v1/ideogram-v3/generate \ -H "Api-Key: $IDEOGRAM_API_KEY" \ -F "prompt=triage test" -F "rendering_speed=FLASH" echo "" # 3. Check DNS resolution echo -n "DNS: " nslookup api.ideogram.ai 2>/dev/null | grep -A1 "Name:" | tail -1 || echo ...