algolia-incident-runbook

# Algolia Incident Runbook ## Overview Rapid incident response procedures for Algolia search failures. Algolia's infrastructure is distributed across multiple data centers with automatic failover, so true Algolia outages are rare. Most incidents are caused by API key issues, settings drift, or indexing pipeline failures on your side. ## Severity Classification | Level | Definition | Response | Examples | |-------|------------|----------|----------| | P1 | Search completely down | < 15 min | 403/500 on all queries, Algolia unreachable | | P2 | Degraded search | < 1 hour | High latency, partial results, stale data | | P3 | Minor issue | < 4 hours | Analytics not updating, synonyms wrong | | P4 | No user impact | Next day | Monitoring gap, test failures | ## Quick Triage (Run These First) ```bash #!/bin/bash echo "=== ALGOLIA TRIAGE ===" # 1. Is Algolia's infrastructure up? echo -n "Algolia Status: " curl -s https://status.algolia.com/api/v2/status.json | jq -r '.status.description' # 2. Can we reach our Algolia app? echo -n "API Connectivity: " curl -s -o /dev/null -w "HTTP %{http_code} (%{time_total}s)" \ "https://${ALGOLIA_APP_ID}-dsn.algolia.net/1/indexes" \ -H "X-Algolia-Application-Id: ${ALGOLIA_APP_ID}" \ -H "X-Algolia-API-Key: ${ALGOLIA_ADMIN_KEY}" echo "" # 3. Can we actually search? echo -n "Search test: " curl -s "https://${ALGOLIA_APP_ID}-dsn.algolia.net/1/indexes/products/query" \ -H "X-Algolia-Application-Id: ${ALGOLIA_APP_ID}" \ -H "X-Algolia-AP...