grafana-prometheus-alertinglisted

# Grafana native alerting via provisioning-as-code Covers discovering that a monitoring stack looks complete but doesn't actually alert, and closing that gap using Grafana's own alerting engine plus metrics `node_exporter` is already exposing — without building a separate distributed alerting mechanism. ## First, check whether alert rules actually exist A Prometheus + Grafana + `node_exporter` stack can look fully monitored (dashboards render, scrape targets show `up`) while having **zero real alerting** wired up. Don't assume rules exist just because the stack is deployed — check directly: ```bash # Prometheus's own rules API -- an empty groups list means no rules at all, regardless of # how populated the dashboards look. curl -s http://localhost:9090/api/v1/rules | python3 -m json.tool # Grafana's OWN unified alerting is a SEPARATE thing from Prometheus rule_files -- check this # too, via a scoped API token: curl -s -H "Authorization: Bearer $GRAFANA_TOKEN" \ http://<grafana-host>:3000/api/v1/provisioning/alert-rules | python3 -m json.tool ``` If Grafana has exactly one rule (commonly a generic "Host Down" watching `up{job="..."}`), that only proves the *notification pipe* (contact point → Telegram/email/etc.) works — it says nothing about individual service failures on an otherwise-healthy host. Don't conflate "the alerting pipe is proven" with "the fleet is alerted." ## Detect per-service failures without building new infrastructure Before designing a custom `On