← ClaudeAtlas

fabric-spark-monitoringlisted

Use for diagnosing Fabric Spark performance through the monitoring REST APIs — listing Livy sessions (/workspaces/{ws}/spark/livySessions with queuedDuration/runningDuration, HC_ session naming), pulling the Spark History Server mirror (.../notebooks/{nb}/livySessions/{livy}/applications/{appId}/jobs) for job timelines and gap analysis, attributing notebook wall-clock to queue/boot/work/teardown phases, and verifying high-concurrency session reuse (sessionSource created vs reused).
wardawgmalvicious/claude-config · ★ 2 · DevOps & Infrastructure · score 75
Install: claude install-skill wardawgmalvicious/claude-config
# Fabric Spark monitoring APIs Attribute Fabric Spark notebook / pipeline wall-clock time using the Spark monitoring REST surface: Livy session listings plus the embedded Spark History Server (SHS) mirror. Every endpoint here is a GET — read-only diagnostics, no state changes. All endpoint shapes and field names below were exercised live on 2026-08-17. The surface is preview-era — re-verify response shapes before building anything that depends on their stability. ## Auth & base URL ```bash az account get-access-token --resource https://api.fabric.microsoft.com --query accessToken -o tsv ``` Delegated token. On a Fabric-only tenant (no Azure subscription) the login itself needs `az login --allow-no-subscriptions`. Base URL is `https://api.fabric.microsoft.com/v1`; plain `curl` with a bearer header works from there — no SDK or MCP server required. Full token audience table and 401 debugging: **fabric-auth** skill. ## Listing Livy sessions ``` GET /workspaces/{workspaceId}/spark/livySessions?maxResults=N # workspace-wide GET /workspaces/{workspaceId}/notebooks/{notebookId}/livySessions # per-item ``` Field guide for the session objects: | Field | Meaning | |---|---| | `sparkApplicationId` | YARN application id — required for SHS drill-down below | | `livyId` | The Livy session id used in per-application URLs | | `livyName` | `HC_<NotebookName>_<livyId>` prefix marks a high-concurrency session | | `state` | Session lifecycle state | | `origin`, `submitter`, `jobType`