groq-observability

Featured

Set up observability for Groq integrations: latency histograms, token throughput, rate limit gauges, cost tracking, and Prometheus alerts. Trigger with phrases like "groq monitoring", "groq metrics", "groq observability", "monitor groq", "groq alerts", "groq dashboard".

AI & Automation 2,926 stars 433 forks Updated yesterday MIT

Install

View on GitHub

Quality Score: 99/100

Stars 20%
100
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Groq Observability ## Overview Monitor Groq LPU inference for latency, token throughput, rate limit utilization, and cost. Groq's defining advantage is speed (280-560 tok/s), so latency degradation is the highest-priority signal. The API returns rich timing metadata (`queue_time`, `prompt_time`, `completion_time`) and rate limit headers on every response. ## Key Metrics to Track | Metric | Type | Source | Why | |--------|------|--------|-----| | TTFT (time to first token) | Histogram | Client-side timing | Groq's main value prop | | Tokens/second | Gauge | `usage.completion_time` | Throughput degradation | | Total latency | Histogram | Client-side timing | End-to-end performance | | Rate limit remaining | Gauge | `x-ratelimit-remaining-*` headers | Prevent 429s | | Token usage | Counter | `usage.total_tokens` | Cost attribution | | Error rate by code | Counter | Error handler | Availability | | Estimated cost | Counter | Tokens * model price | Budget tracking | ## Instructions ### Step 1: Instrumented Groq Client ```typescript import Groq from "groq-sdk"; const groq = new Groq(); interface GroqMetrics { model: string; latencyMs: number; ttftMs: number; tokensPerSec: number; promptTokens: number; completionTokens: number; totalTokens: number; queueTimeMs: number; estimatedCostUsd: number; } const PRICE_PER_1M: Record<string, { input: number; output: number }> = { "llama-3.1-8b-instant": { input: 0.05, output: 0.08 }, "llama-3.3-70b-versatile": {...

Details

Author
jeremylongshore
Repository
jeremylongshore/claude-code-plugins-plus-skills
Created
1 years ago
Last Updated
yesterday
Language
Python
License
MIT

Integrates with

Bundled in these plugins

Similar Skills

Semantically similar based on skill content — not just same category