llm-tracing-and-observability-setup

# LLM Tracing and Observability Setup ## What this skill does This skill sets up production-grade observability for LLM applications. Without tracing, debugging a broken LLM pipeline means guessing — you can't see what prompt was sent, what the model returned, which tool call failed, or why latency spiked. This skill configures the right tracing layer for your stack and shows what to instrument. ## How to use ### Claude Code / Cline Copy this file to `.agents/skills/llm-tracing-setup/SKILL.md` in your project root. Then ask: - *"Use the LLM Tracing Setup skill to add observability to our LangChain app."* - *"Set up Langfuse tracing for our OpenAI API calls."* Provide: - LLM framework in use (LangChain, direct API, LlamaIndex, custom) - Preferred tracing backend (LangSmith, Langfuse, Helicone, or open to suggestions) - Language (Python or TypeScript) - Whether you need cost tracking, latency alerting, or user feedback collection ### Cursor / Codex Paste your LLM call code alongside these instructions and specify the tracing backend. ## The Prompt / Instructions for the Agent ### Step 1 — Choose a tracing backend | Backend | Best for | Cost model | |---|---|---| | **LangSmith** | LangChain / LangGraph apps | Free tier + usage | | **Langfuse** | Any LLM stack, self-hostable | Free tier + open source | | **Helicone** | OpenAI / Anthropic direct API | Per-request fee | | **OpenTelemetry + Jaeger** | Full control, existing OTel infra | Self-hosted | | **Braintrust** | E...