reduce-llm-costlisted

Use this to cut the cost of an LLM app using observability data. Trigger on "my OpenAI/Anthropic bill is too high", "reduce token usage", "the app is expensive", "optimize LLM cost", "why am I spending so much on the API". Find the expensive spans first (measure), then apply the cheapest wins. Don't guess - the trace tells you where the money goes.
ContextJet-ai/awesome-llm-observability · ★ 26 · AI & Automation · score 72

Install: claude install-skill ContextJet-ai/awesome-llm-observability

# Reduce LLM cost (measure first) Most LLM bills are dominated by a few patterns you can *see* in traces. Measure before optimizing - the biggest cost is rarely where people assume. ## Step 1 - find where the money goes From your observability tool, sort spans by cost (or `input_tokens`). You're looking for: - **The highest-token spans** - usually bloated context or a whole chat history re-sent every turn. - **Retry storms** - the same call repeated N times (rate limits / transient errors) multiplying cost. - **The most-frequent call** × its per-call cost - a cheap call made 10,000×/day beats one expensive call. - **Model overkill** - using a frontier model for a task a small/cheap model handles fine. If you have no cost data yet, add tracing first (see `instrument-llm-observability`) - you can't optimize what you can't see. ## Step 2 - apply wins, cheapest-effort first 1. **Right-size the model.** Route easy calls (classification, extraction, routing) to a small/cheap model; reserve the frontier model for hard reasoning. Biggest lever for most apps. 2. **Trim the context.** Stop re-sending the full history/system prompt every turn. Send only what's needed; summarize old turns. For RAG, retrieve fewer/better chunks, not more. 3. **Cache.** Enable prompt caching (Anthropic/OpenAI) for stable prefixes; cache identical requests (a gateway like Helicone/Portkey/LiteLLM does this for free). 4. **Cap `max_tokens`.** Unbounded outputs cost unbounded money; set a sane ceiling.