anth-performance-tuning

# Anthropic Performance Tuning ## Overview Optimize Claude API latency and throughput via prompt caching, model selection, streaming, and request optimization. The biggest wins come from prompt caching (90% input cost reduction) and model selection (Haiku is 4x faster than Sonnet). ## Prompt Caching (Biggest Win) ```python import anthropic client = anthropic.Anthropic() # Mark long, reusable content with cache_control # Cached content: 90% cheaper on subsequent requests, near-zero latency for cached portion message = client.messages.create( model="claude-sonnet-4-20250514", max_tokens=1024, system=[ { "type": "text", "text": "You are an expert on the following 50-page document: ...<long document>...", "cache_control": {"type": "ephemeral"} # Cache this block } ], messages=[{"role": "user", "content": "What does section 3.2 say?"}] ) # Check cache performance print(f"Cache read tokens: {message.usage.cache_read_input_tokens}") # Free/cheap print(f"Cache creation tokens: {message.usage.cache_creation_input_tokens}") # First call only print(f"Uncached input tokens: {message.usage.input_tokens}") ``` **Cache requirements:** Minimum 1,024 tokens for Sonnet/Opus, 2,048 for Haiku. Cache lives for 5 minutes (refreshed on each hit). ## Model Selection for Speed | Model | Speed | Cost (per MTok in/out) | Best For | |-------|-------|----------------------|----------| | Claude Haiku | Fastest | $0.80 / ...