clade-performance-tuning

# Anthropic Performance Tuning ## Overview Claude latency has two components: **time to first token (TTFT)** and **tokens per second (TPS)**. Different strategies target each. ## Latency Benchmarks (approximate) | Model | TTFT (p50) | TTFT (p95) | Output TPS | |-------|-----------|-----------|------------| | Claude Haiku 4.5 | 200ms | 600ms | ~150 | | Claude Sonnet 4 | 400ms | 1.2s | ~90 | | Claude Opus 4 | 800ms | 2.5s | ~40 | ## Optimization Strategies ## Instructions ### Step 1: Always Stream ```typescript // Streaming delivers the first token ASAP — user sees response instantly // instead of waiting for the full response to generate const stream = client.messages.stream({ model: 'claude-sonnet-4-20250514', max_tokens: 1024, messages, }); // First token arrives in ~400ms (Sonnet) // Full response may take 5-10s, but user sees progress immediately for await (const event of stream) { if (event.type === 'content_block_delta') { yield event.delta.text; } } ``` ### Step 2: Prompt Caching — Faster TTFT ```typescript // Cached prompts skip re-processing — dramatically lower TTFT for large system prompts const message = await client.messages.create({ model: 'claude-sonnet-4-20250514', max_tokens: 1024, system: [{ type: 'text', text: largeSystemPrompt, // 10K+ tokens cache_control: { type: 'ephemeral' }, }], messages, }, { headers: { 'claude-beta': 'prompt-caching-2024-07-31' }, }); // TTFT drops from ~2s to ~500ms on cache hit with la...