openrouter-performance-tuning

# OpenRouter Performance Tuning ## Overview OpenRouter adds minimal overhead (~50-100ms) to direct provider calls. Most latency comes from the upstream model. Key levers: model selection (smaller = faster), streaming (lower TTFT), parallel requests, prompt size reduction, and provider routing to faster infrastructure. This skill covers benchmarking, streaming optimization, concurrent processing, and connection tuning. ## Benchmark Latency ```python import os, time, statistics from openai import OpenAI client = OpenAI( base_url="https://openrouter.ai/api/v1", api_key=os.environ["OPENROUTER_API_KEY"], default_headers={"HTTP-Referer": "https://my-app.com", "X-Title": "my-app"}, ) def benchmark_model(model: str, prompt: str = "Say hello", n: int = 5) -> dict: """Benchmark a model's latency over N requests.""" latencies = [] for _ in range(n): start = time.monotonic() response = client.chat.completions.create( model=model, messages=[{"role": "user", "content": prompt}], max_tokens=50, ) latencies.append((time.monotonic() - start) * 1000) return { "model": model, "p50_ms": round(statistics.median(latencies)), "p95_ms": round(sorted(latencies)[int(len(latencies) * 0.95)]), "avg_ms": round(statistics.mean(latencies)), "min_ms": round(min(latencies)), "max_ms": round(max(latencies)), } # Compare fast vs slow models for model in ["o...