langchain-performance-tuning

# LangChain Performance Tuning ## Overview Optimize LangChain apps for production: measure baseline latency, implement caching, batch with concurrency control, stream for perceived speed, optimize prompts for fewer tokens, and select the right model for each task. ## Step 1: Benchmark Baseline ```typescript async function benchmark( chain: { invoke: (input: any) => Promise<any> }, input: any, iterations = 5, ) { const times: number[] = []; for (let i = 0; i < iterations; i++) { const start = performance.now(); await chain.invoke(input); times.push(performance.now() - start); } times.sort((a, b) => a - b); return { mean: (times.reduce((a, b) => a + b, 0) / times.length).toFixed(0) + "ms", median: times[Math.floor(times.length / 2)].toFixed(0) + "ms", p95: times[Math.floor(times.length * 0.95)].toFixed(0) + "ms", min: times[0].toFixed(0) + "ms", max: times[times.length - 1].toFixed(0) + "ms", }; } // Usage const results = await benchmark(chain, { input: "test" }, 10); console.table(results); ``` ## Step 2: Streaming (Perceived Performance) ```typescript import { ChatOpenAI } from "@langchain/openai"; import { ChatPromptTemplate } from "@langchain/core/prompts"; import { StringOutputParser } from "@langchain/core/output_parsers"; const chain = ChatPromptTemplate.fromTemplate("{input}") .pipe(new ChatOpenAI({ model: "gpt-4o-mini", streaming: true })) .pipe(new StringOutputParser()); // Non-streaming: user waits 2-3s...