langfuse-rate-limits

# Langfuse Rate Limits ## Overview Handle Langfuse API rate limits with optimized SDK batching, exponential backoff with jitter, concurrent request limiting, and configurable sampling for ultra-high-volume workloads. ## Prerequisites - Langfuse SDK installed and configured - High-volume trace workload (1,000+ events/minute) ## Instructions ### Step 1: Optimize SDK Batching Configuration The Langfuse SDK batches events internally before sending. Tuning batch settings is the first defense against rate limits. ```typescript // v3 Legacy: Direct configuration import { Langfuse } from "langfuse"; const langfuse = new Langfuse({ flushAt: 50, // Events per batch (default: 15, max ~200) flushInterval: 10000, // Milliseconds between flushes (default: 10000) requestTimeout: 30000, // Timeout per batch request }); // v4+: Configure via OTel span processor import { LangfuseSpanProcessor } from "@langfuse/otel"; import { NodeSDK } from "@opentelemetry/sdk-node"; const processor = new LangfuseSpanProcessor({ exportIntervalMillis: 10000, // Flush interval maxExportBatchSize: 50, // Events per batch }); const sdk = new NodeSDK({ spanProcessors: [processor] }); sdk.start(); ``` ### Step 2: Implement Retry with Exponential Backoff For custom API calls (scores, datasets, prompts) that hit rate limits: ```typescript async function withRetry<T>( fn: () => Promise<T>, options: { maxRetries?: number; baseDelayMs?: number; maxDelayMs?: number } = {} ): Prom...