brightdata-performance-tuning

# Bright Data Performance Tuning ## Overview Optimize Bright Data scraping performance through connection pooling, response caching, concurrent request tuning, and smart product selection. Web Unlocker latency is typically 5-30s due to CAPTCHA solving; Scraping Browser sessions are 10-60s. ## Prerequisites - Bright Data zone configured - Understanding of async patterns - Redis or file cache available (optional) ## Latency Benchmarks | Product | P50 | P95 | P99 | Notes | |---------|-----|-----|-----|-------| | Web Unlocker (simple) | 3s | 8s | 15s | No CAPTCHA | | Web Unlocker (CAPTCHA) | 10s | 25s | 45s | With CAPTCHA solving | | Scraping Browser | 8s | 20s | 40s | Full browser render | | SERP API (sync) | 2s | 5s | 10s | Search results | | Residential Proxy | 1s | 3s | 8s | Raw proxy, no unblocking | ## Instructions ### Step 1: Choose the Right Product ```typescript // Product selection matrix function selectProduct(target: { js: boolean; captcha: boolean; structured: boolean }) { if (target.structured) return 'serp_api'; // Pre-parsed JSON if (!target.js && !target.captcha) return 'residential'; // Fastest if (target.js) return 'scraping_browser'; // Browser rendering return 'web_unlocker'; // Best default } ``` ### Step 2: Connection Pooling with Keep-Alive ```typescript import { Agent } from 'https'; import axios from 'axios'; // Reuse TCP connections to brd.superproxy.io const httpsAgent = new Agent({ keepAli...