firecrawl-rate-limits

Featured

Implement Firecrawl rate limiting, backoff, and request queuing patterns. Use when handling 429 errors, implementing retry logic, or optimizing API request throughput for Firecrawl. Trigger with phrases like "firecrawl rate limit", "firecrawl throttling", "firecrawl 429", "firecrawl retry", "firecrawl backoff".

AI & Automation 2,926 stars 433 forks Updated yesterday MIT

Install

View on GitHub

Quality Score: 99/100

Stars 20%
100
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Firecrawl Rate Limits ## Overview Firecrawl enforces rate limits per API key measured in requests per minute and concurrent connections. When exceeded, the API returns `429 Too Many Requests` with a `Retry-After` header. This skill covers backoff strategies, request queuing, and proactive throttling. ## Rate Limit Tiers | Plan | Scrape RPM | Crawl Concurrency | Credits/Month | |------|-----------|-------------------|---------------| | Free | 10 | 2 | 500 | | Hobby | 20 | 3 | 3,000 | | Standard | 50 | 5 | 50,000 | | Growth | 100 | 10 | 500,000 | | Scale | 500+ | 50+ | Custom | Concurrent crawl jobs count against concurrency limits. If the queue is full, new jobs are rejected with 429. ## Instructions ### Step 1: Exponential Backoff with Jitter ```typescript import FirecrawlApp from "@mendable/firecrawl-js"; const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY!, }); async function withBackoff<T>( operation: () => Promise<T>, config = { maxRetries: 5, baseDelayMs: 1000, maxDelayMs: 32000 } ): Promise<T> { for (let attempt = 0; attempt <= config.maxRetries; attempt++) { try { return await operation(); } catch (error: any) { if (attempt === config.maxRetries) throw error; const status = error.statusCode || error.status; // Only retry on 429 (rate limit) and 5xx (server error) if (status && status !== 429 && status < 500) throw error; // Exponential delay with random jitter to prevent thundering h...

Details

Author
jeremylongshore
Repository
jeremylongshore/claude-code-plugins-plus-skills
Created
1 years ago
Last Updated
yesterday
Language
Python
License
MIT

Integrates with

Bundled in these plugins

Similar Skills

Semantically similar based on skill content — not just same category