apify-performance-tuning

Featured

Optimize Apify Actor performance: crawl speed, memory usage, concurrency, and proxy rotation. Use when Actors are slow, consuming too much memory, or being blocked by target sites. Trigger: "apify performance", "optimize apify actor", "apify slow", "crawlee concurrency", "apify memory tuning", "scraper performance".

AI & Automation 2,926 stars 433 forks Updated yesterday MIT

Install

View on GitHub

Quality Score: 99/100

Stars 20%
100
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Apify Performance Tuning ## Overview Optimize Apify Actors for speed, cost, and reliability. Covers Crawlee concurrency settings, memory profiling, proxy rotation strategies, request batching, and crawler selection for different workloads. ## Prerequisites - Existing Actor with measurable baseline performance - Understanding of `apify-sdk-patterns` - Access to Actor run stats in Apify Console ## Performance Baseline Measure before optimizing. Key metrics from run stats: ```typescript const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.run('RUN_ID').get(); console.log({ totalDurationSecs: run.stats?.runTimeSecs, pagesPerMinute: (run.stats?.requestsFinished ?? 0) / ((run.stats?.runTimeSecs ?? 1) / 60), failedRequests: run.stats?.requestsFailed, retryRequests: run.stats?.requestsRetries, memoryAvgMb: run.stats?.memAvgBytes ? run.stats.memAvgBytes / 1e6 : null, memoryMaxMb: run.stats?.memMaxBytes ? run.stats.memMaxBytes / 1e6 : null, computeUnits: run.usage?.ACTOR_COMPUTE_UNITS, costUsd: run.usageTotalUsd, }); ``` ## Instructions ### Step 1: Choose the Right Crawler | Crawler | Speed | JS Rendering | Memory | Use When | |---------|-------|-------------|--------|----------| | `CheerioCrawler` | Very fast | No | Low (~50MB) | Static HTML, SSR pages | | `PlaywrightCrawler` | Moderate | Yes | High (~200MB/page) | SPAs, dynamic content | | `PuppeteerCrawler` | Moderate | Yes | High (~200MB/page) | Chromium-specific ...

Details

Author
jeremylongshore
Repository
jeremylongshore/claude-code-plugins-plus-skills
Created
1 years ago
Last Updated
yesterday
Language
Python
License
MIT

Integrates with

Bundled in these plugins

Similar Skills

Semantically similar based on skill content — not just same category