scraperapi-research-agentlisted

# ScraperAPI Research Agent End-to-end autonomous research: ScraperAPI finds and fetches sources → Anthropic Files API ingests them as cited documents → Claude synthesizes a report. **Run it:** ```bash # Install dependencies pip install requests anthropic # Set env vars export SCRAPERAPI_API_KEY=your-key export ANTHROPIC_API_KEY=your-key # Run python skills/scraperapi-research-agent/scripts/research_agent.py \ --question "What are the best practices for rate limiting in web APIs?" \ --max-sources 5 \ --output report.md ``` See [scripts/research_agent.py](scripts/research_agent.py) for the full implementation. --- ## Planning Checklist Before starting a research run, establish: - [ ] **Question clarity** — Is the question specific enough to produce useful search queries? Vague questions like "tell me about AI" produce noise. Better: "What are the tradeoffs between RAG and fine-tuning for domain-specific LLMs?" - [ ] **Source count** — How many sources are needed? 3–5 is usually sufficient for a factual summary; 8–10 for a comparative analysis. More sources = more ScraperAPI credits. - [ ] **Recency** — Does the answer depend on recent events? Search queries will use recent date filters. - [ ] **Credit budget** — Each source costs ~1 credit to scrape (more with JS rendering). 5 sources = ~5–10 credits total. - [ ] **Stop condition** — Define when to stop. The default stop is `--max-sources` (5). Do not loop indefinitely. --- ## Research Loop ``` 1. PLAN ↓