free-tier-scraper-apislisted

# Free-Tier APIs for Scraper Pipelines Reference for the common free stack: `requests`/BeautifulSoup or Playwright for collection, Gemini free tier for enrichment, Notion API for storage. ## Gemini free tier Model IDs and free-tier limits (API key from AI Studio, no billing): | Model | RPM | RPD | |---|---|---| | `gemini-2.0-flash-lite` | 30 | 1500 | | `gemini-2.0-flash` | 15 | 1500 | | `gemini-2.5-flash` | 10 | 500 | | `gemini-flash-lite-latest` | alias, tracks current flash-lite | - | ### REST endpoint ``` POST https://generativelanguage.googleapis.com/v1beta/models/{MODEL}:generateContent?key={API_KEY} ``` Payload for JSON-out enrichment: ```json { "contents": [{"parts": [{"text": "your prompt"}]}], "generationConfig": { "responseMimeType": "application/json", "temperature": 0.3, "maxOutputTokens": 2048 } } ``` Response text lives at `candidates[0].content.parts[0].text`. Even with `responseMimeType: application/json` some models wrap output in ` ```json ` fences; strip them before `json.loads`. Keep `maxOutputTokens` at 2048+ for batch responses, otherwise truncated JSON fails to parse. ### Fallback chain and batching - On HTTP 429 (quota) or 404 (model retired), retry the same prompt on the next model: `gemini-2.0-flash-lite -> gemini-2.0-flash -> gemini-2.5-flash -> gemini-flash-lite-latest`. Model IDs churn; the `-latest` alias survives retirements. - Batch items into one prompt (about 5 per call, response as `{"analyses": [...]}` in input