web-scraper-api

# Oxylabs Web Scraper API ## Authentication Requires HTTP Basic Auth with credentials from environment variables: ```bash curl -u "$OXY_WSA_USERNAME:$OXY_WSA_PASSWORD" ... ``` ## Endpoint ``` POST https://realtime.oxylabs.io/v1/queries # immediate response POST https://data.oxylabs.io/v1/queries # Push-Pull jobs, callbacks, storage Content-Type: application/json ``` ## Core Parameters | Parameter | Required | Description | |-----------|----------|-------------| | `source` | Yes | Target scraper (e.g., `universal`, `amazon_product`, `google_search`) | | `url` | Conditional | URL to scrape (for `universal` and `*_url` sources) | | `query` | Conditional | Search query or product ID (for `*_search` and `*_product` sources) | | `parse` | No | Enable structured data parsing (recommended for supported sources) | | `render` | No | JavaScript rendering: `html` or `png` | | `geo_location` | No | Geographic targeting: country/state/city, ZIP/postcode, coordinates, or Criteria ID where supported | | `session_id` | No | Reuse the same proxy IP across multiple jobs | | `content_encoding` | No | Set to `base64` when downloading image files via Realtime or Push-Pull | | `user_agent_type` | No | Device/browser preset, e.g., `desktop_chrome`, `mobile_ios`, `tablet_android` | | `locale` | No | Interface language / `Accept-Language`, e.g., `de-DE` | | `callback_url` | No | Push-Pull callback endpoint | | `storage_type`, `storage_url` | No | Push-Pull cloud upload target (`gcs`, `s...