web-scraping-lead-datalisted

# Scraping for Lead Data When an ICP is defined by something no database has a field for — companies present on a marketplace, running a particular integration, publishing a certain kind of page — the data has to come off the web directly. Two things decide whether this works: extracting into a schema rather than free text, and staying within limits that keep you unblocked and defensible. ## Extract into a schema The most common failure is scraping page text and trying to parse meaning out of it later. Decide the shape first, then extract into it. ```json { "company_name": "string, required", "domain": "string, required, registrable domain only", "category": "string | null", "location": "string | null", "employee_hint": "string | null", "contact_name": "string | null", "contact_title": "string | null", "email": "string | null", "signal": "string | null, the qualifying fact", "source_url": "string, required", "extracted_at": "ISO 8601, required" } ``` Three fields are non-negotiable and routinely missing: - **`source_url`** — where each fact came from. It makes claims checkable, it's how you catch fabrication, and it's what makes "where did you get my details" answerable in one sentence. - **`extracted_at`** — data has a shelf life. Without a timestamp you can't tell fresh from stale. - **Nullable everything optional** — a scraper that must fill every field will produce garbage for the pages where th