schema-design-for-scraped-datalisted

# Schema Design for Extracted Data The most expensive mistake in a scraping project is made in the first hour: extracting into whatever shape the page happens to have, then discovering the data is unusable when something tries to consume it. Decide the output shape first. The extractor's job is then to fill it, and every page that can't becomes an explicit, visible gap rather than a silent inconsistency. ## Design the target, not the source Write the schema from what the consumer needs, not from what the page shows. If three sources will feed the same table, the schema is the contract they all conform to — and that's what makes them comparable. ```json { "company_name": "string, required", "domain": "string, required, registrable domain only, lowercase", "category": "string | null", "location_city": "string | null", "location_country":"string | null, ISO 3166-1 alpha-2", "employee_hint": "string | null, free text as published", "employee_count": "integer | null, only when stated numerically", "contact_name": "string | null", "contact_email": "string | null, lowercase, trimmed", "signal": "string | null, the qualifying fact", "signal_date": "date | null, ISO 8601", "source_url": "string, required", "extracted_at": "datetime, required, ISO 8601 with timezone", "extractor_version":"string, required", "extraction_method":"string, required" } ``` ## The four fields that are always required Ever