harvest-structured

Solid

Structured data extraction - tables, pricing, products, API endpoints with schema

AI & Automation 552 stars 45 forks Updated 6 days ago MIT

Install

View on GitHub

Quality Score: 92/100

Stars 20%
91
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Harvest Structured Extract structured data from web pages using user-defined schemas. Turns messy HTML into clean JSON/CSV - pricing tables, product listings, API endpoint docs, comparison matrices. ## Usage ``` /scrape <url> --schema "<field descriptions>" ``` ## Examples ```bash # Extract pricing data /scrape https://example.com/pricing --schema "plan_name, price, features[], cta_text" # Extract product listings /scrape https://store.example.com/products --schema "name, price, rating, reviews_count, image_url" # Extract API endpoints /scrape https://docs.api.com/reference --schema "method, path, description, parameters[], response_code" ``` ## Schema Definition Define fields as comma-separated names. Use `[]` for arrays: ``` name → Single text value price → Single value (auto-detects currency) features[] → Array of items description → Long text url → Auto-detects links image_url → Auto-detects image sources ``` ## How It Works 1. Fetch page content 2. Parse schema definition 3. Use CSS selectors or LLM extraction to match fields 4. Validate extracted data against schema 5. Output as JSON (default) or CSV ## Output Format ### JSON (default) ```json [ { "plan_name": "Pro", "price": "$29/mo", "features": ["Unlimited projects", "Priority support", "API access"], "source_url": "https://example.com/pricing" } ] ``` ### CSV ```csv plan_name,price,features,source_url Pro,"$29/mo","Unlimited project...

Details

Author
vibeeval
Repository
vibeeval/vibecosystem
Created
8 months ago
Last Updated
6 days ago
Language
C#
License
MIT

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category