firecrawl-core-workflow-a

# Firecrawl Core Workflow A — Scrape & Crawl ## Overview Primary workflow for Firecrawl: convert websites into clean LLM-ready markdown. Covers single-page scraping with `scrapeUrl`, multi-page crawling with `crawlUrl`, async crawl jobs with polling, and content processing pipelines. ## Prerequisites - `@mendable/firecrawl-js` installed - `FIRECRAWL_API_KEY` environment variable set - Target URL(s) identified ## Instructions ### Step 1: Single-Page Scrape ```typescript import FirecrawlApp from "@mendable/firecrawl-js"; const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY!, }); // Scrape a single page to clean markdown const result = await firecrawl.scrapeUrl("https://docs.example.com/api", { formats: ["markdown"], onlyMainContent: true, // strips nav, footer, sidebars waitFor: 2000, // wait 2s for JS to render }); if (result.success) { console.log("Title:", result.metadata?.title); console.log("Source:", result.metadata?.sourceURL); console.log("Markdown:", result.markdown?.substring(0, 200)); } ``` ### Step 2: Multi-Page Synchronous Crawl ```typescript // Crawl a site — Firecrawl follows links, renders JS, returns all pages const crawlResult = await firecrawl.crawlUrl("https://docs.example.com", { limit: 50, // max pages to crawl maxDepth: 3, // link depth from start URL includePaths: ["/docs/*", "/api/*"], // only these paths excludePaths: ["/blog/*", "/changelog/*"], allowB...