notion-load-scale

# Notion Load & Scale ## Overview Patterns for high-volume Notion API usage within the 3 requests/second rate limit. Covers parallel request orchestration with `p-queue`, worker queue architecture for background processing, full database pagination at scale (100K+ records), incremental sync using `last_edited_time` filters to avoid re-fetching unchanged data, and memory management for bulk operations using streaming and chunked processing. ## Prerequisites - `@notionhq/client` v2.x installed (`npm install @notionhq/client`) - `p-queue` for rate-limited concurrency (`npm install p-queue`) - Python: `notion-client` installed (`pip install notion-client`) - `NOTION_TOKEN` set (each token gets its own 3 req/s limit) - Test database in Notion (dedicated for load testing) ## Instructions ### Step 1: Parallel Requests Within Rate Limits Notion enforces 3 requests/second per integration token. Use `p-queue` to maximize throughput without hitting 429 errors. ```typescript import { Client } from '@notionhq/client'; import PQueue from 'p-queue'; const notion = new Client({ auth: process.env.NOTION_TOKEN }); // Rate-limited queue: 3 requests per second, single concurrency // Use intervalCap + interval instead of concurrency alone const apiQueue = new PQueue({ concurrency: 1, interval: 340, // ~3 per second with safety margin intervalCap: 1, }); // Metrics tracking let totalRequests = 0; let rateLimitHits = 0; const startTime = Date.now(); function logThroughput() {...