← ClaudeAtlas

import-data-from-csvlisted

Use when building a bulk CSV/spreadsheet import endpoint — streaming parse, per-row validation with a row-numbered error report, data normalization (Excel quotes, multi-format dates), atomic chunked upsert in a transaction, partial-success response, and fan-out to workers for large files. NestJS/TypeORM reference, framework-flexible.
kennguyen887/agent-foundation · ★ 1 · Data & Documents · score 77
Install: claude install-skill kennguyen887/agent-foundation
# Import data from CSV A bulk-import endpoint (members, products, locations, …) that ingests a CSV/spreadsheet and reports exactly which rows failed. Examples NestJS/TS, neutral `listing` domain. principle → **▸ Example** → **▸ Other stacks**. ## Core principle **Validate per row; never all-or-nothing-silently.** Parse → validate each row → collect valid rows AND a row-numbered error list → write the valid ones atomically → return a **partial-success report** (N imported, M failed + why). The user must learn *which* row failed and *why*, not just "import failed". ## 1. Parse — streaming + normalized - **Stream the CSV** (don't load a huge file fully into memory). Validate the **header row first** (required columns present) before processing data rows; skip empty rows. ```ts csv({ delimiter: 'auto', trim: true }).fromString(normalized) .on('header', (h) => assertRequiredHeaders(h)) .subscribe((row, lineNo) => { /* §2 */ }, onError, () => resolve({ valid, errors })); ``` - **Normalize spreadsheet quirks** first: collapse Excel triple-quotes (`"""x"""` → `"x"`), trim cells, blank → `undefined`, upper-case headers for matching. - **Parse dates defensively** — try a list of accepted formats, then timezone-normalize; if none match, record a row error (don't silently store an invalid/shifted date): ```ts const d = ['DD/MM/YYYY','YYYY-MM-DD','D-MMM-YY'].map((f) => dayjs(v, f, true)).find((x) => x.isValid()); return d ? d.tz(TZ) : null; // null → row er