← ClaudeAtlas

database-bulk-loadinglisted

Designing and diagnosing high-volume database ingestion from the JVM across PostgreSQL, MySQL, and SQL Server: JDBC batching and statement rewrite, native COPY/LOAD DATA/Bulk Copy, staging, transaction and partial-error semantics, idempotent restart, upsert races, logging, parallelism, and post-load validation. Use when a backfill, import, migration, or batch window is too slow or unsafe. Not routine ORM fetch/write tuning, which belongs to orm-fetch-and-batching-performance.
robsonkades/agent-skills · ★ 2 · API & Backend · score 75
Install: claude install-skill robsonkades/agent-skills
# Database Bulk Loading ## Purpose Choose the least costly ingestion mechanism that still preserves the required validation, atomicity, recoverability, and online workload. “More threads” and “larger batch” are not goals; the job is done when useful rows per second rise without violating correctness or guardrails. ## Inputs required ```text engine, server/driver versions, topology, durability and replication mode: source format/location/trust, rows and bytes, row width, and error distribution: target constraints, triggers, indexes, generated keys, and online traffic: required atomicity, duplicate/upsert semantics, rejection policy, and restart point: current mechanism, batch size, transaction size, throughput, CPU/I/O/log/network, and heap: window/SLO, staging/disk/log headroom, privileges, and rollback constraints: ``` For JVM implementation changes, inspect compiler release/toolchains, runtime images, resolved JDBC driver and ORM versions, connection properties and transaction-manager ownership. This skill imposes no Java baseline; preserve the project's target and dependencies. Without phase timings or failure semantics, propose a bounded pilot and mark mechanism/sizing conclusions conditional. ## Workflow 1. Measure rows/s and bytes/s by phase. Attribute time to client materialization, network round-trips, statement processing, per-row engine work, log/WAL flush, indexes/constraints, or replication. A single total duration cannot select a mechanism. 2. Choose