← ClaudeAtlas

data-sourceslisted

Choose and use the data module's sources (CSV, JDBC, Parquet via DuckDB, JSON/YAML/TOON), the ColumnarDataSource vs forward-only contract, and the uniqueness/key checker. Use when reading tabular data, adding a new data source, running a uniqueness check, or when the user says "data source", "uniqueness check", or "find a key".
adamw7/tools · ★ 9 · Data & Documents · score 74
Install: claude install-skill adamw7/tools
# Data Sources Skill Pick the right data source in the `data` module, respect the schema contract that keeps forward-only sources away from schema-dependent callers, and run the uniqueness checker to find whether a set of columns can serve as a key. ## When to Use - Reading tabular data from CSV, a JDBC query, Parquet, JSON, YAML, or TOON - Adding a brand-new data source - Checking whether columns are unique / finding a smaller key - The user says "data source" / "uniqueness check" / "find a key" ## In-memory vs iterative — pick first Every format ships in two variants: - **`InMemory…`** — loads all rows once (`readAll()`), then runs multiple recursive checks cheaply. Use when the data fits in heap. - **`Iterable…`** — holds one row at a time (tiny heap), but re-reads the source for each recursive pass. Use for large data or streaming. ## The schema contract (don't fight it) - **`IterableDataSource`** — the base forward-only contract: `open()`, `nextRow()`, `hasMoreData()`, `reset()`, `nextRows(int batchSize)`. - **`ColumnarDataSource extends IterableDataSource`** — adds `getColumnNames()` for sources whose columns are known up front. - Forward-only sources that discover keys as they stream — **iterable JSON, YAML, TOON** — deliberately do **not** implement `ColumnarDataSource`. Callers that need the schema (e.g. the uniqueness check) depend on the narrower `ColumnarDataSource`, so a schema-less source can never be handed in and answer with `null`. Don't w