← ClaudeAtlas

blend-datalisted

Blend two or more Rockfish datasets (typically time series — real + synthetic, several synthetic runs, or several sources with the same shape) into one dataset with a validated common schema, globally unique session keys, optional per-source mixing ratios, a provenance column, and a time-ordered result, then verify it with server-side SQL. Covers the multi-parent workflow pattern (one DatasetLoad → SQL branch per input feeding a single DatasetSave with concat_tables), the single-SQL UNION ALL alternative, and the pitfalls of today's SDK (append failures on schema drift, stale session metadata, un-offset session keys, unsorted output). Trigger on phrases like "blend these datasets", "merge/combine/concatenate two datasets", "mix real and synthetic data", "stack two time series into one", "union these tables", "augment real data with synthetic sessions", "multiple parents into DatasetSave", or "concat_tables".
Rockfish-Data/rockfish-skills · ★ 0 · Data & Documents · score 70
Install: claude install-skill Rockfish-Data/rockfish-skills
# Blend data Combine several datasets that describe the same kind of thing into one dataset — e.g. real `jobs` sessions plus synthetic `jobs` sessions — so the result can be trained on, evaluated, or shipped as a single table. Blending is **row-wise** (stack sessions from every input); joining columns is a different job (use `ra.SQL` with a `JOIN`). The reference implementation, [`reference/blend.py`](reference/blend.py), does the whole flow and is verified end-to-end against a Rockfish backend. Read it before writing blend code; adapt it rather than starting from scratch. ## When to use this skill - The user has two or more dataset IDs (or names) with the same logical schema and wants **one** dataset out. - Mixing **real and synthetic** sessions, or several synthetic runs, possibly at a chosen ratio ("80 real jobs + 160 synthetic"). - The user asks why `DatasetSave(concat_tables=True)` with multiple parents failed or produced odd session keys. Not for: column-wise joins, deduplicating overlapping records, or schema-less appends of unrelated tables. If inputs need real data prep (renames, unit changes, type fixes), do that first — the blend assumes one schema. ## Concept A blend is a DAG with one branch per input converging on one sink: ``` DatasetLoad(A) → SQL(align A) [→ Sample] ─┐ DatasetLoad(B) → SQL(align B) [→ Sample] ─┼→ DatasetSave(concat_tables=True, concat_session_key="session_key") ... ─┘ ``` `DatasetSave` saves the f