← ClaudeAtlas

dolphindb-data-importlisted

DolphinDB data import guide for IoT and general scenarios. Covers CSV/TXT, JSON, binary, plugin-based (HDF5, Parquet, MySQL, ODBC), and IoT real-time ingestion (MQTT, OPC UA). Includes IOTDB engine, date/time conversion, array vector import, null handling, and binary record import. Does NOT cover database administration, permission management, stream compute engine configuration, or data export.
dolphindb/DolphinX_Skill · ★ 8 · Data & Documents · score 63
Install: claude install-skill dolphindb/DolphinX_Skill
## DolphinDB Data Import Guide Use this skill when the user asks about importing, loading, ingesting, or migrating data into DolphinDB, including CSV/text files, JSON, binary files, external databases, Parquet/HDF5 files, or IoT real-time data streams. ### Import Method Decision Tree 1. **CSV / text file → memory table**: use `loadText` (single-thread) or `ploadText` (parallel, faster for files > 16 MB). 2. **CSV / text file → distributed table**: use `loadTextEx` (recommended, direct-to-disk, avoids OOM for large files). 3. **Very large CSV file (> memory)**: use `textChunkDS` + `mr` to chunk and import. 4. **JSON string**: use `fromStdJson` (standard JSON) or `parseJsonTable` (JSON to table). 5. **Binary file with string columns**: use `loadRecord(filename, schema)`. Schema must be provided by user. 6. **Binary file without string columns**: use `readRecord!(handle, holder)`. Preferred for pure numeric files — simpler and safer. 7. **Parquet file → memory table**: use `parquet::loadParquet(filePath, [schema], [columnsToLoad], [startRowGroup], [rowGroupNum])`. Supports partial column loading via `columnsToLoad` and row group selection via `startRowGroup`/`rowGroupNum`. 8. **Parquet file → distributed table**: use `parquet::loadParquetEx(dbHandle, tableName, partitionColumns, fileName, [schema], [columnsToLoad], [startRowGroup], [rowGroupNum], [transform])`. Note: no `sortColumns` parameter; if sortColumns is needed, use `parquet::loadParquet` + `replaceColumn!` + `append!