geo-data-engineering

Solid

Always invoke when geospatial data must be acquired, prepared, repaired, scaled, or moved through a repeatable pipeline. Covers open-data/OSM/STAC acquisition, spatial formats, CRS transforms, quality checks, and batch ETL architecture for growing or recurring joins. Invoke alongside PostGIS for database execution and alongside SWE standards when code is delivered. Do not trigger merely because another specialist reads analysis-ready data.

Data & Documents 3 stars 0 forks Updated today MIT

Install

View on GitHub

Quality Score: 79/100

Stars 20%
20
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
80
License 10%
100
Description 5%
100

Skill Content

# Geospatial Data Engineering Purpose: get spatial data into a clean, validated, analysis-ready state with a repeatable pipeline — the stage where most real-world GIS time is spent and most silent errors are born. ## Format selection | Format | Use for | Avoid because | |---|---|---| | **GeoParquet** | Analysis interchange, big vector, columnar workflows | Not yet readable by some legacy desktop GIS | | **GeoPackage** | Desktop GIS exchange, multi-layer projects | Slower than Parquet at scale; SQLite locking | | **FlatGeobuf** | Streaming, HTTP range reads | Single layer | | **COG** (Cloud-Optimized GeoTIFF) | All raster deliverables | — (make every GeoTIFF a COG) | | **Zarr/NetCDF** | Multi-dimensional (time × band × y × x) | Overkill for single rasters | | Shapefile | Only when a legacy tool demands it | 10-char columns, 2 GB cap, encoding chaos, multi-file fragility | | CSV + WKT/lon-lat | Simple point exchange | No CRS metadata — document it explicitly | ## Acquisition playbook - **OpenStreetMap**: small areas → `osmnx`; large extracts → Geofabrik PBF + `pyrosm`/`osmium`. Respect tag heterogeneity: always inspect tag value distributions before filtering. - **Buildings/places at scale**: Overture Maps (GeoParquet on S3/Azure, query with DuckDB spatial — often the fastest path). - **Satellite/raster**: STAC APIs via `pystac-client` + `odc-stac` — see `remote-sensing-analysis`; planetary archives → `google-earth-engine`. - **Boundaries**: authoritative national ...

Details

Author
muend
Repository
muend/geoai-skills
Created
1 weeks ago
Last Updated
today
Language
Python
License
MIT

Integrates with

SQLite · Database

Similar Skills

Semantically similar based on skill content — not just same category

API & Backend Solid

postgis-spatial-sql

Invoke whenever spatial SQL or its execution backend is the decision: PostGIS, DuckDB Spatial, SpatiaLite, ST_* functions, recurring spatial joins, concurrent/growing workloads, or large GeoParquet queries. Covers backend selection, schemas, GiST/BRIN indexes, KNN, geometry versus geography, correctness benchmarks, and EXPLAIN optimization. Use PostGIS for managed concurrent services and embedded engines for bounded local analytics when evidence supports that choice. Use geo-data-engineering for acquisition, conversion, and file-based ETL without spatial SQL.

3 Updated today
muend
AI & Automation Featured

geopandas

Guidance and local audit tools for Python workflows that directly use GeoPandas GeoSeries, GeoDataFrame, spatial operations, or vector-data I/O.

31,883 Updated today
K-Dense-AI
AI & Automation Featured

geomaster

Comprehensive geospatial science skill covering remote sensing, GIS, spatial analysis, machine learning for earth observation, and 30+ scientific domains. Supports satellite imagery processing (Sentinel, Landsat, MODIS, SAR, hyperspectral), vector and raster data operations, spatial statistics, point cloud processing, network analysis, cloud-native workflows (STAC, COG, Planetary Computer), and 8 programming languages (Python, R, Julia, JavaScript, C++, Java, Go, Rust) with 500+ code examples. Use for remote sensing workflows, GIS analysis, spatial ML, Earth observation data processing, terrain analysis, hydrological modeling, marine spatial analysis, atmospheric science, and any geospatial computation task.

31,883 Updated today
K-Dense-AI