data-processing

# Data Processing Query, filter, and transform structured data (JSON, YAML, TOML) efficiently from the command line. ## Tools | Tool | Command | Use For | |------|---------|---------| | jq | `jq '.key' file.json` | JSON processing | | yq | `yq '.key' file.yaml` | YAML/TOML processing | ## jq Essentials ```bash # Extract single field jq '.name' package.json # Extract nested field jq '.scripts.build' package.json # Extract from array jq '.dependencies[0]' package.json # Extract multiple fields jq '{name, version}' package.json # Navigate deeply nested jq '.data.users[0].profile.email' response.json # Filter by condition jq '.users[] | select(.active == true)' data.json # Transform each element jq '.users | map({id, name})' data.json # Count elements jq '.users | length' data.json # Raw string output jq -r '.name' package.json ``` ## yq Essentials ```bash # Extract field yq '.name' config.yaml # Extract nested yq '.services.web.image' docker-compose.yml # List all keys yq 'keys' config.yaml # List all service names (Docker Compose) yq '.services | keys' docker-compose.yml # Get container images (K8s) yq '.spec.template.spec.containers[].image' deployment.yaml # Update value (in-place) yq -i '.version = "2.0.0"' config.yaml # TOML to JSON yq -p toml -o json '.' config.toml ``` ## Quick Reference | Task | jq | yq | |------|----|----| | Get field | `jq '.key'` | `yq '.key'` | | Array element | `jq '.[0]'` | `yq '.[0]'` | | Filter array | `jq '.[] \| select(.x...