s5cmdlisted

Blazing-fast parallel S3 and local-filesystem data movement with the `s5cmd` CLI (v2.3.0). Use when copying, uploading, downloading, moving, syncing, listing, or deleting objects in S3 or S3-compatible stores (MinIO, Ceph, R2, GCS, B2, Wasabi); when transfers via `aws s3` / `aws s3 cp` / `boto3` are too slow; when batching thousands of object operations; streaming objects to/from stdin/stdout (`cat`, `pipe`); running SQL `select` over CSV/JSON/Parquet objects; generating presigned URLs; reading object metadata/`head`; computing storage usage (`du`); managing buckets (`mb`, `rb`, `bucket-version`); or moving training data, checkpoints, datasets, and backups at scale. Trigger on mentions of 's5cmd', 'S3 sync', 'fast S3 copy/download/upload', 'parallel S3 transfer', or custom S3 endpoints.
4thel00z/s5cmd-skill · ★ 0 · AI & Automation · score 62

Install: claude install-skill 4thel00z/s5cmd-skill

# s5cmd ## Overview `s5cmd` is a massively parallel S3 and local-filesystem command-line tool — typically 10–30x faster than `aws s3` for bulk transfers because it parallelizes both across objects (`--numworkers`, default 256) and within large objects (`--concurrency`, multipart). Use it for any high-throughput S3 data movement. **Core mental model:** `s5cmd [GLOBAL OPTIONS] <command> [command options] <args>`. Global options (auth, `--dry-run`, `--endpoint-url`, `--json`, `--numworkers`) come **before** the command; command options come **after**. **The #1 rule: always quote wildcards.** `s5cmd cp "s3://bucket/*" dir/` — without quotes the shell expands `*` locally and the command does the wrong thing or fails. ## Auth & configuration s5cmd uses the standard AWS credential chain (env vars, `~/.aws/credentials`, IAM roles). Key global options: | Option | Purpose | |---|---| | `--endpoint-url URL` | Custom S3 host (MinIO/Ceph/R2/GCS/Wasabi). Also `$S3_ENDPOINT_URL`. | | `--profile NAME` | Named profile from credentials file. | | `--credentials-file PATH` | Non-default credentials file. | | `--no-sign-request` | Anonymous access to public buckets (no creds loaded). | | `--request-payer requester` | Access requester-pays buckets. | | `--no-verify-ssl` | Skip TLS cert verification (self-signed endpoints). | | `--numworkers N` | Parallel workers across objects (default **256**). | | `--retry-count N`, `-r` | Retries per request on failure (default 10). | | `--dry-run` | Pri