This article follows the official documentation. We haven’t tested this tool for a hands-on review.
When a coding agent needs to inspect files stored in Amazon S3, it can use s5cmd from the shell to list objects and download selected files. S3 stores files as objects inside buckets, and each object has a key, its name within the bucket. s5cmd is a command-line tool for working with S3 and compatible storage, as well as local files.
The agent can run common storage commands directly instead of writing a custom script. A read-only listing helps it find the right object; downloading that object writes a local copy but does not change the remote file. The s5cmd documentation explains how to run commands on many objects and how to run commands from a file. It also describes a dry-run option for previewing supported write commands without making changes.
What the agent can do
An agent can use s5cmd to list, copy, sync, or delete objects. It can also run multiple commands from a file. The tool runs work in parallel, which helps with large batches but means dependent commands should not be placed in a batch that runs them concurrently.
A coding agent can locate a dataset export, then download one object. The agent can inspect the downloaded object with the project’s existing tools. s5cmd ls lists objects at an S3 path. s5cmd cp copies an object between S3 and the local filesystem. The official command examples show these operations and the tool’s support for wildcards.
Install and authenticate
On macOS, install s5cmd with Homebrew by running the command listed in the official repository. The repository’s releases page also has prebuilt binaries for Linux, macOS and Windows.
brew install peak/tap/s5cmd
Then check the installed command’s options:
s5cmd --help
s5cmd uses the official AWS SDK, so it reads AWS credentials in the usual ways. You can pass a named profile with --profile or set the standard AWS environment variables. On an EC2 instance or in EKS, it can also use the IAM role, the AWS permissions attached to that machine. The repository documents these options. The agent needs access to credentials through its execution environment or configured AWS profile. Keep secrets there; do not paste them into the agent prompt. The credentials’ permissions determine which bucket operations succeed.
List and download an object
Replace YOUR_BUCKET with a bucket the configured credentials can access. The path below is an example and is not a public test location:
s5cmd ls 's3://YOUR_BUCKET/datasets/'
The command contacts S3 and prints matching objects. Copy an object key from the listing into the next command, replacing OBJECT_KEY with the actual key. Then create a local directory and download the object:
mkdir -p ./agent-data
s5cmd cp 's3://YOUR_BUCKET/datasets/OBJECT_KEY' ./agent-data/
The first command makes a network request to list remote objects. The second downloads one object and creates a local copy; it does not modify the S3 object. The local directory command changes only the agent’s working environment. Review the listing before asking an agent to download objects under a broad prefix. A prefix is the start of an object's key, and one prefix can match many objects.
A concise prompt for this task is: “List objects under s3://YOUR_BUCKET/datasets/, find the export relevant to this code change, download only that object to ./agent-data/, and summarize its contents. Do not upload or delete anything.” Replace the bucket placeholder before use. The prompt should describe the task, not carry credentials.
Preview writes and watch parallel commands
For a later upload, s5cmd’s documented --dry-run option prints the operations a command would perform without carrying them out. It works with commands that change something, like cp, mv and rm. Quote wildcard patterns so the shell passes them to s5cmd rather than expanding them first:
s5cmd --dry-run cp './reports/*.parquet' 's3://YOUR_BUCKET/reports/'
Replace the bucket and local path with real values. Review the preview first. When you run the same cp command without --dry-run, s5cmd makes the remote change. Check the installed version’s help and the repository documentation for current option syntax.
When you run a batch, s5cmd run executes commands from a file with several parallel workers. Do not rely on one line finishing before another starts, such as when a later command depends on a file created by an earlier one. Run dependent commands separately. Use parallel commands for independent storage tasks. Run shell commands sequentially when each step needs the result of the previous step.