Newman for coding agents

When a coding agent changes an API, it needs a way to check whether the API still behaves as its tests expect. Newman runs Postman collections from a shell. A collection is a saved set of API requests and checks, such as a request to a health endpoint and a check that it returns the expected status.

An agent with shell access can run Newman after changing API code and use the report to find failing checks. Newman sends the requests defined in the collection, so the collection determines whether a run stays local or contacts another service. Newman provides a command-line runner rather than an agent integration. The agent runs it as a shell command, like any other command. The Newman README on GitHub is the full reference for its options and reporters.

Postman’s documentation now says that Newman won’t receive updates or fixes. Postman recommends its newer Postman CLI for new scripts and CI pipelines, and it publishes a guide for moving Newman commands over. Newman runs collections in the v2.1 JSON format. The Postman CLI also runs the newer v3 format. The same page says Newman doesn’t support OAuth 2.0 authentication or scripts from a team’s Package Library. The steps below still work for an existing Newman setup.

Install Newman

Newman requires Node.js v16 or later. If Node.js and npm are installed, install Newman globally with:

npm install -g newman

The Newman repository also documents Homebrew installation with brew install newman. The global npm command makes newman available in the shell. An agent can then run a collection file with newman run path/to/collection.json.

Run a local API collection

Suppose a repository contains ./tests/health.postman_collection.json, a Postman collection with a health request that uses the variable {{baseUrl}}. The example below points that request at a service listening on the developer’s machine; replace the port placeholder before running it. Replace the collection path or base URL if your repository uses different values.

mkdir -p artifacts

newman run ./tests/health.postman_collection.json \
  --env-var "baseUrl=http://127.0.0.1:<port>" \
  --reporters cli,json \
  --reporter-json-export ./artifacts/newman.json

Start the local service before running this command. Newman sends the requests in the collection to that service; it does not start the service or inspect the API code directly. The cli reporter prints progress and test results in the terminal. The json reporter writes a machine-readable run report to ./artifacts/newman.json. The output file is local, while the API requests go to the address configured by baseUrl. Newman’s reporter options describe the available built-in formats and export flags.

The example assumes the collection defines checks, such as a check that the response has the expected status. Newman reports whether those checks pass or fail; the exact output depends on the collection and the run. If the collection uses another variable name, change baseUrl to match it. Review the collection’s request URLs and methods before running it: a collection can send requests that change data, even when the example points to a local service.

Handle API credentials

Newman does not require a Postman account to run a collection saved as a local file. However, requests in that collection may require credentials for the API they call. Keep credentials outside the agent prompt and pass them through the shell environment or a protected environment file. For example, when the collection expects a variable named token, the command can use a shell variable:

newman run ./tests/api.postman_collection.json \
  --env-var "token=$API_TOKEN"

Set API_TOKEN through the developer’s normal secret-handling method before running the command. Do not commit credentials in the collection or report. A local collection can still contact a remote API if its request URLs point there, so check those URLs before allowing an agent to run it.

Give an agent a focused task

Newman fits a task where an agent checks an API change against the existing tests for individual API requests. A useful prompt names the collection, the service target, and the expected follow-up:

Run the health checks in tests/health.postman_collection.json against the local service at its configured URL. Review the collection before running it. Report any failed checks and the relevant response details. Do not change the collection or call a remote service.

The agent can run Newman, read the terminal report, and inspect the JSON report when it needs structured details. Newman runs only the requests and checks already defined in the collection. If the collection has no request for a behavior or check for its result, Newman will not test that behavior.