CLI — restura-cli
restura-cli runs your collections headlessly with the same protocol implementations as the desktop and web apps. Drop it into CI to assert your API, get JUnit XML for your dashboard, ship an HTML report to your team.

How it fits
Section titled “How it fits”flowchart LR CI[CI runner] -->|exec| CLI[restura run] CLI -->|load| COL[(OpenCollection<br/>dir / .yaml)] CLI -->|exec| H[HTTP / GraphQL] CLI -->|exec| G[gRPC Connect / gRPC-Web] CLI -->|exec| S[SSE] CLI -->|exec| M[MCP] H --> R1[tui · live · json · junit · html · stats] G --> R1 S --> R1 M --> R1 R1 -->|exit code| CI classDef cli fill:#7b6ef6,stroke:#4f46e5,color:#fff; classDef proto fill:#6366f122,stroke:#7b6ef6,color:#7b6ef6; class CLI cli; class H,G,S,M proto;
Install
Section titled “Install”npm install -g restura-cli# or, no install:npx restura-cli run ./my-collectionRequires Node.js 24+.
Quick start
Section titled “Quick start”Save the collection to disk from the Restura desktop app (collection menu → Save to Files, which writes the OpenCollection directory layout), or export a single OpenCollection .yml from the collection menu → Export → OpenCollection (YAML), then:
restura run ./my-collection --reporter junit --reporter-output junit=results.xmlExit code is 0 when every request passed, 1 if any failed, 2 on internal errors (missing collection, bad flags).
Interactive mode
Section titled “Interactive mode”Run restura (or restura run) with no collection in a terminal to launch a guided wizard — pick the collection, reporters, output paths, an optional env file, and whether to allow localhost — then it runs.

restura # bare → wizardrestura run # also → wizard when no collection is givenBy default an interactive run shows the live tui dashboard; piped output and CI fall back to plain line output (live). See Reporters.
Supported collection formats
Section titled “Supported collection formats”The loader auto-detects three layouts:
| Layout | Detected when… |
|---|---|
| OpenCollection directory (preferred) | the target directory contains opencollection.yml (or .yaml) |
| OpenCollection bundled file | the target path ends in .yaml / .yml |
| Legacy file-collection (deprecated) | the target directory contains _collection.yaml |
The legacy format prints a stderr deprecation warning the first time it’s loaded.
Supported protocols
Section titled “Supported protocols”| Protocol | Notes |
|---|---|
| HTTP / REST | Full support |
| GraphQL | Runs as HTTP with body type graphql. {{vars}} are substituted inside JSON string values of the saved variables, so a value holding " or \ stays valid JSON; variables that are not valid JSON (or not an object) fail the item instead of sending {}. @defer / @stream multipart responses are merged incrementally, so pm.response.json() sees the merged result |
| gRPC | Unary only. Honours the request’s wire protocol: auto / Connect uses the Connect protocol (JSON, no proto compilation needed); gRPC-Web (binary or text) encodes protobuf with the request’s saved schema (reflection descriptors or uploaded .proto) and fails with FAILED_PRECONDITION if there is none; native gRPC (HTTP/2) fails with UNIMPLEMENTED |
| SSE | Captures events for --sse-duration ms, or until --sse-events N. An event without an event: field is a message event, as in the app. Sign-at-wire and challenge auth (AWS SigV4, OAuth 1.0a, WSSE, Digest, NTLM) is refused, not sent anonymously |
| MCP | Runs the saved operation (defaultMethod + defaultParams, e.g. tools/call with {"name":"…","arguments":{…}}, {{var}}s resolved) over one initialized session per server per run, over streamable-http or http-sse. Test scripts get the JSON-RPC reply: rs.response.json().result (incl. isError) or rs.response.json().error. Without assertions, a tool error or JSON-RPC error fails the item; with assertions, the assertions decide. An MCP item with no method fails with an actionable error instead of sending a ping. See MCP. |
| Kafka / MQTT presets | Skipped with a notice on stderr — connection presets open in the desktop app; the CLI has no broker transport |
| WebSocket / Socket.IO collection items | Skipped with a [restura] skipping WebSocket preset "…" (or Socket.IO preset) notice on stderr, like Kafka / MQTT — they open in the app |
CLI vs the app’s collection runner
Section titled “CLI vs the app’s collection runner”The CLI and the in-app collection runner don’t run the same protocols:
| Protocol | CLI (restura run) | App collection runner |
|---|---|---|
| HTTP / GraphQL / gRPC unary | Runs | Runs |
| gRPC streaming | Unary only | Server streams run; client and bidi streams on desktop only |
| SSE | Runs (captures a window of events) | Skipped |
| MCP | Runs | Runs |
| WebSocket / Socket.IO / Kafka / MQTT | Skipped | Skipped |
Command reference
Section titled “Command reference”restura run [collection] [options][collection] accepts a directory (any supported layout) or a bundled .yaml / .yml file. Omit it in a terminal to pick one via the interactive wizard; in CI it is required.
| Flag | Default | Description |
|---|---|---|
-e, --env <file> | JSON or YAML env file: a flat { "key": "value" } object (string values only) or a Postman environment export (*.postman_environment.json; disabled rows skipped). ${VAR} placeholders are expanded from process.env; an unset one expands to an empty string with a [restura] WARNING: ${VAR} is not set… line on stderr (once per name; the exit code is unchanged). A secret or private variable left empty this way counts as missing: a request that uses it is refused, not sent. Hoppscotch environment exports are not read; import those in the app. | |
-g, --globals <file> | Postman globals export (or a flat key/value file). Lowest precedence: below --env. | |
--env-var <kv...> | Set variables, Newman style: --env-var token=abc base=https://x. Wins over --env and collection / folder variables. | |
-n, --iteration-count <n> | Number of iterations. Without --data, runs the collection n times; with --data, the last row repeats past the end. | |
--delay-request <ms> | Pause between requests. | |
--reporter <list> | auto | Comma-separated: tui, live, json, junit, html, stats. Defaults to tui in a terminal, live otherwise (piped / CI). |
--output <file> | Shorthand for a single file reporter. Two file reporters (json, junit, html) resolving to the same path is an error (exit 2); use --reporter-output to give each its own. | |
--reporter-output <kv...> | Per-reporter output: --reporter-output junit=results.xml html=report.html. | |
--bail | false | Stop on first failure. |
--timeout <ms> | 30000 | Per-request timeout. |
--allow-localhost | false | Permit requests to localhost / 127.0.0.1. Off by default (SSRF guard). The guard also checks every address a host name resolves to when the CLI dials it directly (no proxy): a name resolving to cloud metadata (169.254.169.254, e.g. 169.254.169.254.nip.io) or a private range is refused, and one resolving to loopback only with this flag. Through --proxy / HTTP(S)_PROXY the proxy resolves the target. |
--folder <path> | Only run requests under this folder path (slash-joined). | |
--include <pattern...> | Substring or glob (e.g. users/*). Repeatable. | |
--exclude <pattern...> | Same syntax as --include. Applied after. | |
--data <file> | CSV (with header row; the delimiter — , ; tab or | — is detected from the header line and values keep their whitespace, as in the app) or JSON array (.json is JSON, .csv is CSV, other extensions are sniffed: a leading [ means JSON). Runs the collection once per row. A file with no rows or a ragged CSV row is an error (exit 2). | |
--max-iterations <n> | Cap iterations when a --data file is large. | |
--retry <n> | 0 | Retry attempts per failing request. Backoff doubles from 250 ms; a 429 / 503 with Retry-After (seconds or HTTP-date) waits that long instead, capped at 60 s. |
--retry-on <list> | network,5xx | Comma-separated triggers: network, 5xx, 4xx, or specific status codes (429,503). network means the upstream never answered: the connection was refused or reset, DNS failed, or the call timed out before any response (HTTP / GraphQL, gRPC Connect and gRPC-Web, SSE, MCP, and the OAuth2 token request). Any answer is not a network failure, whatever its status: an HTTP 503, an SSE 404 or a gRPC status the server sent. 5xx means an HTTP 5xx, or for gRPC only the transient statuses UNAVAILABLE, DEADLINE_EXCEEDED and RESOURCE_EXHAUSTED; other gRPC statuses (INVALID_ARGUMENT, NOT_FOUND, PERMISSION_DENIED, INTERNAL, …) are attempted once. A refusal is never retried, whatever the triggers: an unresolvable variable or secret (a desktop secret handle, an external secret reference with no resolver), a blocked or unsupported request, NTLM, a URL or redirect the SSRF guard rejects (reported as 400), a wire-signing failure (500 Auth signing failed), and the CLI’s own response-size cap (HTTP 413, gRPC RESOURCE_EXHAUSTED with Response too large). |
--sse-duration <ms> | 5000 | How long to keep SSE streams open. |
--sse-events <n> | Stop SSE early after N events. | |
--insecure | false | Skip TLS certificate verification (self-signed / staging). |
--ca <file> | PEM CA bundle to trust (private CA) without disabling verification. | |
--client-cert <file> | PEM client certificate for mutual TLS (mTLS). | |
--client-key <file> | PEM client private key for mutual TLS. | |
--cert-passphrase <value> | Passphrase for an encrypted --client-key. | |
--proxy <url> | HTTP(S) proxy URL. Overrides HTTP_PROXY / HTTPS_PROXY; composes with the TLS flags. | |
--external-secrets-config <file> | YAML/JSON profiles (AWS Secrets Manager, Google Secret Manager, Azure Key Vault) for resolving external secret references. Identity metadata only — never credentials. Without it, a request that uses an external reference errors instead of being sent unauthenticated. |
Per-request TLS and proxy. An HTTP/GraphQL request’s own TLS and proxy settings (the desktop app’s request Settings: verifySsl: false, caCert, clientCert PEM or PFX, minTlsVersion, cipherSuites, serverCipherOrder, and an HTTP(S) proxy with its bypassList and Basic proxy auth) are applied to that request on a dedicated connection, layered over the run’s TLS flags (verifySsl: true is the default and does not undo --insecure); a request-level proxy replaces --proxy for that request. Settings the CLI cannot apply refuse the request instead of sending it differently: a SOCKS proxy, or a client-cert passphrase / proxy password held in the desktop keychain or an external secret store. Without --proxy, a request with its own TLS options connects directly rather than through the HTTP_PROXY / HTTPS_PROXY env vars, and restura run prints one warning for the run when that applies. The CLI agent tools (DNS-pinned transport) refuse requests that carry such settings. Other protocols (gRPC, SSE, WebSocket, MCP) use the run’s flags only.
OWS workflows
Section titled “OWS workflows”restura workflow run <workspace> <workflow-id> [--env ci.env] [--timeout 30000] [--allow-localhost] [--allow-mutations]--allow-localhost permits loopback targets and --allow-mutations permits saved GraphQL mutation calls; both are off by default. Ctrl-C / SIGTERM stops the workflow cooperatively (status stopped, exit 1).
Variables follow the app’s workflow runner: the workflow starts from the --env values with the collection’s enabled variables over them (collection wins, as in restura run), so if: conditions and set tasks see collection values. Each saved-request call then layers the request’s folder variables (outer → inner) over that, and values earlier workflow steps produced or changed win over both.
The workspace must be an OpenCollection directory with opencollection.yml and a complete workflows/<workflow-id>/ artifact set. The command accepts only Restura’s safe OWS profile: do, set, wait, timeout/cancellation, and binding-only HTTP calls to saved requests. It rejects inline endpoints, credentials, non-HTTP calls, and controls without a bounded runtime before dispatching any network request.
Scripts and assertions
Section titled “Scripts and assertions”Pre-request and test scripts run in a sandboxed QuickJS WASM VM — no DOM, no filesystem, no network escape; 64 MB memory cap, 5 s execution timeout. pm.sendRequest, pm.cookies, and pm.vault are not wired in the CLI and reject with a clear error. See Scripts for the full Postman API surface.
name: Get usermethod: GETurl: "{{API_BASE}}/users/1"testScript: | pm.test("status is 200", () => pm.response.to.have.status(200)); pm.test("response has name", () => { pm.expect(pm.response.json()).to.have.property("name"); });When a test script runs and defines any pm.test(...) assertion, those drive pass/fail. Otherwise pass/fail falls back to the transport outcome (HTTP 2xx, gRPC OK, etc.).
Variables set inside a script (pm.environment.set('K', 'v')) propagate to subsequent requests in the same run, including later iterations. A pre-request pm.execution.skipRequest() sends nothing and skips the test script; the request counts as passed and as skipped in the summary.
Collection, folder and request scripts of one phase run in order, each in its own block scope, so the same const / let at two levels is not a redeclaration error.
Variables
Section titled “Variables”Layered sources, in order of precedence (later wins):
--globalsfile, then the--envfile- Collection variables (declared in
opencollection.ymlor_collection.yaml, or the Postman collection’s variables). Apm.collectionVariables.setin a script updates this layer for later requests in the run, as in Newman and the app (the file on disk is not changed). A secret collection variable has no value in the file: supply it with--env,--env-varor a--datarow, or set it from an earlier script. A request that is about to be sent while it still has no value (an empty value counts as none) is reported as not sent and fails; the rest of the run carries on. - Folder variables of the folders above the request, outer → inner (inner wins). Secret, private and handle-backed folder variables have no value in the file: like a secret collection variable, a request that is about to be sent while one it uses still has no value is reported as not sent and fails.
--env-varvalues- Values a script set in an earlier iteration
- Iteration row (when
--datais set) - Values a script set earlier in this iteration
Substitutions use {{NAME}}. A name is any text without braces, trimmed of surrounding whitespace, so {{my key}} and {{ a:b }} both work. Unknown vars are left in place so the upstream sees them and you notice the gap. A substituted value is not expanded again, except that a value of {{$randomFirstName}} still generates.
Header-based auth (Bearer, Basic, API-key, OAuth2) follows the app’s rule: Basic with an empty username and a password sends Basic base64(":password"), and with neither sends no header. HTTP, MCP, SSE and gRPC items resolve external secret references through the profile given with --external-secrets-config; with no resolver for a reference (or a desktop secret handle) the item is refused, never retried, instead of being sent without credentials. For OAuth2, a pre-supplied access token is used as-is; if the auth instead carries a token endpoint + client id (and no token), the CLI fetches one via the client_credentials grant, caching it for the run only. The token request goes through the same transport as the run: --ca, --insecure, --client-cert/--client-key, --proxy and --timeout apply to it, and a token endpoint that redirects is refused.
Dynamic helpers
Section titled “Dynamic helpers”Postman-compatible {{$random*}} / {{$timestamp}} helpers are expanded after user var substitution:
| Helper | Example |
|---|---|
{{$randomUUID}} | f4d2e3... |
{{$timestamp}} | 1700000000 |
{{$isoTimestamp}} | 2026-05-22T13:42:00Z |
{{$randomEmail}} | alice.42@example.com |
{{$randomFirstName}} | Olivia |
{{$randomIP}} | 192.0.2.4 |
Full list in shared/variables/dynamic.ts.
Data-driven runs
Section titled “Data-driven runs”restura run ./users-api --data ./users.csv --reporter junit --reporter-output junit=junit.xmlusername,rolealice,adminbob,viewercharlie,editorEach row exposes username and role as variables, overriding any same-named env or collection variable for that iteration only. Reporter output groups results by iteration index.
Reporters
Section titled “Reporters”The default is chosen automatically: the live tui dashboard when stdout is a terminal, live lines otherwise (piped output / CI). NO_COLOR only turns colour off (the dashboard and the wizard still run in a terminal); FORCE_COLOR colours piped output without making it interactive. Names and error messages from collections and servers are printed with terminal control sequences stripped.
Live, in-place dashboard with a progress bar and a spinner on the in-flight request. The default in an interactive terminal; on completion it collapses to a clean failure list + summary.
Plain one-line-per-request progress — the default when output is piped or in CI (no ANSI cursor control). Force it with --reporter live if the dashboard misbehaves. It also prints each script console.* line (pre-request, then test) to stderr under its request, as [test log] …, with control characters stripped; pm.test results are shown as assertions, not repeated as console lines. The tui dashboard does not print them.
The RunResult dumped as JSON. Path required (--output or --reporter-output json=...). Each request is reported as { relativePath, folderPath, type, request: { name, type, method, url, auth } } with every auth secret field shown as [redacted]; request headers, bodies, scripts and variables are not written. The url is written as saved in the collection, query string included and {{variables}} unresolved, so a credential typed literally into a URL does reach the artifact. Response bodies are not written. From the response, each setCookies entry keeps only its cookie name (sid=[redacted]), and credential headers are shown as [redacted]: Set-Cookie, Set-Cookie2, Authorization, Proxy-Authorization, the app’s shared credential-header list and api-key / x-*-key / x-*-token / x-*-secret names. Script console.* output is written per request as consoleLogs: [{ phase, type, message }]. Secrets in error and assertion messages and in console lines are redacted by the same token scrubber, but a script that logs a credential in a shape the scrubber does not recognise does put it in the artifact. Still written as received: status, timings, other response headers, and SSE / WebSocket stream events (so a credential an upstream puts in an event or an unrecognised custom header does reach the artifact). The HTML and JUnit reports use the same redacted data.
JUnit XML for CI dashboards (GitLab, Azure DevOps, Jenkins, GitHub Actions test reporters). One <testcase> per request; a request skipped by pm.execution.skipRequest() carries <skipped/> and is counted in the suite’s skipped attribute.
Self-contained HTML page with embedded data + summary table. The embedded data is redacted like the json report.
Latency percentiles (p50/p90/p95/p99) + throughput at the end of the run.
The live reporter — also what CI logs get — prints one line per request:

Combine with a comma: --reporter live,junit --reporter-output junit=results.xml.
Exit codes
Section titled “Exit codes”| Code | Meaning |
|---|---|
0 | Every request passed AND at least one request ran |
1 | One or more requests failed or errored (or no requests matched after filtering), or the run was interrupted: the first SIGINT / SIGTERM (Ctrl-C) cancels cooperatively — in-flight requests are aborted, no further request or retry starts, and reporters still write the partial result (interrupted: true in JSON; the interactive dashboard only restores the terminal and prints the partial summary); a second signal force-quits with 130 |
2 | Internal error: missing collection, bad reporter name, IO failure, … |
Agent eval
Section titled “Agent eval”restura agent eval <suite-or-bundle.json> runs a versioned Agent Suite or portable Agent Bundle in CI and prints a compact JSON summary. It exits 0 only when every trial and every configured baseline gate passes; evaluated failures, errors, and cancellations exit 1; invalid input or unsupported configuration exits 2. SIGINT and SIGTERM cancel cooperatively: MCP sessions are closed and a partial --output report is still written; a second signal force-quits.
restura agent eval ./orders.agent-bundle.json --output agent-report.json| Flag | Default | Description |
|---|---|---|
--output <file> | Write the full JSON trace report. | |
--runtime <file> | Runtime manifest binding selected saved HTTP requests and MCP servers as tools. | |
--env <file> | JSON or YAML env file used by saved requests: a flat { "key": "value" } object (string values only) or a Postman environment export. Hoppscotch environment exports are not read. | |
--timeout <ms> | 30000 | Per saved-HTTP-request timeout. |
--allow-localhost | false | Permit localhost saved-request targets. |
--telemetry-config <file> | Metadata-only Langfuse or OTLP export; credentials must be environment references. The collector hostname is resolved once and every address validated: private, link-local, metadata, and loopback-via-hostname results exit 2 before the run. |
The CLI is intentionally narrower than the desktop AI Lab. It supports stateless OpenAI Responses, Anthropic Messages, Azure OpenAI (azure.openai), Gemini (google.generateContent) and Bedrock (aws.bedrock.converse) with environment credentials, deterministic fixtures, the QuickJS code sandbox (sandboxId: "quickjs"), and an optional runtime manifest for selected saved HTTP, MCP and A2A tools. Azure and Bedrock endpoints come from the manifest’s providerEndpoints, never from the suite. It refuses desktop secret handles, suite base URLs and agent judge graders. A runtime manifest can only expose explicitly listed read-only HTTP requests, read-only MCP tools with an allowlist, and A2A endpoints marked readOnly: true. --allow-localhost stays off by default.
Cost budgets (limits.maxCostUSD) and cost graders need a known price. OpenAI and Anthropic models in Restura’s shared price table are costed from reported usage; cached tokens with no known cache rate make the cost unknown rather than guessed. A suite using a budget or cost grader on an unpriced model exits 2 unless the runtime manifest prices it:
{ "schemaVersion": 1, "sources": [], "modelPrices": { "my-finetune": { "inputUSDPerMTok": 2, "outputUSDPerMTok": 8, "cacheReadUSDPerMTok": 0.5 } }}A policy profile (agents[].policyId) exits 2 unless it has ciEligible: true (default false) and autoApprove contains only read. Suites that reference no profile are unaffected. See the CLI README for the other refusals.
Dataset eval
Section titled “Dataset eval”restura eval run <file.eval.json> runs an AI Lab dataset eval exported with Evals → Export for CLI. Judge and pairwise scorers are allowed (ADR 0038). Keys come from the environment variables the file names per endpoint.
OPENAI_API_KEY=... ANTHROPIC_API_KEY=... restura eval run ./support.eval.json --junit eval.xml --max-cost 2| Flag | Default | Description |
|---|---|---|
--output <file> | Write the full JSON report: every cell, output and score. | |
--junit <file> | Write JUnit XML, one test case per case × model. | |
--max-cost <usd> | Stop once model + judge spend reaches this amount. Unpriced models are refused up front, and a call whose cost cannot be computed (no usage reported, or cached tokens with no cache rate) cancels the run with cost unknown. While a budget is set, model calls run one at a time whatever --concurrency says, so only the call that crosses the budget can be spent past it. | |
--split <split> | Run only the train or test cases. | |
--concurrency <n> | file | Override the file’s concurrency (1-32). |
--allow-localhost | false | Permit local runtime endpoints (Ollama, OpenAI-compatible). |
Exit codes: 0 when the share of passing scored cells meets the eval’s pass threshold (default 100%), 1 below it, on Ctrl-C, when --max-cost is reached, or when a call’s cost is unknown under --max-cost (a partial report is still written), and 2 for invalid input before any model call (a malformed file, a missing key, an unsafe endpoint, self-judging, no scorers, or --max-cost on an unpriced model). With judge scorers, set a pass threshold below 100%: judges are stochastic. The http-exec target is desktop-only.
MCP conformance
Section titled “MCP conformance”restura mcp conformance <url> checks an MCP server in the legacy and modern (2026-07-28) protocol eras with the same engine as the app’s Conformance action. --eras defaults to legacy,modern over streamable-http and to legacy over http-sse; asking for the modern era over http-sse exits 2. Exit codes: 0 when at least one requested era connected and no finding reaches --fail-on, 1 when a finding does or when no requested era connected (nothing was checked; this ignores --fail-on), 2 for invalid input or I/O failure. See the CLI README for every flag.
Example: GitHub Actions
Section titled “Example: GitHub Actions”name: API smoke testson: [push, pull_request]jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: { node-version: '24' } - run: npx restura-cli run ./api-tests --env env.staging.json --reporter live,junit --reporter-output junit=results.xml - uses: dorny/test-reporter@v1 if: always() with: name: API smoke tests path: results.xml reporter: java-junitTroubleshooting
Section titled “Troubleshooting”| Symptom | Likely cause / fix |
|---|---|
No recognised collection layout | Target needs opencollection.yml / .yaml or legacy _collection.yaml. Re-export from the Restura app. |
Invalid URL | URL after {{var}} resolution isn’t a valid absolute URL. Check --env is loaded and names match. |
Localhost URLs are not allowed | Add --allow-localhost for local upstreams. Off by default to prevent SSRF in shared CI. |
gRPC fails with “sends “auto” requests over the Connect protocol only” (UNIMPLEMENTED, UNKNOWN or UNAVAILABLE) | Upstream doesn’t speak the Connect protocol (404 / 415, or an HTTP/2-only native gRPC server). The CLI sends auto / Connect requests as Connect-over-HTTP and has no HTTP/2 gRPC transport; switch the request to gRPC-Web if the server supports it. |
auth uses a desktop-only secret handle… | Your auth references a secret handle only the desktop app can decrypt. The request errors (it is not sent unauthenticated); re-export with inline values for CI use. |
Development
Section titled “Development”# from cli/npm installnpm test # vitestnpm run type-check # tsc --noEmitnpm run build # tsup → dist/The CLI imports from the parent project at compile-time via path aliases (@/, @shared/); cli/tsconfig.json controls which parent modules participate in type-checking.
Related
Section titled “Related”- Scripts — the Postman
pm.*API. - Workflows — request chaining (also runs in CLI).
- OpenCollection — the collection format.
- AI Lab — desktop authoring and execution for agent suites.
- ADR 0005 — CLI runner.