Skip to content

CLI — restura-cli

restura-cli runs your collections headlessly with the same protocol implementations as the desktop and web apps. Drop it into CI to assert your API, get JUnit XML for your dashboard, ship an HTML report to your team.

Live dashboard — restura run

flowchart LR
  CI[CI runner] -->|exec| CLI[restura run]
  CLI -->|load| COL[(OpenCollection<br/>dir / .yaml)]
  CLI -->|exec| H[HTTP / GraphQL]
  CLI -->|exec| G[gRPC Connect / gRPC-Web]
  CLI -->|exec| S[SSE]
  CLI -->|exec| M[MCP]
  H --> R1[tui · live · json · junit · html · stats]
  G --> R1
  S --> R1
  M --> R1
  R1 -->|exit code| CI

  classDef cli fill:#7b6ef6,stroke:#4f46e5,color:#fff;
  classDef proto fill:#6366f122,stroke:#7b6ef6,color:#7b6ef6;
  class CLI cli;
  class H,G,S,M proto;
Terminal window
npm install -g restura-cli
# or, no install:
npx restura-cli run ./my-collection

Requires Node.js 24+.

Save the collection to disk from the Restura desktop app (collection menu → Save to Files, which writes the OpenCollection directory layout), or export a single OpenCollection .yml from the collection menu → Export → OpenCollection (YAML), then:

Terminal window
restura run ./my-collection --reporter junit --reporter-output junit=results.xml

Exit code is 0 when every request passed, 1 if any failed, 2 on internal errors (missing collection, bad flags).

Run restura (or restura run) with no collection in a terminal to launch a guided wizard — pick the collection, reporters, output paths, an optional env file, and whether to allow localhost — then it runs.

Interactive wizard

Terminal window
restura # bare → wizard
restura run # also → wizard when no collection is given

By default an interactive run shows the live tui dashboard; piped output and CI fall back to plain line output (live). See Reporters.

The loader auto-detects three layouts:

LayoutDetected when…
OpenCollection directory (preferred)the target directory contains opencollection.yml (or .yaml)
OpenCollection bundled filethe target path ends in .yaml / .yml
Legacy file-collection (deprecated)the target directory contains _collection.yaml

The legacy format prints a stderr deprecation warning the first time it’s loaded.

ProtocolNotes
HTTP / RESTFull support
GraphQLRuns as HTTP with body type graphql. {{vars}} are substituted inside JSON string values of the saved variables, so a value holding " or \ stays valid JSON; variables that are not valid JSON (or not an object) fail the item instead of sending {}. @defer / @stream multipart responses are merged incrementally, so pm.response.json() sees the merged result
gRPCUnary only. Honours the request’s wire protocol: auto / Connect uses the Connect protocol (JSON, no proto compilation needed); gRPC-Web (binary or text) encodes protobuf with the request’s saved schema (reflection descriptors or uploaded .proto) and fails with FAILED_PRECONDITION if there is none; native gRPC (HTTP/2) fails with UNIMPLEMENTED
SSECaptures events for --sse-duration ms, or until --sse-events N. An event without an event: field is a message event, as in the app. Sign-at-wire and challenge auth (AWS SigV4, OAuth 1.0a, WSSE, Digest, NTLM) is refused, not sent anonymously
MCPRuns the saved operation (defaultMethod + defaultParams, e.g. tools/call with {"name":"…","arguments":{…}}, {{var}}s resolved) over one initialized session per server per run, over streamable-http or http-sse. Test scripts get the JSON-RPC reply: rs.response.json().result (incl. isError) or rs.response.json().error. Without assertions, a tool error or JSON-RPC error fails the item; with assertions, the assertions decide. An MCP item with no method fails with an actionable error instead of sending a ping. See MCP.
Kafka / MQTT presetsSkipped with a notice on stderr — connection presets open in the desktop app; the CLI has no broker transport
WebSocket / Socket.IO collection itemsSkipped with a [restura] skipping WebSocket preset "…" (or Socket.IO preset) notice on stderr, like Kafka / MQTT — they open in the app

The CLI and the in-app collection runner don’t run the same protocols:

ProtocolCLI (restura run)App collection runner
HTTP / GraphQL / gRPC unaryRunsRuns
gRPC streamingUnary onlyServer streams run; client and bidi streams on desktop only
SSERuns (captures a window of events)Skipped
MCPRunsRuns
WebSocket / Socket.IO / Kafka / MQTTSkippedSkipped
Terminal window
restura run [collection] [options]

[collection] accepts a directory (any supported layout) or a bundled .yaml / .yml file. Omit it in a terminal to pick one via the interactive wizard; in CI it is required.

FlagDefaultDescription
-e, --env <file>JSON or YAML env file: a flat { "key": "value" } object (string values only) or a Postman environment export (*.postman_environment.json; disabled rows skipped). ${VAR} placeholders are expanded from process.env; an unset one expands to an empty string with a [restura] WARNING: ${VAR} is not set… line on stderr (once per name; the exit code is unchanged). A secret or private variable left empty this way counts as missing: a request that uses it is refused, not sent. Hoppscotch environment exports are not read; import those in the app.
-g, --globals <file>Postman globals export (or a flat key/value file). Lowest precedence: below --env.
--env-var <kv...>Set variables, Newman style: --env-var token=abc base=https://x. Wins over --env and collection / folder variables.
-n, --iteration-count <n>Number of iterations. Without --data, runs the collection n times; with --data, the last row repeats past the end.
--delay-request <ms>Pause between requests.
--reporter <list>autoComma-separated: tui, live, json, junit, html, stats. Defaults to tui in a terminal, live otherwise (piped / CI).
--output <file>Shorthand for a single file reporter. Two file reporters (json, junit, html) resolving to the same path is an error (exit 2); use --reporter-output to give each its own.
--reporter-output <kv...>Per-reporter output: --reporter-output junit=results.xml html=report.html.
--bailfalseStop on first failure.
--timeout <ms>30000Per-request timeout.
--allow-localhostfalsePermit requests to localhost / 127.0.0.1. Off by default (SSRF guard). The guard also checks every address a host name resolves to when the CLI dials it directly (no proxy): a name resolving to cloud metadata (169.254.169.254, e.g. 169.254.169.254.nip.io) or a private range is refused, and one resolving to loopback only with this flag. Through --proxy / HTTP(S)_PROXY the proxy resolves the target.
--folder <path>Only run requests under this folder path (slash-joined).
--include <pattern...>Substring or glob (e.g. users/*). Repeatable.
--exclude <pattern...>Same syntax as --include. Applied after.
--data <file>CSV (with header row; the delimiter — , ; tab or | — is detected from the header line and values keep their whitespace, as in the app) or JSON array (.json is JSON, .csv is CSV, other extensions are sniffed: a leading [ means JSON). Runs the collection once per row. A file with no rows or a ragged CSV row is an error (exit 2).
--max-iterations <n>Cap iterations when a --data file is large.
--retry <n>0Retry attempts per failing request. Backoff doubles from 250 ms; a 429 / 503 with Retry-After (seconds or HTTP-date) waits that long instead, capped at 60 s.
--retry-on <list>network,5xxComma-separated triggers: network, 5xx, 4xx, or specific status codes (429,503). network means the upstream never answered: the connection was refused or reset, DNS failed, or the call timed out before any response (HTTP / GraphQL, gRPC Connect and gRPC-Web, SSE, MCP, and the OAuth2 token request). Any answer is not a network failure, whatever its status: an HTTP 503, an SSE 404 or a gRPC status the server sent. 5xx means an HTTP 5xx, or for gRPC only the transient statuses UNAVAILABLE, DEADLINE_EXCEEDED and RESOURCE_EXHAUSTED; other gRPC statuses (INVALID_ARGUMENT, NOT_FOUND, PERMISSION_DENIED, INTERNAL, …) are attempted once. A refusal is never retried, whatever the triggers: an unresolvable variable or secret (a desktop secret handle, an external secret reference with no resolver), a blocked or unsupported request, NTLM, a URL or redirect the SSRF guard rejects (reported as 400), a wire-signing failure (500 Auth signing failed), and the CLI’s own response-size cap (HTTP 413, gRPC RESOURCE_EXHAUSTED with Response too large).
--sse-duration <ms>5000How long to keep SSE streams open.
--sse-events <n>Stop SSE early after N events.
--insecurefalseSkip TLS certificate verification (self-signed / staging).
--ca <file>PEM CA bundle to trust (private CA) without disabling verification.
--client-cert <file>PEM client certificate for mutual TLS (mTLS).
--client-key <file>PEM client private key for mutual TLS.
--cert-passphrase <value>Passphrase for an encrypted --client-key.
--proxy <url>HTTP(S) proxy URL. Overrides HTTP_PROXY / HTTPS_PROXY; composes with the TLS flags.
--external-secrets-config <file>YAML/JSON profiles (AWS Secrets Manager, Google Secret Manager, Azure Key Vault) for resolving external secret references. Identity metadata only — never credentials. Without it, a request that uses an external reference errors instead of being sent unauthenticated.

Per-request TLS and proxy. An HTTP/GraphQL request’s own TLS and proxy settings (the desktop app’s request Settings: verifySsl: false, caCert, clientCert PEM or PFX, minTlsVersion, cipherSuites, serverCipherOrder, and an HTTP(S) proxy with its bypassList and Basic proxy auth) are applied to that request on a dedicated connection, layered over the run’s TLS flags (verifySsl: true is the default and does not undo --insecure); a request-level proxy replaces --proxy for that request. Settings the CLI cannot apply refuse the request instead of sending it differently: a SOCKS proxy, or a client-cert passphrase / proxy password held in the desktop keychain or an external secret store. Without --proxy, a request with its own TLS options connects directly rather than through the HTTP_PROXY / HTTPS_PROXY env vars, and restura run prints one warning for the run when that applies. The CLI agent tools (DNS-pinned transport) refuse requests that carry such settings. Other protocols (gRPC, SSE, WebSocket, MCP) use the run’s flags only.

Terminal window
restura workflow run <workspace> <workflow-id> [--env ci.env] [--timeout 30000] [--allow-localhost] [--allow-mutations]

--allow-localhost permits loopback targets and --allow-mutations permits saved GraphQL mutation calls; both are off by default. Ctrl-C / SIGTERM stops the workflow cooperatively (status stopped, exit 1).

Variables follow the app’s workflow runner: the workflow starts from the --env values with the collection’s enabled variables over them (collection wins, as in restura run), so if: conditions and set tasks see collection values. Each saved-request call then layers the request’s folder variables (outer → inner) over that, and values earlier workflow steps produced or changed win over both.

The workspace must be an OpenCollection directory with opencollection.yml and a complete workflows/<workflow-id>/ artifact set. The command accepts only Restura’s safe OWS profile: do, set, wait, timeout/cancellation, and binding-only HTTP calls to saved requests. It rejects inline endpoints, credentials, non-HTTP calls, and controls without a bounded runtime before dispatching any network request.

Pre-request and test scripts run in a sandboxed QuickJS WASM VM — no DOM, no filesystem, no network escape; 64 MB memory cap, 5 s execution timeout. pm.sendRequest, pm.cookies, and pm.vault are not wired in the CLI and reject with a clear error. See Scripts for the full Postman API surface.

request.http.yaml
name: Get user
method: GET
url: "{{API_BASE}}/users/1"
testScript: |
pm.test("status is 200", () => pm.response.to.have.status(200));
pm.test("response has name", () => {
pm.expect(pm.response.json()).to.have.property("name");
});

When a test script runs and defines any pm.test(...) assertion, those drive pass/fail. Otherwise pass/fail falls back to the transport outcome (HTTP 2xx, gRPC OK, etc.).

Variables set inside a script (pm.environment.set('K', 'v')) propagate to subsequent requests in the same run, including later iterations. A pre-request pm.execution.skipRequest() sends nothing and skips the test script; the request counts as passed and as skipped in the summary.

Collection, folder and request scripts of one phase run in order, each in its own block scope, so the same const / let at two levels is not a redeclaration error.

Layered sources, in order of precedence (later wins):

  1. --globals file, then the --env file
  2. Collection variables (declared in opencollection.yml or _collection.yaml, or the Postman collection’s variables). A pm.collectionVariables.set in a script updates this layer for later requests in the run, as in Newman and the app (the file on disk is not changed). A secret collection variable has no value in the file: supply it with --env, --env-var or a --data row, or set it from an earlier script. A request that is about to be sent while it still has no value (an empty value counts as none) is reported as not sent and fails; the rest of the run carries on.
  3. Folder variables of the folders above the request, outer → inner (inner wins). Secret, private and handle-backed folder variables have no value in the file: like a secret collection variable, a request that is about to be sent while one it uses still has no value is reported as not sent and fails.
  4. --env-var values
  5. Values a script set in an earlier iteration
  6. Iteration row (when --data is set)
  7. Values a script set earlier in this iteration

Substitutions use {{NAME}}. A name is any text without braces, trimmed of surrounding whitespace, so {{my key}} and {{ a:b }} both work. Unknown vars are left in place so the upstream sees them and you notice the gap. A substituted value is not expanded again, except that a value of {{$randomFirstName}} still generates.

Header-based auth (Bearer, Basic, API-key, OAuth2) follows the app’s rule: Basic with an empty username and a password sends Basic base64(":password"), and with neither sends no header. HTTP, MCP, SSE and gRPC items resolve external secret references through the profile given with --external-secrets-config; with no resolver for a reference (or a desktop secret handle) the item is refused, never retried, instead of being sent without credentials. For OAuth2, a pre-supplied access token is used as-is; if the auth instead carries a token endpoint + client id (and no token), the CLI fetches one via the client_credentials grant, caching it for the run only. The token request goes through the same transport as the run: --ca, --insecure, --client-cert/--client-key, --proxy and --timeout apply to it, and a token endpoint that redirects is refused.

Postman-compatible {{$random*}} / {{$timestamp}} helpers are expanded after user var substitution:

HelperExample
{{$randomUUID}}f4d2e3...
{{$timestamp}}1700000000
{{$isoTimestamp}}2026-05-22T13:42:00Z
{{$randomEmail}}alice.42@example.com
{{$randomFirstName}}Olivia
{{$randomIP}}192.0.2.4

Full list in shared/variables/dynamic.ts.

Terminal window
restura run ./users-api --data ./users.csv --reporter junit --reporter-output junit=junit.xml
users.csv
username,role
alice,admin
bob,viewer
charlie,editor

Each row exposes username and role as variables, overriding any same-named env or collection variable for that iteration only. Reporter output groups results by iteration index.

The default is chosen automatically: the live tui dashboard when stdout is a terminal, live lines otherwise (piped output / CI). NO_COLOR only turns colour off (the dashboard and the wizard still run in a terminal); FORCE_COLOR colours piped output without making it interactive. Names and error messages from collections and servers are printed with terminal control sequences stripped.

Live, in-place dashboard with a progress bar and a spinner on the in-flight request. The default in an interactive terminal; on completion it collapses to a clean failure list + summary.

The live reporter — also what CI logs get — prints one line per request:

Line reporter output

Combine with a comma: --reporter live,junit --reporter-output junit=results.xml.

CodeMeaning
0Every request passed AND at least one request ran
1One or more requests failed or errored (or no requests matched after filtering), or the run was interrupted: the first SIGINT / SIGTERM (Ctrl-C) cancels cooperatively — in-flight requests are aborted, no further request or retry starts, and reporters still write the partial result (interrupted: true in JSON; the interactive dashboard only restores the terminal and prints the partial summary); a second signal force-quits with 130
2Internal error: missing collection, bad reporter name, IO failure, …

restura agent eval <suite-or-bundle.json> runs a versioned Agent Suite or portable Agent Bundle in CI and prints a compact JSON summary. It exits 0 only when every trial and every configured baseline gate passes; evaluated failures, errors, and cancellations exit 1; invalid input or unsupported configuration exits 2. SIGINT and SIGTERM cancel cooperatively: MCP sessions are closed and a partial --output report is still written; a second signal force-quits.

Terminal window
restura agent eval ./orders.agent-bundle.json --output agent-report.json
FlagDefaultDescription
--output <file>Write the full JSON trace report.
--runtime <file>Runtime manifest binding selected saved HTTP requests and MCP servers as tools.
--env <file>JSON or YAML env file used by saved requests: a flat { "key": "value" } object (string values only) or a Postman environment export. Hoppscotch environment exports are not read.
--timeout <ms>30000Per saved-HTTP-request timeout.
--allow-localhostfalsePermit localhost saved-request targets.
--telemetry-config <file>Metadata-only Langfuse or OTLP export; credentials must be environment references. The collector hostname is resolved once and every address validated: private, link-local, metadata, and loopback-via-hostname results exit 2 before the run.

The CLI is intentionally narrower than the desktop AI Lab. It supports stateless OpenAI Responses, Anthropic Messages, Azure OpenAI (azure.openai), Gemini (google.generateContent) and Bedrock (aws.bedrock.converse) with environment credentials, deterministic fixtures, the QuickJS code sandbox (sandboxId: "quickjs"), and an optional runtime manifest for selected saved HTTP, MCP and A2A tools. Azure and Bedrock endpoints come from the manifest’s providerEndpoints, never from the suite. It refuses desktop secret handles, suite base URLs and agent judge graders. A runtime manifest can only expose explicitly listed read-only HTTP requests, read-only MCP tools with an allowlist, and A2A endpoints marked readOnly: true. --allow-localhost stays off by default.

Cost budgets (limits.maxCostUSD) and cost graders need a known price. OpenAI and Anthropic models in Restura’s shared price table are costed from reported usage; cached tokens with no known cache rate make the cost unknown rather than guessed. A suite using a budget or cost grader on an unpriced model exits 2 unless the runtime manifest prices it:

{
"schemaVersion": 1,
"sources": [],
"modelPrices": {
"my-finetune": { "inputUSDPerMTok": 2, "outputUSDPerMTok": 8, "cacheReadUSDPerMTok": 0.5 }
}
}

A policy profile (agents[].policyId) exits 2 unless it has ciEligible: true (default false) and autoApprove contains only read. Suites that reference no profile are unaffected. See the CLI README for the other refusals.

restura eval run <file.eval.json> runs an AI Lab dataset eval exported with Evals → Export for CLI. Judge and pairwise scorers are allowed (ADR 0038). Keys come from the environment variables the file names per endpoint.

Terminal window
OPENAI_API_KEY=... ANTHROPIC_API_KEY=... restura eval run ./support.eval.json --junit eval.xml --max-cost 2
FlagDefaultDescription
--output <file>Write the full JSON report: every cell, output and score.
--junit <file>Write JUnit XML, one test case per case × model.
--max-cost <usd>Stop once model + judge spend reaches this amount. Unpriced models are refused up front, and a call whose cost cannot be computed (no usage reported, or cached tokens with no cache rate) cancels the run with cost unknown. While a budget is set, model calls run one at a time whatever --concurrency says, so only the call that crosses the budget can be spent past it.
--split <split>Run only the train or test cases.
--concurrency <n>fileOverride the file’s concurrency (1-32).
--allow-localhostfalsePermit local runtime endpoints (Ollama, OpenAI-compatible).

Exit codes: 0 when the share of passing scored cells meets the eval’s pass threshold (default 100%), 1 below it, on Ctrl-C, when --max-cost is reached, or when a call’s cost is unknown under --max-cost (a partial report is still written), and 2 for invalid input before any model call (a malformed file, a missing key, an unsafe endpoint, self-judging, no scorers, or --max-cost on an unpriced model). With judge scorers, set a pass threshold below 100%: judges are stochastic. The http-exec target is desktop-only.

restura mcp conformance <url> checks an MCP server in the legacy and modern (2026-07-28) protocol eras with the same engine as the app’s Conformance action. --eras defaults to legacy,modern over streamable-http and to legacy over http-sse; asking for the modern era over http-sse exits 2. Exit codes: 0 when at least one requested era connected and no finding reaches --fail-on, 1 when a finding does or when no requested era connected (nothing was checked; this ignores --fail-on), 2 for invalid input or I/O failure. See the CLI README for every flag.

name: API smoke tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: '24' }
- run: npx restura-cli run ./api-tests --env env.staging.json --reporter live,junit --reporter-output junit=results.xml
- uses: dorny/test-reporter@v1
if: always()
with:
name: API smoke tests
path: results.xml
reporter: java-junit
SymptomLikely cause / fix
No recognised collection layoutTarget needs opencollection.yml / .yaml or legacy _collection.yaml. Re-export from the Restura app.
Invalid URLURL after {{var}} resolution isn’t a valid absolute URL. Check --env is loaded and names match.
Localhost URLs are not allowedAdd --allow-localhost for local upstreams. Off by default to prevent SSRF in shared CI.
gRPC fails with “sends “auto” requests over the Connect protocol only” (UNIMPLEMENTED, UNKNOWN or UNAVAILABLE)Upstream doesn’t speak the Connect protocol (404 / 415, or an HTTP/2-only native gRPC server). The CLI sends auto / Connect requests as Connect-over-HTTP and has no HTTP/2 gRPC transport; switch the request to gRPC-Web if the server supports it.
auth uses a desktop-only secret handle…Your auth references a secret handle only the desktop app can decrypt. The request errors (it is not sent unauthenticated); re-export with inline values for CI use.
Terminal window
# from cli/
npm install
npm test # vitest
npm run type-check # tsc --noEmit
npm run build # tsup → dist/

The CLI imports from the parent project at compile-time via path aliases (@/, @shared/); cli/tsconfig.json controls which parent modules participate in type-checking.