Fields
Hosts and agents are referenced by name and resolved when you save: an unknown name is rejected. A host or agent used by a scenario cannot be deleted until no scenario targets it.
Agents
Only the agents the scenario lists take part in the run. If the gateway routes the ticket to another agent - for example a second agent on the same host - its task fails with “not part of this benchmark run”. List every agent that could plausibly receive the work. A scenario with no agents and no hosts grades the gateway alone: whether it takes the ticket, routes it, or refuses it. See the gateway-only example.Steps
prepare and restore hold steps of two kinds. Every step can have a name, shown in run records and errors, and an on, the host it runs on (one of hosts, default the first).
Run a command
Write a file
sudo writes as root (Linux only). Use it for anything longer than a one-liner instead of an echo with fragile quoting.
Order and failure
Steps run one at a time, in order. The first failing step stops the phase. For setup, that means no ticket is sent and the run ends in Error. For restore, the remaining restore steps are skipped and the run says Restore failed.How commands run
A command runs on the host through its agent’s connection - the same SSH or WinRM credentials, jump hosts, shell, environment, and user the agent uses. Nothing runs as root unless you ask for it. Every step is a fresh shell. No variable, working directory, or environment carries over from one step to the next. Recompute what you need inside each step:Linux
- Commands start with
set -u: an unset variable is an error, not an empty string. Write${VAR:-}when empty is acceptable. set -eis not set. In a multi-command step only the last exit code counts, so chain with&&when every part must succeed.- Write
sudoandtimeoutinto the command itself.
Windows
- Commands run in PowerShell with
$ErrorActionPreference = 'Stop'andSet-StrictMode -Version Latest. - The step fails when the last statement failed, even silently.
Remove-Item C:\x -ErrorAction SilentlyContinueon a missing file exits non-zero. Guard it (if (Test-Path C:\x) { Remove-Item C:\x }) or end idempotent cleanup with; exit 0. - Escape backslashes in JSON:
"C:\\bench\\run.ps1". If you generate scenarios with a script, check it did not turn\b,\n,\r, or\tin a path into control characters. - Keep commands short. For a long script, use a
writestep to drop a.ps1file, then a short step that runs it.
Secrets
Write${secret:name} to use a credential from your organization, by name or id. Its value is exported as an environment variable for that command, never inlined, and scrubbed from the recorded output. Never paste a secret value into a scenario.
Multi-host scenarios
List one host and one agent per machine. Seton on every step and host_state check that does not target the first host.

