> ## Documentation Index
> Fetch the complete documentation index at: https://docs.2501.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Benchmark in Command Center
> The Benchmarks pages, the scenario form, launching runs, and stopping or deleting them
Everything about benchmarks lives in **Command Center → Benchmarks**. This page is the reference for each screen. For a guided first run, see the [Quickstart](/0.17/benchmark/quickstart).
## The pages
| Page | What it shows |
| - | - |
| **Benchmarks** | Average pass rate, compliance, and remediation over a time range, with the change from the previous period; a trend chart; every scenario with its pass rate, trend, and history; and the recent benchmarks. Filter by time range and engine. **New Scenario** and **Run Benchmark** start here. |
| **A scenario** | Its run history - verdict, compliance, remediation, tokens, engines, time to resolve - with **Edit scenario**, **Target Setup**, **Target Restore**, and **Run now**. |
| **A benchmark** | Each scenario in it with its live stage, tokens, duration, compliance, and remediation. **Stop** and **Delete benchmark** are here. |
| **A scenario run** | One run's verdict, compliance, remediation, engines, timings, and every validation check with what it inspected. |
## The scenario form
**New Scenario** and **Edit scenario** open the same form, with six sections:
| Section | What you fill in |
| - | - |
| **Scenario Definition** | Name, description, and the ticket the agent receives: Ticket Title and Ticket Description. |
| **Evaluated Agents** | The agents to benchmark. Each agent's host becomes a target: setup, restore, and `host_state` blocks appear per host. Removing an agent removes its host's blocks. Leave it empty for a scenario that only grades gateway behaviour. |
| **Variables** | Name/value pairs shared by every phase. Write `${name}` in any command. |
| **Target Setup** | The commands that break the machine, per host. |
| **Validation** | The rules that grade the run - see [Validation rules](/0.17/benchmark/validation). |
| **Target Restore** | The commands that put the machine back, per host. |
Target Setup, Validation, and Target Restore each have a **Form / JSON** toggle. The JSON is the [scenario format](/0.17/benchmark/scenario): switch to it to paste an example or edit many steps at once.
While editing, each setup and restore block has a **Run it** button that saves the block and runs it on its host, so you can build a phase step by step. The status selector sets `draft`, `published`, or `disabled`, and **History** lists every saved version.
Only an **Admin** can create, edit, or delete scenarios.
## Testing setup and restore
On a scenario's page, **Target Setup** and **Target Restore** run just that phase, with each step's status and output. Nothing is sent to an agent and nothing is graded, so these runs never appear in the results.
These buttons run the steps of the scenario's **first host**. For a multi-host scenario, run each host's block with **Run it** in the form, or run the whole phase on every host through the [API](/0.17/benchmark/api#2-test-setup-and-restore).
Use them to:
* check new commands before a graded run,
* leave a host broken on purpose to look at it (**Target Setup** alone),
* clean up a host an interrupted run left dirty (**Target Restore** alone).
These phase runs go one at a time per organization, and wait for any benchmark using the same host.
## Running a benchmark
* **Run now** on a scenario's page runs that scenario as a one-scenario benchmark.
* **Run Benchmark** on the Benchmarks page picks several scenarios - draft and published - for one benchmark.
Both dialogs let you choose:
| Option | Effect |
| - | - |
| **Gateway** | **Runner** (default) creates the ticket inside 2501. **ServiceNow** creates a real incident - see [below](#running-through-servicenow). |
| **Main engine**, **Secondary engine** | Override the evaluated agents' models for this benchmark. The agents themselves are not changed. |
| **Gateway engine** | Override the model the gateway routes the ticket with. |
| **Run sequentially** | Run the scenarios one at a time, in order. |
Leave an engine on **Leave unchanged** to use the current configuration. Whatever you choose is frozen at launch, along with each scenario's current version.
Without **Run sequentially**, each scenario holds every host it targets until it has restored them: scenarios sharing a host run one at a time, scenarios on different hosts run together.
A benchmark holds up to 50 scenarios.
### Running through ServiceNow
With the ServiceNow gateway, each run creates a real incident on your instance and waits up to 10 minutes for your ServiceNow gateway to pick it up. This tests the whole path a customer ticket takes, including ingestion.
The organization's ServiceNow gateway needs three extra metadata keys:
| Key | Purpose |
| - | - |
| `servicenow_assigned_group_id` | The assignment group the incident is created in. |
| `servicenow_caller_id` | The caller set on the incident. |
| `environment` | The environment tag that routes the incident to this organization. |
Point benchmarks at an assignment group that real incidents do not use.
## Stopping a run
**Stop** on a benchmark cancels every scenario in it; **Stop** on a scenario run cancels that run. A stopped run still runs **Target Restore**, so the host is put back.
## When restore fails
If **Target Restore** fails, the run says **Restore failed** and the benchmark page flags the scenario **host left dirty**. Later runs on that host would score against a broken machine. Fix the restore, then run **Target Restore** from the scenario's page to clean the host - see [Keeping scenarios healthy](/0.17/benchmark/maintaining).
## If the engine restarts
A run caught mid-way by an engine restart is not resumed. It ends as **Error** (or **Cancelled** if you had already clicked Stop), and a restore-only run is queued for it so its host is put back. The other scenarios in the benchmark keep running.
## Deleting
* **Delete benchmark** on a benchmark's page permanently deletes it with its runs and the jobs, tasks, and tickets they created. It cannot be undone.
* **Delete** on a scenario removes the scenario. Its past runs stay in the results.
* To retire a scenario without losing it, set it to `disabled` instead: it stays visible and editable, but cannot be launched.
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.