> ## Documentation Index > Fetch the complete documentation index at: https://docs.2501.ai/llms.txt > Use this file to discover all available pages before exploring further. # Why Benchmark > Measure how your agents handle real incidents on your own machines, before they meet production tickets **Benchmark** puts your agents in front of incidents you stage yourself - a full disk, a stopped service, a blocked port - on real machines you can afford to break, and grades what they did: did the agent **fix it**, and did it **fix it the right way**. Everything happens in **Command Center → Benchmarks** or through the [API](/0.17/benchmark/api). You write the scenario, test it on the machine, run it against the models you want, and read a graded result. Benchmarks ## Why benchmark Before letting a specialty handle a new kind of ticket, stage that failure on a test machine and watch how the agent handles it across many runs. A scenario that used to pass starts failing after a specialty, rule, or model change. The trend line shows it the same day. Run the same scenarios with a different main engine. The scores tell you which model behaves better, not just which is faster. Every run keeps a record: the commands the agent ran, the operational rules it was given, and which of your rules passed. ## The two concepts | Concept | What it is | | - | - | | **Scenario** | One reproducible incident: the commands that break the machine (**Target Setup**), the ticket the agent receives, the rules that grade the run (**Validation**), and the commands that put the machine back (**Target Restore**). | | **Benchmark** | One or more scenarios launched together with the same gateway and models. Each scenario in it produces a **scenario run** with its own result. | A scenario targets real hosts and agents from your organization. There are no special benchmark machines: any host with an agent can be a target, so pick hosts you can afford to break. ## How a scenario runs ``` Target Setup ─▶ Flush ─▶ Dispatch ─▶ Agent works ─▶ Collect ─▶ Evaluate ─▶ Target Restore │ │ │ │ │ │ │ break the clear the file the the agent's gather apply the put the machine agents' ticket job runs commands, validation machine memory through like any messages, rules back the gateway other job resolution ``` 1. **Target Setup** runs your setup commands in order. If one fails, no ticket is sent, the run ends in **Error**, and restore still runs. 2. **Flush** clears the evaluated agents' memory, so an earlier run cannot leak into this one. 3. **Dispatch** files the ticket. With the **Runner** gateway, the ticket is created inside 2501. With **ServiceNow**, a real incident is created on your instance and 2501 waits up to 10 minutes for the gateway to pick it up. 4. **The agent works** the ticket exactly as it would a real one. The run waits up to 45 minutes for the job to end. 5. **Collect** and **Evaluate** gather what happened and apply the [validation rules](/0.17/benchmark/validation), including commands on the host to check its final state. 6. **Target Restore** runs your restore commands. It runs on every path - success, failure, or stop - so the next run starts from a clean machine. If restore fails, the run says **Restore failed**: the machine may still be broken. Two things are frozen when you launch: the **scenario version** (editing a scenario never changes a run already queued) and the **models**. A run is always graded against the definition and engines it started with. ## How a run is graded Grading is deterministic: no model judges the run. A built-in rule, **Job completed**, is always checked first, then your rules. Each scenario run reports three results: | Result | What it means | | - | - | | **Pass / Fail** | The verdict. A run passes when every required rule passes, including Job completed. | | **Compliance** | The share of rules that passed - required, optional, and Job completed - from 0 to 100. It tells you how close a failing run got. | | **Remediation** | Whether the job resolved as `success`. It is reported on its own and does not change the verdict. | A run that never reached grading - its setup failed, or the job died for a technical reason - shows **Error**, not Fail. An Error means the scenario or the platform needs fixing; a Fail is a real result about the agent. See [Reading results](/0.17/benchmark/results). ## Prerequisites * **A Runner gateway.** Create one in **Gateways → New Gateway**, type **Runner**. It needs no configuration. To run through ServiceNow instead, see [Running through ServiceNow](/0.17/benchmark/command-center#running-through-servicenow). * **A remote-execution agent on every target host.** Setup, restore, and `host_state` checks reuse that agent's connection: its SSH or WinRM credentials, jump hosts, shell, environment, and user. Linux and Windows hosts are supported; network devices are not. * **Hosts you can afford to break.** Setup commands change the machine for real, usually as root. * **A working model configuration.** The models are checked and frozen at launch. ## Who can do what | Role | Can | | - | - | | **Admin** | Create, edit, publish, disable, and delete scenarios, plus everything a User can do. | | **User** | Run scenarios and benchmarks (including Target Setup and Target Restore on their own), stop them, and read results. | | **Auditor** | Read scenarios and results. | ## What to read next Build, test, and run your first scenario in Command Center. The rules that make a scenario reliable, and how to grade it fairly. Every field, step type, and shell rule. Stop drift from quietly corrupting your results. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.