> ## Documentation Index > Fetch the complete documentation index at: https://docs.2501.ai/llms.txt > Use this file to discover all available pages before exploring further. # Quickstart > Run your first benchmark scenario end to end Goal: run a single nginx-broken-config scenario against an agent, see the result, and read the report. \~5 minutes once the prerequisites are in place. ## Prerequisites | You need | How | | -------------------------------------------------------- | ------------------------------------------------- | | A running 2501 instance with at least one host + agent | See [Quickstart](/0.8/getting-started/quickstart) | | `2501` CLI installed and signed in (`2501 status` works) | See [CLI](/0.8/cli/overview) | | Ansible on PATH | `brew install ansible` / `apt install ansible` | | A sandbox host the agent can SSH into | A spare VM, container, or Lima/Incus instance | If you'd rather have the runner provision an ephemeral VM for you, jump to [VM Sandbox](/0.8/benchmark/sandbox) instead — the rest of this page assumes `--mode host` with a pre-provisioned target. ## Step 1 — get a scenarios directory The runner reads scenarios from a directory. The fastest path is to clone the 2501 scenarios examples repo (your account team can share the URL), or write your own. ```bash theme={null} mkdir -p ./scenarios/nginx/001-broken-config cd ./scenarios/nginx/001-broken-config ``` ## Step 2 — write `scenario.json` ```json scenario.json theme={null} { "key": "nginx/001-broken-config", "title": "Nginx fails to start due to broken config", "description": "Nginx config has a syntax error. Agent must find it, fix it, and bring the service back up on port 80.", "ticket_title": "Nginx not responding", "ticket_body": "@2501 the nginx service on sandbox-web-01 is down. It was working yesterday but stopped after a config change. Investigate, fix the config, ensure nginx is running and serving traffic on port 80.", "tags": ["nginx", "web"], "hosts": [ { "host_id": "hst_REPLACE_WITH_YOUR_HOST_ID" } ], "agents": [ { "agent_id": "agt_REPLACE_WITH_YOUR_AGENT_ID", "host_id": "hst_REPLACE_WITH_YOUR_HOST_ID" } ], "validation": { "job": [ { "label": "Job resolved successfully", "validator": "job_resolution_status", "pattern": "success" } ], "tasks": [ { "label": "Agent tested config before restarting", "validator": "pattern_match", "pattern": "nginx -t", "where": "executed_commands" }, { "label": "Agent restarted nginx", "validator": "pattern_match", "pattern": "systemctl.*(restart|reload|start).*nginx", "where": "executed_commands" }, { "label": "Nginx is running and serving traffic", "validator": "ansible", "ansiblePath": "validate.yml" } ] } } ``` Replace the two IDs with real ones from your tenant (find them in Command Center → Hosts and → Agents). ## Step 3 — write the playbooks The runner needs three Ansible playbooks: `prepare.yml` introduces the failure, `validate.yml` checks the fix worked, `restore.yml` resets the host. ```yaml prepare.yml theme={null} - name: Break nginx config hosts: web become: true tasks: - name: Ensure nginx is installed apt: { name: nginx, state: present, update_cache: true } - name: Write a config with a missing semicolon copy: dest: /etc/nginx/sites-available/default content: | server { listen 80 root /var/www/html; index index.html; } - name: Attempt reload (will fail, intentional) systemd: { name: nginx, state: restarted } ignore_errors: true ``` ```yaml validate.yml theme={null} - name: Verify nginx is healthy hosts: web become: true tasks: - name: Config syntax is valid command: nginx -t - name: Service is active command: systemctl is-active nginx - name: Port 80 responds uri: { url: http://localhost:80, status_code: [200, 301, 302] } ``` ```yaml restore.yml theme={null} - name: Reset nginx hosts: web become: true tasks: - name: Stop nginx systemd: { name: nginx, state: stopped, enabled: false } ignore_errors: true - name: Remove broken config file: { path: /etc/nginx/sites-available/default, state: absent } ignore_errors: true ``` Add an `inventory.ini` so Ansible knows how to reach the host: ```ini inventory.ini theme={null} [web] sandbox-web-01 ansible_host=10.0.1.10 ansible_user=ubuntu ansible_ssh_private_key_file=~/.ssh/sandbox.pem ``` ## Step 4 — run it From the repo root that contains `./scenarios/`: ```bash theme={null} 2501 runner start -s nginx/001-broken-config ``` The runner walks through Provision → Prepare → Execute → Validate → Restore, prints a summary table, and writes a ScenarioReport to the database. ## Step 5 — read the report Two places: * **Terminal output** — pass/fail per rule, plus the two aggregate scores. * **Command Center → Benchmarks** — the same run, charted over time alongside others. Pass an iteration count if you want more confidence than a single run gives you: ```bash theme={null} 2501 runner start -s nginx/001-broken-config -i 10 ``` 10 runs is usually enough to surface flakes and gauge consistency. For full confidence, run **100+ iterations** as part of a CI job. ## What to read next Every field of `scenario.json`, with examples. Every validator, every `where:` target, the scoring model. What prepare / validate / restore each do, and when they run. Skip the manual host setup — let the runner provision a fresh VM per run.