Skip to main content
Goal: run a single nginx-broken-config scenario against an agent, see the result, and read the report. ~5 minutes once the prerequisites are in place.

​
Prerequisites

You needHow
A running 2501 instance with at least one host + agentSee Quickstart
2501 CLI installed and signed in (2501 status works)See CLI
Ansible on PATHbrew install ansible / apt install ansible
A sandbox host the agent can SSH intoA spare VM, container, or Lima/Incus instance
If you’d rather have the runner provision an ephemeral VM for you, jump to VM Sandbox instead — the rest of this page assumes --mode host with a pre-provisioned target.

​
Step 1 — get a scenarios directory

The runner reads scenarios from a directory. The fastest path is to clone the 2501 scenarios examples repo (your account team can share the URL), or write your own. 
mkdir -p ./scenarios/nginx/001-broken-config
cd ./scenarios/nginx/001-broken-config

​
Step 2 — write scenario.json

scenario.json
{
  "key": "nginx/001-broken-config",
  "title": "Nginx fails to start due to broken config",
  "description": "Nginx config has a syntax error. Agent must find it, fix it, and bring the service back up on port 80.",
  "ticket_title": "Nginx not responding",
  "ticket_body": "@2501 the nginx service on sandbox-web-01 is down. It was working yesterday but stopped after a config change. Investigate, fix the config, ensure nginx is running and serving traffic on port 80.",
  "tags": ["nginx", "web"],
  "hosts": [
    { "host_id": "hst_REPLACE_WITH_YOUR_HOST_ID" }
  ],
  "agents": [
    { "agent_id": "agt_REPLACE_WITH_YOUR_AGENT_ID", "host_id": "hst_REPLACE_WITH_YOUR_HOST_ID" }
  ],
  "validation": {
    "job": [
      { "label": "Job resolved successfully", "validator": "job_resolution_status", "pattern": "success" }
    ],
    "tasks": [
      { "label": "Agent tested config before restarting", "validator": "pattern_match", "pattern": "nginx -t", "where": "executed_commands" },
      { "label": "Agent restarted nginx",                "validator": "pattern_match", "pattern": "systemctl.*(restart|reload|start).*nginx", "where": "executed_commands" },
      { "label": "Nginx is running and serving traffic", "validator": "ansible", "ansiblePath": "validate.yml" }
    ]
  }
}
Replace the two IDs with real ones from your tenant (find them in Command Center → Hosts and → Agents).

​
Step 3 — write the playbooks

The runner needs three Ansible playbooks: prepare.yml introduces the failure, validate.yml checks the fix worked, restore.yml resets the host.
prepare.yml
- name: Break nginx config
  hosts: web
  become: true
  tasks:
    - name: Ensure nginx is installed
      apt: { name: nginx, state: present, update_cache: true }
    - name: Write a config with a missing semicolon
      copy:
        dest: /etc/nginx/sites-available/default
        content: |
          server {
            listen 80
            root /var/www/html;
            index index.html;
          }
    - name: Attempt reload (will fail, intentional)
      systemd: { name: nginx, state: restarted }
      ignore_errors: true
validate.yml
- name: Verify nginx is healthy
  hosts: web
  become: true
  tasks:
    - name: Config syntax is valid
      command: nginx -t
    - name: Service is active
      command: systemctl is-active nginx
    - name: Port 80 responds
      uri: { url: http://localhost:80, status_code: [200, 301, 302] }
restore.yml
- name: Reset nginx
  hosts: web
  become: true
  tasks:
    - name: Stop nginx
      systemd: { name: nginx, state: stopped, enabled: false }
      ignore_errors: true
    - name: Remove broken config
      file: { path: /etc/nginx/sites-available/default, state: absent }
      ignore_errors: true
Add an inventory.ini so Ansible knows how to reach the host:
inventory.ini
[web]
sandbox-web-01 ansible_host=10.0.1.10 ansible_user=ubuntu ansible_ssh_private_key_file=~/.ssh/sandbox.pem

​
Step 4 — run it

From the repo root that contains ./scenarios/: 
2501 runner start -s nginx/001-broken-config
The runner walks through Provision → Prepare → Execute → Validate → Restore, prints a summary table, and writes a ScenarioReport to the database.

​
Step 5 — read the report

Two places:
  • Terminal output — pass/fail per rule, plus the two aggregate scores.
  • Command Center → Benchmarks — the same run, charted over time alongside others.
Pass an iteration count if you want more confidence than a single run gives you: 
2501 runner start -s nginx/001-broken-config -i 10
10 runs is usually enough to surface flakes and gauge consistency. For full confidence, run 100+ iterations as part of a CI job.

Scenario structure

Every field of scenario.json, with examples.

Validation rules

Every validator, every where: target, the scoring model.

Playbooks

What prepare / validate / restore each do, and when they run.

VM Sandbox

Skip the manual host setup — let the runner provision a fresh VM per run.