Skip to main content
2501-runner run -s <scenarios> [options]

​
Execution Flow

Every 2501-runner run call goes through these phases: Pre-flight — validates the full environment: DB connectivity, org/tenant/user IDs, gateway or engine API reachability, Ansible availability. Scenario Discovery — scans the scenarios directory, loads all scenario.json files, resolves the -s argument against available keys and tags. Per-Scenario Loop — for each matched scenario (and each iteration when -i > 1):
  • Provision: resolve host and agent from the DB, flush agent memory
  • Prepare: run restore.yml silently, then run prepare.yml (abort if it fails)
  • Execute: dispatch the scenario through the selected entry point, poll for completion
  • Validate: evaluate all validation rules, run validate.yml if declared
  • Restore: run restore.yml to reset the host (non-fatal if fails)
Report — prints a summary table, persists a ScenarioReport to the DB, exits 0 or 1.

​
Options

OptionDefaultDescription
-s, --scenarios <list>requiredScenario keys or tags, comma-separated. Mix freely.
--from <type>gatewayEntry point: gateway | ticket
-m, --mode <mode>hostExecution mode: host | incus | lima
-g, --gateway <type>Gateway type. Required with --from gateway. Must be servicenow.
-i, --iter <n>1Number of iterations per scenario
-p, --scenarios-path <path>auto-detectedPath to the scenarios root directory
--env-file <path>auto-detectedPath to the env file
--parallelRun scenarios concurrently using available RAM. Requires --mode incus or --mode lima.
--parallel-ram-cap <gb>Cap the RAM budget for parallel execution in GB. Requires --parallel.
--log-file <path>Mirror all output (ANSI-stripped) to a file
-v / -vv-v: show only failed checks. -vv: show all checks with full debug output.

​
Entry Points

The --from flag controls how each scenario is dispatched. --from gateway (default) — creates a ticket in ServiceNow. Exercises the full integration: gateway → job → agent → ticket resolution. Requires --gateway servicenow and SERVICENOW_* env vars. 
2501-runner run -s nginx/001-broken-config --from gateway --gateway servicenow
--from ticket — POSTs directly to the engine’s internal ticket endpoint. Requires ENGINE_API_URL and ENGINE_API_KEY env vars. 
2501-runner run -s nginx/001-broken-config --from ticket

​
Environment Variables

VariableRequiredDescription
DATABASE_URLalwaysPostgreSQL connection string
ORG_IDalwaysYour organization ID
TENANT_IDalwaysYour tenant ID
USER_IDalwaysThe user ID under which scenarios run
SERVICENOW_API_URL--from gatewayServiceNow instance URL
SERVICENOW_USERNAME--from gatewayServiceNow username
SERVICENOW_PASSWORD--from gatewayServiceNow password
SERVICENOW_ASSIGNMENT_GROUP_ID--from gatewayAssignment group for created tickets
SERVICENOW_CALLER_ID--from gatewayCaller ID for created tickets
ENGINE_API_URL--from ticketBase URL of the 2501 engine API
ENGINE_API_KEY--from ticketAPI key for engine authentication

​
Common Patterns

# Run a single scenario
2501-runner run -s nginx/001-broken-config

# Run all scenarios with a tag
2501-runner run -s nginx

# Mix tags and explicit keys
2501-runner run -s nginx,disk/001-cleanup

# 5 iterations
2501-runner run -s nginx/001-broken-config -i 5

# CI with log file
2501-runner run -s regression-suite -i 3 --log-file ./runner.log

# Parallel VM runs
2501-runner run -s nginx --mode lima --parallel --parallel-ram-cap 16