Kubernetes - 2501.ai

2501 runs on any Kubernetes distribution - vanilla k8s, OpenShift, k3s, EKS, GKE, AKS, Minikube. The engine runs each tool executor in its own isolated sandbox. On Docker and Docker Swarm that sandbox is a container; on Kubernetes the engine creates a short-lived Kubernetes Pod instead - a pod is created when work starts, the engine attaches to it over stdin and stdout, and the pod is deleted when the task ends. This page covers how to generate the deployment manifests with the 2501 CLI, how the Kubernetes backend selects itself, and the RBAC the engine needs.

Docker Swarm remains the default deployment target, with a fully managed deploy lifecycle. The Kubernetes target is generate-only: the CLI renders a complete manifest tree and you apply it with kubectl (or your distribution’s equivalent CLI). This keeps you in control of how changes reach the cluster, including through GitOps tooling.

Generating the manifests

Run the deploy command with the Kubernetes target:

2501 infra deploy --target kubernetes

The CLI prompts for a stack name, namespace, image tag, Command Center public URL, and your Elasticsearch and AWS (registry pull) values, then renders a ready-to-apply manifest tree under <configDir>/kubernetes/:

kubernetes/
├── namespace.yml
├── configmap/        # engine, command-center
├── secret/           # engine, command-center, postgres, ecr-pull-secret, license-public-key
├── deployment/       # engine, command-center, postgres
├── service/          # engine, command-center, postgres
├── pvc/              # postgres
└── rbac/             # executor ServiceAccount, Role, RoleBinding

For non-interactive use, pass the values as flags:

2501 infra deploy --target kubernetes --image-tag <tag> \
  --stack-name ai-2501 --namespace ai-2501 --yes

--image-tag (alias --tag) is required with --yes. --stack-name and --namespace both default to ai-2501, and both are validated as DNS-1123 labels.

Flags

2501 infra deploy -h

Usage: 2501 infra deploy [options]

Deploy 2501 stack

Options:
  --remove                       Remove the deployed stack
  --stop                         Stop the deployed stack
  -t, --tag <tag>                Image tag to deploy (overrides config, e.g. for
                                 testing a build)
  --target <target>              Deployment target: swarm or kubernetes
                                 (overrides config)
  --stack-name <name>            Kubernetes: stack name for Service names/labels
                                 (default: ai-2501)
  --namespace <name>             Kubernetes: target namespace (default: ai-2501)
  --image-tag <tag>              Kubernetes: image tag to deploy (required for
                                 --target kubernetes; alias of --tag)
  -r, --restart [service]        Restart with env reload (optional: engine,
                                 command-center)
  --history                      Show deployment history
  -d, --dir <path>               Config directory (default: /etc/2501)
  -y, --yes                      Non-interactive mode - skip prompts and
                                 seeding, fail on errors
  --cleanup-resources            Clean up unused images and resources before
                                 deployment
  --cleanup-docker-resources     Alias for --cleanup-resources (deprecated)
  --disable-services [services]  Disable services (comma-separated: engine,
                                 command-center, or omit to disable all)
  -h, --help                     display help for command

--remove, --stop, --restart, --history, --cleanup-resources, and --disable-services drive the Swarm lifecycle; on the Kubernetes target the relevant options are --target, --stack-name, --namespace, --image-tag, -d, --dir, and -y, --yes.

Filling in secrets

Secret manifests use stringData, so every value is plain text and the cluster encodes it server-side on apply. Before applying, fill in any blank values directly in the YAML:

AWS_* and ELASTICSEARCH_* in secret/engine.yml and secret/command-center.yml
BASE_URL in configmap/command-center.yml (your real ingress hostname)
The license public key in secret/license-public-key.yml, unless a PEM file already exists at <configDir>/license.public.pem

Generation is safe to re-run. Auto-generated secrets (the PostgreSQL password, credential encryption key, engine API key, auth secret, and log encryption key) are created once and read back from the existing YAML on subsequent runs, and values you typed into the files by hand are preserved. An empty prompt answer never overwrites a previously supplied value.

Applying

kubectl apply -f <configDir>/kubernetes/ -R

(Substitute your distribution’s CLI if you don’t use kubectl - oc, k3s kubectl, etc.)

The registry pull secret is a short-lived ECR token that expires after roughly 12 hours. Recreate it before each apply:

kubectl create secret docker-registry ecr-pull-secret \
  --docker-server=<registry> --docker-username=AWS \
  --docker-password=$(aws ecr get-login-password --region <region>) \
  -n <namespace> --dry-run=client -o yaml | kubectl apply -f -

Executor backend selection

The engine picks an executor backend automatically:

When it detects that it is running inside a Kubernetes cluster, it uses the Kubernetes pod backend.
Otherwise it falls back to Docker / Swarm.

You can force a backend with the EXECUTOR_MCP_BACKEND environment variable, set to kube or docker. An executor must answer the MCP handshake within 20 seconds; one that does not is started once more in a fresh container before the task fails. MCP_HANDSHAKE_TIMEOUT_MS on the engine changes that bound. An executor that runs through a satellite gets instead as long as its session waits for a silent workload before closing: SATELLITE_SILENCE_MS, 40 seconds by default.

The engine talks to the in-cluster Kubernetes API directly using the pod’s auto-mounted service-account token. It does not shell out to kubectl, and you do not supply a kubeconfig.

MCP settings of the engine

The generator does not render the engine’s MCP settings. Put them in an optional Secret named engine-extra-env in the namespace: the engine Deployment reads it last, so its keys win, and generating the manifests again never overwrites it. Create or change it, then restart the engine (oc on OpenShift):

kubectl create secret generic engine-extra-env -n <namespace> \
  --from-literal=MCP_OAUTH_ALLOW_PRIVATE_PROVIDERS=true \
  --dry-run=client -o yaml | kubectl apply -f -
kubectl rollout restart deploy/engine -n <namespace>

Setting	Unset	Set
`MCP_OAUTH_ALLOW_PRIVATE_PROVIDERS`	OAuth plugins reach public identity providers only	`true` lets them use one on a private address, still over HTTPS. An installation without internet access that connects OAuth plugins needs it, since its identity provider is internal.
`MCP_HANDSHAKE_TIMEOUT_MS`	An executor answers the MCP handshake within 20000 ms	Another bound, in milliseconds
`MCP_HOSTED_HANDSHAKE_TIMEOUT_MS`	A hosted MCP server answers it within 60000 ms	Another bound, for an image slow to start
`MCP_HOSTED_IMAGE_PULL_SECRETS`	Hosted MCP server pods name the executors’ pull secret	The pull secrets they name instead, comma-separated; empty names none, so the namespace’s `default` service account’s apply

Required access (RBAC)

The generated rbac/executor.yml provisions a dedicated engine-executor ServiceAccount with a namespace-scoped Role and RoleBinding granting:

Resource	Verbs
`pods`	`create`, `get`, `list`, `watch`, `delete`
`pods/attach`	`create`, `get`
`pods/status`	`get`
`pods/log`	`get`
`secrets`	`create`

The secrets rule is for hosted MCP servers: a hosted MCP server’s environment, its credentials included, reaches its pod through a Secret that the pod owns, so Kubernetes deletes it with the pod. Hosted MCP server pods mount no service-account token. They run non-root, with the RuntimeDefault seccomp profile and without capabilities or privilege escalation, so a namespace that enforces the restricted Pod Security Standard admits them. They run as user and group 65532, except on OpenShift, which the engine recognizes by its security.openshift.io API group: there the pods name no user, and the restricted-v2 constraint assigns one from the namespace’s range. If the engine cannot read the cluster’s API groups, it names user 65532 and logs a warning. The engine reads pods/log to report the last output of a hosted MCP server pod that ended before it was ready, and lists pods to remove the hosted MCP server pods it left when it restarted. Executor pods spawn in the same namespace as the engine. To use a separate sandbox namespace, change the RoleBinding’s namespace and override EXECUTOR_MCP_K8S_NAMESPACE on the engine pod, which is otherwise set to the engine’s own namespace via the Downward API. Hosted MCP server pods in another namespace than the engine’s are not owned by the engine pod; the engine removes the ones it left when it starts its first hosted MCP server again. The generated networkpolicy/mcp-hosted.yml applies the mcp-hosted-isolation NetworkPolicy, on OpenShift too. It selects only the pods labelled app.kubernetes.io/managed-by: 2501-mcp-hosted, so every other pod is left as it is, even on a namespace without a default-deny: nothing may reach those pods, and they may reach the cluster’s DNS pods (ports 53 and 5353) and public addresses only, no private range and no link-local address such as a cloud metadata service. Policies add up, so to let hosted MCP servers reach an internal service, apply a NetworkPolicy of your own that selects the same label and allows that range, for example allow-mcp-hosted-internal from kubernetes-overlays/networkpolicy.example.yml. A cluster whose pods resolve names through NodeLocal DNSCache (169.254.20.10) needs that address allowed the same way. Enforcement depends on the CNI supporting NetworkPolicy. Size the namespace for hosted MCP servers too. An engine runs at most 32 of them at once, each pod requesting a quarter of a CPU, 512 MiB and 64 MiB of ephemeral storage and limited to half a CPU, 512 MiB and 256 MiB of ephemeral storage: up to 8 CPUs, 16 GiB and 2 GiB of ephemeral storage of requests (16 CPUs and 8 GiB of ephemeral storage of limits) on top of the engine, the Command Center, the databases and the executors. They share the namespace’s ResourceQuota with the executors, so check the quota against that: once it is used up, a hosted MCP server or an executor fails to start with Kubernetes’ exceeded quota message, and the hosted MCP server is tried again later. The engine Deployment sets terminationGracePeriodSeconds: 150. On stop, the engine gives its running tasks up to 110 seconds, then stops its hosted MCP servers and ends its MCP sessions; the Kubernetes default of 30 seconds would stop it in the middle. Keep at least 120 seconds if you change it. On Amazon EKS with the VPC CNI enforcing network policies, a new pod’s traffic is filtered only once the CNI applies its policies, a few seconds after the pod starts, so a hosted MCP server can reach private addresses in that window. Setting NETWORK_POLICY_ENFORCING_MODE=strict on the VPC CNI add-on closes it: every new pod is denied all traffic until its policies apply, so every workload in the cluster needs the policies that allow its traffic first.

gMSA-bound Windows targets are not supported on the Kubernetes executor backend. They depend on a Docker-volume Kerberos sidecar that this backend does not provide. Use the Docker or Swarm backend for those targets. See Docker Swarm.

For the full set of engine and Command Center environment variables (database, security keys, LLM providers, optional integrations), see Configuration.

​Generating the manifests

​Flags

​Filling in secrets

​Applying

​Executor backend selection

​MCP settings of the engine

​Required access (RBAC)

Generating the manifests

Flags

Filling in secrets

Applying

Executor backend selection

MCP settings of the engine

Required access (RBAC)