> ## Documentation Index
> Fetch the complete documentation index at: https://docs.2501.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Docker Swarm

> Prepare, deploy and check 2501 with the release compose file and your site settings

Deploy 2501 with the release's `compose.yaml`, your `.env` settings and Docker secrets. Keep site-specific changes in a separate `site.yaml` so they survive upgrades.

```mermaid theme={null}
flowchart LR
  A[Prepare managers and registry] --> B[Write settings]
  B --> C[Create secrets]
  C --> D[Review and deploy stack]
  D --> E[Check migration and sign in]
```

## Prepare the Swarm

**1. Download the build**

Get `compose.yaml` and `env.example` from [installation releases](https://github.com/2501-ai/install/releases). [Verify the files](https://docs.2501.ai/0.19/deployment/verify-downloads.md#verify-the-installation-files).

**2. Prepare Docker**

Use Docker 23.0 or newer on every node. Initialize the Swarm if needed and join its nodes.

**3. Choose the data node**

Run `docker node update --label-add 2501-data=true <node>` for the node that stores the supplied PostgreSQL database.

**4. Copy the release images**

Copy every image into your registry under the same paths and digests. Set its prefix as `IMAGE_REGISTRY` below.

The engine runs on a manager and needs access to its Docker socket. Database volumes are local to the labelled data node. See [registry access](https://docs.2501.ai/0.19/deployment/pull-from-2501.md) for image copying; 2501's registry login expires after 12 hours and Swarm cannot renew it.

List the images to copy from the release file:

```bash theme={null}
grep -oE '[a-z-]+@sha256:[0-9a-f]{64}' compose.yaml | sort -u
```

## Write your .env

Copy `env.example` to `.env` beside `compose.yaml`. Use Docker secrets for passwords and keys.

| Variable | Example or action |
| - | - |
| `COMMAND_CENTER_HOST` | `2501.example.com` |
| `COMMAND_CENTER_PORT` | `3000`; publish HTTP behind your TLS load balancer |
| `DOCKER_GID` | Group returned by `stat -c %g /var/run/docker.sock` on the managers |
| `IMAGE_REGISTRY` | `registry.bank.internal:5000/2501.ai/release` |
| `TENANT_NAME`, `ORGANIZATIONS`, `ADMIN_EMAIL` | Optional initial identity; quote spaces, such as `TENANT_NAME='Acme Bank'` |
| `TENANT_ID` | Set the tenant ID supplied with your license, if applicable |
| `LLM_PROVIDER`, `LLM_BASE_URL`, `LLM_MODEL` | Supply all three or leave all three out |

The first three variables are required. Identity and LLM values initialize a new installation only; change them later in Command Center. Without initial LLM settings, add a model after sign-in. See the [configuration reference](https://docs.2501.ai/0.19/deployment/configuration.md) for other settings.

## Create the secrets

Run the secret-creation commands at the top of `compose.yaml` on a manager, after replacing every placeholder. Existing secrets are skipped. Before the first deploy, remove and recreate any secret accidentally made with a placeholder.

Create `llm-api-key-v1` even without LLM settings; any value is sufficient when those settings are absent because the stack still mounts the secret.

> [!WARNING]
>
> Preserve the credential key, engine API key, authentication secret, log encryption key and database password across upgrades. Replacing them can make existing data inaccessible. Never commit secret values.

## Deploy

Load the settings and sign in to your registry:

```bash theme={null}
STACK=ai-2501
set -a
. ./.env
set +a
[ -z "$IMAGE_REGISTRY" ] || docker login "${IMAGE_REGISTRY%%/*}"
docker stack config -c compose.yaml
docker stack deploy --with-registry-auth --prune -c compose.yaml "$STACK"
docker service logs "${STACK}_bootstrap"
```

If you use `site.yaml`, add `-c site.yaml` to both stack commands. Review the merged configuration before deploying. Pruning removes services dropped by a build and retains data volumes.

The bootstrap service migrates the database and initializes missing identity and model settings. It retries while the database starts. Investigate persistent migration errors in its logs instead of treating a running web service as a completed installation.

Bootstrap retries stop at the configured limit of 10 attempts. Fix the cause shown in the logs, then remove the failed bootstrap service and redeploy the stack:

```bash theme={null}
docker service rm "${STACK}_bootstrap"
docker stack deploy --with-registry-auth --prune -c compose.yaml "$STACK"
docker service logs "${STACK}_bootstrap"
```

Include `-c site.yaml` again if you use it.

## Adapt to your site

Keep the published `compose.yaml` unchanged. Start `site.yaml` with its `version:` value and merge the required recipes below. Apply with both files after each change.

> [accordion] HTTP proxy and corporate CA
>
> Create the CA configuration with `docker config create corporate-ca corporate-ca.pem`:
>
> ```yaml theme={null}
> version: "3.8"
> x-proxy: &proxy
>   HTTP_PROXY: http://proxy.bank.internal:3128
>   HTTPS_PROXY: http://proxy.bank.internal:3128
>   NO_PROXY: engine,command-center,postgres,elasticsearch,127.0.0.1,localhost
> services:
>   engine:
>     environment:
>       <<: *proxy
>       NODE_EXTRA_CA_CERTS: /etc/2501/customer-ca.crt
>     configs:
>       - { source: customer-ca, target: /etc/2501/customer-ca.crt }
>   command-center:
>     environment:
>       <<: *proxy
>       CA_BUNDLE_FILES: /etc/2501/customer-ca.crt
>     configs:
>       - { source: customer-ca, target: /etc/2501/customer-ca.crt }
> configs:
>   customer-ca: { name: corporate-ca, external: true }
> ```
>
> Keep only the proxy or CA lines if you need one. Add your internal domains to `NO_PROXY`.

> [accordion] HTTPS to the engine
>
> Issue a certificate for `engine`. Create Docker secrets `engine-tls.tls.crt` and `engine-tls.tls.key` from its certificate and key:
>
> ```yaml theme={null}
> version: "3.8"
> services:
>   engine:
>     environment:
>       API_BASE_URL: https://engine:1337
>       API_TLS_CERT_FILE: /run/secrets/engine-tls.tls.crt
>       API_TLS_KEY_FILE: /run/secrets/engine-tls.tls.key
>     secrets: [engine-tls.tls.crt, engine-tls.tls.key]
>   command-center:
>     environment:
>       ENGINE_BASE_URL: https://engine:1337
>       CA_BUNDLE_FILES: /run/secrets/engine-tls.tls.crt
>     secrets: [engine-tls.tls.crt]
> secrets:
>   engine-tls.tls.crt: { external: true }
>   engine-tls.tls.key: { external: true }
> ```
>
> With a corporate CA too, list both paths in `CA_BUNDLE_FILES`, separated by a space. Renew certificates under new secret names and update the references.

> [accordion] Your PostgreSQL
>
> Create `pg-connection-v1.DATABASE_URL` and `pg-connection-v1.DIRECT_URL` as Docker secrets containing your connection strings with TLS verification. The direct URL bypasses any pooler.
>
> ```yaml theme={null}
> version: "3.8"
> x-database: &database
>   SECRET_FILE_DATABASE_URL: /run/secrets/pg-connection-v1.DATABASE_URL
>   SECRET_FILE_DIRECT_URL: /run/secrets/pg-connection-v1.DIRECT_URL
> x-database-secrets: &database-secrets
>   - pg-connection-v1.DATABASE_URL
>   - pg-connection-v1.DIRECT_URL
> services:
>   engine: { environment: *database, secrets: *database-secrets }
>   command-center: { environment: *database, secrets: *database-secrets }
>   bootstrap: { environment: *database, secrets: *database-secrets }
>   postgres: { deploy: { replicas: 0 } }
> secrets:
>   pg-connection-v1.DATABASE_URL: { external: true }
>   pg-connection-v1.DIRECT_URL: { external: true }
> ```
>
> Keep `postgres-password-v1`: services still mount it. Check the bootstrap logs after deploying.

## Check

```bash theme={null}
docker stack services "$STACK"
curl -f "http://<a node>:$COMMAND_CENTER_PORT/health"
curl -f "http://<a node>:$COMMAND_CENTER_PORT/ready"
```

| Check | Expected result |
| - | - |
| Stack services | Expected replicas are running |
| Bootstrap logs | Migration and initialization complete |
| `/health` | 200: Command Center is running |
| `/ready` | 200: this process has confirmed its required migrations |

Swarm's supplied health check uses `/health`; it does not wait for database readiness. Check `/ready` separately before opening access. After its first success, readiness remains successful even during a later database outage.

Sign in at `https://<your host>`. The engine stops before its replacement starts during upgrades; allow running tasks to finish first.

## gMSA hosts

For Windows targets using gMSA, the engine's manager must reach domain controllers over LDAPS and Kerberos. Include the build's `kerberos-worker` image when copying release images. The worker starts automatically when needed.

The worker service is separate from the stack. If the engine did not stop cleanly before `docker stack rm`, remove the remaining worker with `docker service rm kerberos-worker`.

## Hosted MCP servers

[Hosted plugins](https://docs.2501.ai/0.19/configure/plugins.md) run as containers on the engine's manager. Supply private-registry access in the `registry-auth` secret's Docker `config.json`, or pre-pull images on each manager.

After the hosted network first exists, apply isolation on every eligible manager before assigning production work:

```bash theme={null}
SUBNET=$(docker network inspect 2501-mcp-hosted -f '{{(index .IPAM.Config 0).Subnet}}')
for RANGE in 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 100.64.0.0/10 169.254.0.0/16; do
  iptables -I DOCKER-USER -s "$SUBNET" -d "$RANGE" -j DROP
done
iptables -I INPUT -s "$SUBNET" -j DROP
```

These rules block private destinations and the manager itself. Arrange persistent firewall rules and review them if the Docker network changes. Add narrowly scoped exceptions for required internal services. Reserve manager capacity for plugin containers alongside the engine.

## Centrals

Local [Centrals](https://docs.2501.ai/0.19/configure/centrals.md) need a dedicated internal, non-attachable overlay network and a controlled dialback proxy. Configure `CENTRAL_LOCAL_SWARM_NETWORK` and `CENTRAL_LOCAL_SWARM_PROXY` in the engine's `site.yaml` environment. Set `CENTRAL_LOCAL_DIALBACK_URL` to the engine's reachable HTTPS address. The network and proxy settings must contain `{org_id}` for organization-specific routing.

The standard stack does not provision this environment. Use the preparation recipe below, or select a registered [cluster](https://docs.2501.ai/0.19/configure/clusters.md) that meets these requirements. Run the Central's dry run before assigning work; it checks execution and cleanup, not task credentials.

> [accordion] Prepare local Central execution
>
> Centrals require Docker Engine API 1.47 or newer and registry access at each launch. Prepare a separate network per organization. Use your actual tenant and organization IDs for `TENANT_ID` and `ORG_ID` below.
>
> **1. Create the dedicated network**
>
> Use the command below only after your site has qualified the proxy and network filtering. Do not attach application infrastructure or Host executors.
>
> **2. Deploy your controlled proxy**
>
> Attach your approved proxy service to the network. Give the service the labels below, using the same tenant and organization IDs.
>
> **3. Verify isolation**
>
> Permit the installation's HTTPS callback and required tool destinations through the proxy. Verify denial of private infrastructure, metadata and unwanted incoming traffic before approving this network.
>
> **4. Configure and test**
>
> Set the engine environment below in `site.yaml`, redeploy, then run the Central's **Test** in **Dry run** and confirm cleanup.
>
> ```bash theme={null}
> docker network create --driver overlay --internal \
>   --label 2501.ai/central-isolation=verified \
>   --label 2501.ai/central-ingress=default-deny \
>   --label 2501.ai/central-egress=default-deny \
>   --label 2501.ai/tenant-id="$TENANT_ID" \
>   --label 2501.ai/org-id="$ORG_ID" \
>   "central-$ORG_ID"
> ```
>
> Proxy service labels belong under `deploy.labels` in its Compose definition:
>
> ```yaml theme={null}
> deploy:
>   labels:
>     2501.ai/managed-by: 2501-central-egress
>     2501.ai/tenant-id: <tenant-id>
>     2501.ai/org-id: <organization-id>
> ```
>
> Labels record your verification; they do not implement filtering. Swarm permits traffic between tasks on the same overlay. Provide and verify an external filter if you require task-to-task isolation.
>
> ```yaml theme={null}
> version: "3.8"
> services:
>   engine:
>     environment:
>       CENTRAL_LOCAL_SWARM_NETWORK: "central-{org_id}"
>       CENTRAL_LOCAL_SWARM_PROXY: "http://central-proxy-{org_id}:3128"
>       CENTRAL_LOCAL_DIALBACK_URL: "https://2501-callback.example.com"
> ```
>
> Replace the proxy and callback addresses with your reachable services. The callback certificate must be trusted. Use a separate proxy service for each organization, with its corresponding network and labels. Requalify the setup after network or proxy changes before admitting more Central work.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.