> ## Documentation Index
> Fetch the complete documentation index at: https://docs.2501.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Docker Swarm

> Prepare, deploy and check 2501 with the release compose file and your site settings

Deploy 2501 with the release's `compose.yaml`, your `.env` settings and Docker secrets. Keep site-specific changes in a separate `site.yaml` so they survive upgrades.

```mermaid theme={null}
flowchart LR
  A[Prepare managers and registry] --> B[Write settings]
  B --> C[Create secrets]
  C --> D[Review and deploy stack]
  D --> E[Check migration and sign in]
```

## Prepare the Swarm

**1. Download the build**

Get `compose.yaml` and `env.example` from [installation releases](https://github.com/2501-ai/install/releases). [Verify the files](https://docs.2501.ai/0.20/deployment/verify-downloads.md#verify-the-installation-files).

**2. Prepare Docker**

Use Docker 23.0 or newer on every node. Initialize the Swarm if needed and join its nodes.

```bash theme={null}
docker swarm init
docker swarm join --token <token> <manager-ip>:2377   # on each other node
```

**3. Choose the data node**

Run `docker node update --label-add 2501-data=true <node>` for the node that stores the supplied PostgreSQL database.

**4. Copy the release images**

Copy every image into your registry under the same paths and digests. Set its prefix as `IMAGE_REGISTRY` below.

The engine runs on a manager and needs access to its Docker socket. Database volumes are local to the labelled data node. See [registry access](https://docs.2501.ai/0.20/deployment/pull-from-2501.md) for image copying; 2501's registry login expires after 12 hours and Swarm cannot renew it.

List the images to copy from the release file:

```bash theme={null}
grep -oE '[a-z-]+@sha256:[0-9a-f]{64}' compose.yaml | sort -u
```

## Write your .env

Copy `env.example` to `.env` beside `compose.yaml`. Use Docker secrets for passwords and keys.

| Variable | Example or action |
| - | - |
| `COMMAND_CENTER_HOST` | The host name users reach Command Center on |
| `COMMAND_CENTER_PORT` | `8080` in `env.example`. Publishes HTTP on every node. Terminate TLS on your load balancer or reverse proxy. |
| `DOCKER_GID` | The group of `/var/run/docker.sock` on the managers: `stat -c %g /var/run/docker.sock` |
| `TENANT_NAME`, `ORGANIZATIONS`, `ADMIN_EMAIL` | Optional: the first tenant, its organizations (comma-separated) and the first administrator's email. Left out: tenant and organization `Default`, administrator `admin@2501.ai` |
| `TENANT_ID` | Optional: the tenant id your license is issued for, `ten_<uuid>`. Left out, the first run picks one |
| `LLM_PROVIDER`, `LLM_BASE_URL`, `LLM_MODEL`, `LLM_CONTEXT_LENGTH` | Optional: the first LLM provider and model, the first three together and with the LLM key secret, or none of them: with only some, the first run fails until you fix it. Left out, add the LLM in Command Center |
| `IMAGE_REGISTRY` | Your registry and the path the images were copied under, such as `registry.bank.internal:5000/2501.ai/release`. Unset, they come from 2501's registry, whose login lasts 12 hours: for a trial only |
| `LOG_LEVEL` | Optional: `debug`, `info` (default) or `warn` |
| `LOG_RETENTION_DAYS` | Optional: how many days the engine's and Command Center's logs are kept in Elasticsearch, 30 by default |

The first three variables are required. Quote values with spaces, such as `TENANT_NAME='Acme Bank'`. Identity and LLM values initialize a new installation only; change them later in Command Center. Without initial LLM settings, add a model after sign-in. See the [configuration reference](https://docs.2501.ai/0.20/deployment/configuration.md) for other settings.

## Create the secrets

Run the secret-creation commands at the top of `compose.yaml` on a manager, after replacing every placeholder. Existing secrets are skipped. Before the first deploy, remove and recreate any secret accidentally made with a placeholder.

Create `llm-api-key-v1` even without LLM settings; any value is sufficient when those settings are absent because the stack still mounts the secret.

> [!WARNING]
>
> Preserve the credential key, engine API key, authentication secret, log encryption key and database password across upgrades. Replacing them can make existing data inaccessible. Never commit secret values.

## Deploy

Load the settings and sign in to your registry:

```bash theme={null}
STACK=ai-2501
set -a
. ./.env
set +a
[ -z "$IMAGE_REGISTRY" ] || docker login "${IMAGE_REGISTRY%%/*}"
docker stack config -c compose.yaml
docker stack deploy --with-registry-auth --prune -c compose.yaml "$STACK"
docker service logs -f "${STACK}_bootstrap"
```

If you use `site.yaml`, add `-c site.yaml` to both stack commands. Review the merged configuration before deploying. Pruning removes services dropped by a build and retains data volumes.

The bootstrap service migrates the database and initializes missing identity and model settings. It retries while the database starts. Investigate persistent migration errors in its logs instead of treating a running web service as a completed installation.

Bootstrap retries stop at the configured limit of 10 attempts. Fix the cause shown in the logs, then remove the failed bootstrap service and redeploy the stack:

```bash theme={null}
docker service rm "${STACK}_bootstrap"
docker stack deploy --with-registry-auth --prune -c compose.yaml "$STACK"
docker service logs "${STACK}_bootstrap"
```

Include `-c site.yaml` again if you use it.

## Adapt to your site

Keep the published `compose.yaml` unchanged. Start `site.yaml` with its `version:` value and merge the required recipes below. Apply with both files after each change.

> [accordion] HTTP proxy and corporate CA
>
> Create the CA configuration with `docker config create corporate-ca corporate-ca.pem`:
>
> ```yaml theme={null}
> version: "3.8"
> x-proxy: &proxy
>   HTTP_PROXY: http://proxy.bank.internal:3128
>   HTTPS_PROXY: http://proxy.bank.internal:3128
>   NO_PROXY: engine,command-center,postgres,elasticsearch,127.0.0.1,localhost
> services:
>   engine:
>     environment:
>       <<: *proxy
>       NODE_EXTRA_CA_CERTS: /etc/2501/customer-ca.crt
>     configs:
>       - { source: customer-ca, target: /etc/2501/customer-ca.crt }
>   command-center:
>     environment:
>       <<: *proxy
>       CA_BUNDLE_FILES: /etc/2501/customer-ca.crt
>     configs:
>       - { source: customer-ca, target: /etc/2501/customer-ca.crt }
> configs:
>   customer-ca: { name: corporate-ca, external: true }
> ```
>
> Keep only the proxy or CA lines if you need one. Add your internal domains to `NO_PROXY`.

> [accordion] HTTPS to the engine
>
> Issue a certificate for `engine`. Create Docker secrets `engine-tls.tls.crt` and `engine-tls.tls.key` from its certificate and key:
>
> ```yaml theme={null}
> version: "3.8"
> services:
>   engine:
>     environment:
>       API_BASE_URL: https://engine:1337
>       API_TLS_CERT_FILE: /run/secrets/engine-tls.tls.crt
>       API_TLS_KEY_FILE: /run/secrets/engine-tls.tls.key
>     secrets: [engine-tls.tls.crt, engine-tls.tls.key]
>   command-center:
>     environment:
>       ENGINE_BASE_URL: https://engine:1337
>       CA_BUNDLE_FILES: /run/secrets/engine-tls.tls.crt
>     secrets: [engine-tls.tls.crt]
> secrets:
>   engine-tls.tls.crt: { external: true }
>   engine-tls.tls.key: { external: true }
> ```
>
> With a corporate CA too, list both paths in `CA_BUNDLE_FILES`, separated by a space. Renew certificates under new secret names and update the references.

> [accordion] Your PostgreSQL
>
> Create `pg-connection-v1.DATABASE_URL` and `pg-connection-v1.DIRECT_URL` as Docker secrets containing your connection strings with TLS verification. The direct URL bypasses any pooler.
>
> ```yaml theme={null}
> version: "3.8"
> x-database: &database
>   SECRET_FILE_DATABASE_URL: /run/secrets/pg-connection-v1.DATABASE_URL
>   SECRET_FILE_DIRECT_URL: /run/secrets/pg-connection-v1.DIRECT_URL
> x-database-secrets: &database-secrets
>   - pg-connection-v1.DATABASE_URL
>   - pg-connection-v1.DIRECT_URL
> services:
>   engine: { environment: *database, secrets: *database-secrets }
>   command-center: { environment: *database, secrets: *database-secrets }
>   bootstrap: { environment: *database, secrets: *database-secrets }
>   postgres: { deploy: { replicas: 0 } }
> secrets:
>   pg-connection-v1.DATABASE_URL: { external: true }
>   pg-connection-v1.DIRECT_URL: { external: true }
> ```
>
> Keep `postgres-password-v1`: services still mount it. Check the bootstrap logs after deploying.

## Check

```bash theme={null}
docker stack services "$STACK"
curl -f "http://<a node>:$COMMAND_CENTER_PORT/health"
curl -f "http://<a node>:$COMMAND_CENTER_PORT/ready"
```

| Check | Expected result |
| - | - |
| Stack services | Expected replicas are running |
| Bootstrap logs | Migration and initialization complete |
| `/health` | 200: Command Center is running |
| `/ready` | 200: this process has confirmed its required migrations |

Swarm's supplied health check uses `/health`. The `/ready` endpoint checks the database once at startup to confirm the required migrations. Check `/ready` separately before opening access.

Sign in at `https://<your host>`. The engine stops before its replacement starts during upgrades; allow running tasks to finish first.

## gMSA hosts

For Windows targets using gMSA, the engine's manager must reach domain controllers over LDAPS and Kerberos. Include the build's `kerberos-worker` image when copying release images. The worker starts automatically when needed.

The worker service is separate from the stack. If the engine did not stop cleanly before `docker stack rm`, remove the remaining worker with `docker service rm kerberos-worker`.

## Hosted MCP servers

[Hosted plugins](https://docs.2501.ai/0.20/configure/plugins.md) run as containers on the engine's manager. Supply private-registry access in the `registry-auth` secret's Docker `config.json`, or pre-pull images on each manager.

After the hosted network first exists, apply isolation on every eligible manager before assigning production work:

```bash theme={null}
SUBNET=$(docker network inspect 2501-mcp-hosted -f '{{(index .IPAM.Config 0).Subnet}}')
for RANGE in 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 100.64.0.0/10 169.254.0.0/16; do
  iptables -I DOCKER-USER -s "$SUBNET" -d "$RANGE" -j DROP
done
iptables -I INPUT -s "$SUBNET" -j DROP
```

These rules block private destinations and the manager itself. Arrange persistent firewall rules and review them if the Docker network changes. Add narrowly scoped exceptions for required internal services. Reserve manager capacity for plugin containers alongside the engine.

If Docker uses custom `default-address-pools`, also block those container ranges. Put narrow ACCEPT exceptions before the DROP rules. Binding a service to localhost alone does not isolate it from hosted containers; verify the firewall from a container on the hosted network.

Each hosted container is limited to 0.5 CPU, 512 MiB memory and 256 processes, with a 64 MiB `/tmp`. Its remaining writable layer has no separate storage quota. The engine permits 32 running hosted servers, so their combined limits reach 16 CPUs and 16 GiB memory, in addition to the installation itself.

## Centrals

Local [Centrals](https://docs.2501.ai/0.20/configure/centrals.md) need a dedicated internal, non-attachable overlay network and a controlled dialback proxy. Configure `CENTRAL_LOCAL_SWARM_NETWORK` and `CENTRAL_LOCAL_SWARM_PROXY` in the engine's `site.yaml` environment. Set `CENTRAL_LOCAL_DIALBACK_URL` to the engine's reachable HTTPS address. The network and proxy settings must contain `{org_id}` for organization-specific routing.

The standard stack does not provision this environment. Use the preparation recipe below, or select a registered [cluster](https://docs.2501.ai/0.20/configure/clusters.md) that meets these requirements. Run the Central's dry run before assigning work; it checks execution and cleanup, not task credentials.

> [accordion] Prepare local Central execution
>
> Centrals require Docker Engine API 1.47 or newer and registry access at each launch. Prepare a separate network per organization. Use your actual tenant and organization IDs for `TENANT_ID` and `ORG_ID` below.
>
> **1. Create the dedicated network**
>
> Use the command below only after your site has qualified the proxy and network filtering. Do not attach application infrastructure or Host executors.
>
> **2. Deploy your controlled proxy**
>
> Attach your approved proxy service to the network. Give the service the labels below, using the same tenant and organization IDs.
>
> **3. Verify isolation**
>
> Permit the installation's HTTPS callback and required tool destinations through the proxy. Verify denial of private infrastructure, metadata and unwanted incoming traffic before approving this network.
>
> **4. Configure and test**
>
> Set the engine environment below in `site.yaml`, redeploy, then run the Central's **Test** in **Dry run** and confirm cleanup.
>
> ```bash theme={null}
> docker network create --driver overlay --internal \
>   --label 2501.ai/central-isolation=verified \
>   --label 2501.ai/central-ingress=default-deny \
>   --label 2501.ai/central-egress=default-deny \
>   --label 2501.ai/tenant-id="$TENANT_ID" \
>   --label 2501.ai/org-id="$ORG_ID" \
>   "central-$ORG_ID"
> ```
>
> Proxy service labels belong under `deploy.labels` in its Compose definition:
>
> ```yaml theme={null}
> deploy:
>   labels:
>     2501.ai/managed-by: 2501-central-egress
>     2501.ai/tenant-id: <tenant-id>
>     2501.ai/org-id: <organization-id>
> ```
>
> Labels record your verification; they do not implement filtering. Swarm permits traffic between tasks on the same overlay. Provide and verify an external filter if you require task-to-task isolation.
>
> ```yaml theme={null}
> version: "3.8"
> services:
>   engine:
>     environment:
>       CENTRAL_LOCAL_SWARM_NETWORK: "central-{org_id}"
>       CENTRAL_LOCAL_SWARM_PROXY: "http://central-proxy-{org_id}:3128"
>       CENTRAL_LOCAL_DIALBACK_URL: "https://2501-callback.example.com"
> ```
>
> Replace the proxy and callback addresses with your reachable services. The callback certificate must be trusted. Use a separate proxy service for each organization, with its corresponding network and labels. Requalify the setup after network or proxy changes before admitting more Central work.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.