Pull the images from 2501's registry

The chart names every image by its digest, in the registry global.imageRegistry names in your values.yaml: 2501’s, an Amazon ECR registry, or your copy of it. The engine, Command Center, executor-mcp and kerberos-worker images also carry latest, the newest build 2501 shipped. The chart uses it only when you set global.imageTag: latest in your values.yaml: those four images then come from the newest build, under your build’s settings, which can differ between versions. The database and Elasticsearch keep their digests. Each time you apply the chart, the engine and Command Center restart on what latest holds then, and the database migrates to it; on Swarm, run docker stack deploy again. Executors pull it as they start. This is for Helm and kubectl only, not ArgoCD: ArgoCD renders the chart again on its own schedule, and each render would restart the engine and migrate the database at a time nobody chose. With ArgoCD, follow a build by its version instead. To move to the newest build with its settings, apply its chart instead: see Upgrade. On Kubernetes your nodes can pull straight from 2501’s. This page shows how, and how to copy the images into your registry instead. ECR gives no lasting password: a pull token expires after 12 hours. So the image pull secret the chart would use does not fit here. Your nodes get their tokens from AWS’s own plugin instead, which renews them before they expire.

Where 2501 runs	Pull straight from 2501’s registry
Amazon EKS	Yes, with the nodes’ IAM role
Another Kubernetes (k3s, RKE2, kubeadm…)	Yes, with AWS’s kubelet plugin and a read-only IAM key
OpenShift	No: copy the images into your registry, see Your own registry
Docker Swarm	No: Swarm keeps the token it deployed with and fails once it expires. Copy the images into your registry and set `IMAGE_REGISTRY` in your `.env`: see Docker Swarm

1. Get access

Send your 2501 account team the AWS role or user your nodes will pull as, for example arn:aws:iam::<your account>:role/<node role>. 2501 grants it read access to the release images. Nothing else in 2501’s account is reachable with it. On your side, that role or user needs this policy:

{
  "Version": "2012-10-17",
  "Statement": [
    { "Effect": "Allow", "Action": "ecr:GetAuthorizationToken", "Resource": "*" },
    {
      "Effect": "Allow",
      "Action": ["ecr:BatchGetImage", "ecr:GetDownloadUrlForLayer", "ecr:BatchCheckLayerAvailability"],
      "Resource": "arn:aws:ecr:eu-west-3:605134458542:repository/2501.ai/release/*"
    }
  ]
}

2. Point the installation at 2501’s registry

In your values.yaml:

global:
  imageRegistry: 605134458542.dkr.ecr.eu-west-3.amazonaws.com/2501.ai/release
  imagePullSecrets: []

The chart still pins every image by digest, unless you set global.imageTag, and its signature check is unchanged: see Verify image signatures.

3. Let the nodes pull

Amazon EKS

Nothing to install: EKS nodes already get ECR tokens with their IAM role. Attach the policy of step 1 to the node role, or use the AWS-managed AmazonEC2ContainerRegistryReadOnly, which covers it.

Another Kubernetes

On every node, the kubelet runs AWS’s credential plugin, ecr-credential-provider, whenever it pulls from 2501’s registry. The kubelet must be 1.26 or newer. k3s and RKE2 look for the plugin and its configuration in /var/lib/rancher/credentialprovider/, and use them once both are there; other distributions take two kubelet settings, in step 4.

Download the plugin, version 1.28 or newer (the first that speaks the kubelet’s v1 plugin API), from Kubernetes’ artifact server. The GitHub releases of cloud-provider-aws carry no binaries, and not every release is on the artifact server: if the download answers 404, pick another release.

V=v1.37.0 ARCH=amd64   # or arm64
URL=https://artifacts.k8s.io/binaries/cloud-provider-aws/$V/linux/$ARCH/ecr-credential-provider-linux-$ARCH
curl -fLo ecr-credential-provider "$URL"
echo "$(curl -fsL "$URL.sha256")  ecr-credential-provider" | sha256sum -c
install -D -m 0755 ecr-credential-provider /var/lib/rancher/credentialprovider/bin/ecr-credential-provider   # k3s, RKE2
install -D -m 0755 ecr-credential-provider /usr/local/bin/ecr-credential-provider                           # others

Store the read-only IAM key of step 1 on the node, readable by root only:

# /etc/kubernetes/ecr-pull.credentials
[default]
aws_access_key_id = <key id>
aws_secret_access_key = <secret>

Tell the kubelet to use the plugin for that registry, in /var/lib/rancher/credentialprovider/config.yaml on k3s and RKE2, in /etc/kubernetes/ecr-credential-provider.yaml elsewhere:

apiVersion: kubelet.config.k8s.io/v1
kind: CredentialProviderConfig
providers:
  - name: ecr-credential-provider
    matchImages:
      - "605134458542.dkr.ecr.eu-west-3.amazonaws.com"
    defaultCacheDuration: "6h"
    apiVersion: credentialprovider.kubelet.k8s.io/v1
    env:
      - name: AWS_SHARED_CREDENTIALS_FILE
        value: /etc/kubernetes/ecr-pull.credentials

Restart the kubelet. On k3s and RKE2, restart the node’s service (k3s or k3s-agent, rke2-server or rke2-agent): its log says Kubelet image credential provider bin dir and configuration file found. Elsewhere, start the kubelet with these two settings first:
--image-credential-provider-config=/etc/kubernetes/ecr-credential-provider.yaml --image-credential-provider-bin-dir=/usr/local/bin

Rotate the IAM key as your policy requires: replace the file on each node; the kubelet reads it on its next renewal.

Check

A pod that only needs its image pulled, which goes through the kubelet as the chart’s pods do. Its security settings let a namespace that enforces the restricted Pod Security Standard admit it:

kubectl -n <namespace> run ecr-check --restart=Never \
  --image=605134458542.dkr.ecr.eu-west-3.amazonaws.com/2501.ai/release/engine:<build> \
  --override-type=strategic --overrides='{"spec":{"securityContext":{"runAsNonRoot":true,"seccompProfile":{"type":"RuntimeDefault"}},"containers":[{"name":"ecr-check","securityContext":{"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]}}}]}}' \
  --command -- true
kubectl -n <namespace> get pod ecr-check   # Completed: the node pulled it
kubectl -n <namespace> delete pod ecr-check

Then install or upgrade the chart as usual. A pod stuck in ImagePullBackOff with no basic auth credentials means the plugin did not run for that registry: check the matchImages entry, and that the kubelet found the plugin and its configuration.

Your own registry

On any Kubernetes, and always on OpenShift, you can copy the images of the build into your registry instead, under the same paths (2501.ai/release/<name>), with their signatures. List the images of a build from its chart, then copy each one:

helm template ai2501 ./2501-<build>.tgz -f values.yaml \
  --set global.imageRegistry=605134458542.dkr.ecr.eu-west-3.amazonaws.com/2501.ai/release --set global.imageTag= |
  grep -o '605134458542[^"]*@sha256:[0-9a-f]*' | sort -u
cosign copy 605134458542.dkr.ecr.eu-west-3.amazonaws.com/2501.ai/release/<image>@sha256:<digest> \
  registry.bank.internal:5000/2501.ai/release/<image>

Then point the installation at your registry, with the secret holding its read-only login if it needs one:

global:
  imageRegistry: registry.bank.internal:5000/2501.ai/release
  imagePullSecrets: [registry-pull]

A digest names the same image in every registry, so the chart runs the same images, executor images included. On OpenShift, the cluster’s global pull secret can hold your registry’s login instead of imagePullSecrets. Copy the new images before each upgrade. With global.imageTag: latest, also copy the four 2501 images under that tag after each release, since a copy by digest carries no tag:

cosign copy 605134458542.dkr.ecr.eu-west-3.amazonaws.com/2501.ai/release/<image>:latest registry.bank.internal:5000/2501.ai/release/<image>:latest

, for engine, command-center, executor-mcp and kerberos-worker.

API

Deploy

Configure

Pull the images from 2501's registry

1. Get access

2. Point the installation at 2501’s registry

3. Let the nodes pull

Amazon EKS

Another Kubernetes

Check

Your own registry

​1. Get access

​2. Point the installation at 2501’s registry

​3. Let the nodes pull

​Amazon EKS

​Another Kubernetes

​Check

​Your own registry

1. Get access

2. Point the installation at 2501’s registry

3. Let the nodes pull

Amazon EKS

Another Kubernetes

Check

Your own registry