Configuring registry mirrors

Registry Mirror

Registry mirrors are designated to mirror images from gcr.io and docker.io. For example, if you try to pull an image gcr.io/kubernetes-e2e-test-images/nautilus:1.0, this only works if your registry service has this image in the exact same path, like 172.18.0.20:5000/kubernetes-e2e-test-images/nautilus:1.0. All non-gcr.io images still work as normal, for example, you can still pull k8s.gcr.io/pause:3.1.

Using a registry mirror helps you save on traffic and offers an alternative to using gcr.io in case you need to insulate your clusters from gcr.io outages. It also allows you to conduct your own vulnerability scanning.

Before you begin

  • You must have a container registry server set up in your network.
  • If your registry server runs a private TLS certificate, you must have the certificate authority (CA) file.
  • If your registry server needs authentication, you must have the proper login credentials or Docker configuration file.

Upload container images to your registry server

Upload the images from the images package to your registry server by running:

actl images push \
    --private-registry=PRIVATE_REGISTRY \
    --images ~/anthos-baremetal-private-mode

Replace the following:

  • PRIVATE_REGISTRY with the private registry address (and port) and the subproject such as 172.18.0.20:5000/test-namespace.

  • Enter your username and password when prompted or select a Docker configuration file. If your registry server doesn't require credentials, then specify --no-registry-credential.

For more information on the actl images push command, run:

actl images push --help

Using your own namespace

If you want to use your own namespace in your registry server instead of the root namespace, containerd can pull from this sub namespace if you provide the API endpoint for your private registry in registryMirrors.endpoint. The endpoint is usually in the format of <REGISTRY_IP:PORT>/v2/<NAMESPACE>. Check your private registry's user guide for specific details.

For example, if you only have access to 172.18.0.20:5000/test-namespace/, you can use the following command to upload all the images under namespace test-namespace:

actl images push \
    --images= ~/anthos-baremetal-private-mode \
    --private-registry=172.18.0.20:5000/test-namespace

Then in the cluster YAML file, you can input the following to make containerd pull from the sub namespace:

registryMirrors:
  - endpoint: https://172.18.0.20:5000/v2/test-namespace

Create clusters from the registry mirror

Below is a sample cluster configuration file that uses your own registry mirror server instead of gcr.io.

If your registry doesn't require a private TLS certificate, then you can leave the caCertPath field blank.

If your registry server doesn't require an authentication Docker configuration file, then you can leave the pullCredentialConfigPath field blank.

For detailed information on cluster configuration, see Cluster config.

# Sample cluster config with registry mirror:
---
sshPrivateKeyPath: /root/ssh-key/id_rsa
registryMirrors:
  - endpoint: https://172.18.0.20:5000/v2/test-namespace
    caCertPath: /root/ca.crt
    pullCredentialConfigPath: /root/.docker/config.json
---
apiVersion: v1
kind: Namespace
metadata:
  name: cluster-admin
---
apiVersion: baremetal.cluster.gke.io/v1
kind: Cluster
metadata:
  name: admin
  namespace: cluster-admin
spec:
  nodeConfig:
    containerRuntime: containerd
...

All nodes in this cluster will use this registry mirror 172.18.0.20:5000 instead of gcr.io (and docker.io).

Fail over to gcr.io

If your cluster fails to pull from your registry mirror, it will automatically fail over to gcr.io. This is why we recommend providing a value for privateRegistryConfigPath in the cluster configuration file. If a value is not provided, your cluster is not able to pull from gcr.io in the event that your registry mirror fails.

# Sample cluster config with registry mirror:
---
privateRegistryConfigPath: /root/.docker/config.json
registryMirrors:
  - endpoint: https://172.18.0.20:5000
    caCertPath: /root/ca.crt
    pullCredentialConfigPath: /root/.docker/config.json

If you don't need the pull failover feature, then you don't need to add a privateRegistryConfigPath or add gcr.io(and docker.io) to your proxy allow list.

Update registry mirror endpoints, certificates, and pull credentials

To update registry mirror endpoints, certificates, or pull credentials:

  1. In the cluster configuration file, update the endpoint, CA certificate file, and pull credential configuration file path.

  2. Apply the changes by running:

    actl clusters baremetal update cluster admin --kubeconfig=ADMIN_KUBECONFIG