Bootstrapping Docker Swarm Part 5: Enabling the Registry

This is part of a multi-part series on getting Docker Swarm up and running. You might want to start with the original post called Bootstrapping Docker Swarm.

Deploying the Registry

Now that we have a front door to our cluster we want to set up a registry so that we can keep our containers in it and serve them up to our cluster. The registry is the ultimate goal of our bootstrap. It allows us to store images that we create so that they can be deployed to our swarm cluster. The registry will run on the infra host. Before we can deploy the registry we need to set up the infra host to store the data created by the registry by running these commands:

mkdir -p /srv/registry/docker/passwords
mkdir -p /srv/registry/docker/data

We’re going to use basic authentication for our registry and the passwords are going to be stored in /srv/registry/docker/passwords/htpasswd. To add a new username and password, run this command:

docker run --entrypoint htpasswd registry:2 -Bbn \
    testuser \
    testpassword \
    >> /srv/registry/docker/passwords/htpasswd

To deploy the registry we only need to create a docker-stack.yml file. There are no additional configurations. Our docker-stack.yml file looks like this:

version: "3.6"

    image: registry:2
      mode: replicated
      replicas: 1
        parallelism: 1
        delay: 10s
        condition: any
          - "node.labels.infra==true"
      REGISTRY_AUTH: htpasswd
      REGISTRY_AUTH_HTPASSWD_PATH: /passwords/htpasswd
      - type: bind
        source: /srv/registry/docker/data
        target: /var/lib/registry
      - type: bind
        source: /srv/registry/docker/passwords
        target: /passwords
      - public_proxy

    name: public_proxy
    external: true

You’ll notice in this configuration that we are only assigning this stack to the host with the infra label. Further, we are connecting it to the public_proxy network that we created with the haproxy stack. This means that the registry and HAproxy can talk directly to one another. Nothing can access the registry except through HAProxy.

There isn’t really anything extra to deploying a registry. The Docker Registry page has a lot of information about additional options that you may wish to use when deploying your registry. You should now be able to log in to your registry with this command:

docker login -u whatever

Obviously change whatever to match your username and then enter the password that you created. You only need to log in once and your credentials are saved in ~/.docker/config.json.

Moving Images to the Registry

After you’ve got this shiny, new, empty registry we want to take all of the images that we just built and move them into it. This includes:

  • syslog – deployed to infra
  • acme-dns – deployed to infra
  • certbot – deployed to infra
  • keepalived – deployed to lb01 and lb02

The steps for moving each image are the same but they’re rather long. The steps are:

  • From your workstation or some other host, build the new image.
  • Push the new image to the registry.
  • Pull the new image to each host.
  • Stop the old container that used the old image.
  • Start the new container that uses the new image.
  • Clean up the mess on the host.

Remember when building the images that we specified some environment variables to the docker-compose.yml file? Like this:

VERSION=latest IMAGE=certbot docker-compose build

To rebuild these images we’re going to start with the original docker-compose.yml file described before and we’re going to just not set the IMAGE variable. So we’ll build them like this:

VERSION=latest docker-compose build

The reason that we can do this is because the docker-compose.yml files set default values for the environment variable called IMAGE and those default values should point to the registry that we just created. Look at each docker-compose.yml file and verify that the new image name matches what you want it to be and then run the above command to build the container. Then run this command to push the image to our registry:

VERSION=latest docker-compose push

After the push, the image will be in the registry. We can now go to each host to push the new images. The easiest way to do this is to put the docker-compose.yml files on each host and run these commands:

VERSION=latest IMAGE=<imagename> docker-compose stop
VERSION=latest docker-compose pull
VERSION=latest docker-compose up -d

This will stop the existing container based on the old image, pull the new image, and start a new container based on the new image. After running that you can remove the docker-compose.yml files.

After you’ve done this you’ll notice a lot of stopped containers and unused images. You can clean up each host by running this command to “prune” unused data:

docker system prune -a -f --volumes

Moving your images to the new registry might be the hardest part simply because you have to be sure to pull the new image before stopping the old image and because if you do it in the wrong order you have to basically start over. But it will make it easier to manage your images later.

Next Steps

In this part we got a registry up and running and we moved our containers to this new registry. In the next part we will set up a firewall to protect all of these resources if you happen to have them running anywhere near the internet.

There are still a lot more steps! Follow on to read the rest of the steps.