This is part of a multi-part series on getting Docker Swarm up and running. You might want to start with the original post called Bootstrapping Docker Swarm.
Deploying the Registry
Now that we have a front door to our cluster we want to set up a registry so that we can keep our containers in it and serve them up to our cluster. The registry is the ultimate goal of our bootstrap. It allows us to store images that we create so that they can be deployed to our swarm cluster. The registry will run on the infra host. Before we can deploy the registry we need to set up the infra host to store the data created by the registry by running these commands:
mkdir -p /srv/registry/docker/passwords mkdir -p /srv/registry/docker/data
We’re going to use basic authentication for our registry and the passwords are going to be stored in /srv/registry/docker/passwords/htpasswd. To add a new username and password, run this command:
docker run --entrypoint htpasswd registry:2 -Bbn \ testuser \ testpassword \ >> /srv/registry/docker/passwords/htpasswd
To deploy the registry we only need to create a docker-stack.yml file. There are no additional configurations. Our docker-stack.yml file looks like this:
version: "3.6" services: registry: image: registry:2 deploy: mode: replicated replicas: 1 update_config: parallelism: 1 delay: 10s restart_policy: condition: any placement: constraints: - "node.labels.infra==true" environment: REGISTRY_HTTP_ADDR: "0.0.0.0:8080" REGISTRY_HTTP_HOST: https://registry.example.com REGISTRY_AUTH: htpasswd REGISTRY_AUTH_HTPASSWD_PATH: /passwords/htpasswd REGISTRY_AUTH_HTPASSWD_REALM: Registry Realm volumes: - type: bind source: /srv/registry/docker/data target: /var/lib/registry - type: bind source: /srv/registry/docker/passwords target: /passwords networks: - public_proxy networks: public_proxy: name: public_proxy external: true
You’ll notice in this configuration that we are only assigning this stack to the host with the infra label. Further, we are connecting it to the
public_proxy network that we created with the haproxy stack. This means that the registry and HAproxy can talk directly to one another. Nothing can access the registry except through HAProxy.
There isn’t really anything extra to deploying a registry. The Docker Registry page has a lot of information about additional options that you may wish to use when deploying your registry. You should now be able to log in to your registry with this command:
docker login https://registry.example.com -u whatever
whatever to match your username and then enter the password that you created. You only need to log in once and your credentials are saved in
Moving Images to the Registry
After you’ve got this shiny, new, empty registry we want to take all of the images that we just built and move them into it. This includes:
- syslog – deployed to infra
- acme-dns – deployed to infra
- certbot – deployed to infra
- keepalived – deployed to lb01 and lb02
The steps for moving each image are the same but they’re rather long. The steps are:
- From your workstation or some other host, build the new image.
- Push the new image to the registry.
- Pull the new image to each host.
- Stop the old container that used the old image.
- Start the new container that uses the new image.
- Clean up the mess on the host.
Remember when building the images that we specified some environment variables to the docker-compose.yml file? Like this:
VERSION=latest IMAGE=certbot docker-compose build
To rebuild these images we’re going to start with the original docker-compose.yml file described before and we’re going to just not set the IMAGE variable. So we’ll build them like this:
VERSION=latest docker-compose build
The reason that we can do this is because the docker-compose.yml files set default values for the environment variable called IMAGE and those default values should point to the registry that we just created. Look at each docker-compose.yml file and verify that the new image name matches what you want it to be and then run the above command to build the container. Then run this command to push the image to our registry:
VERSION=latest docker-compose push
After the push, the image will be in the registry. We can now go to each host to push the new images. The easiest way to do this is to put the docker-compose.yml files on each host and run these commands:
VERSION=latest IMAGE=<imagename> docker-compose stop VERSION=latest docker-compose pull VERSION=latest docker-compose up -d
This will stop the existing container based on the old image, pull the new image, and start a new container based on the new image. After running that you can remove the docker-compose.yml files.
After you’ve done this you’ll notice a lot of stopped containers and unused images. You can clean up each host by running this command to “prune” unused data:
docker system prune -a -f --volumes
Moving your images to the new registry might be the hardest part simply because you have to be sure to pull the new image before stopping the old image and because if you do it in the wrong order you have to basically start over. But it will make it easier to manage your images later.
In this part we got a registry up and running and we moved our containers to this new registry. In the next part we will set up a firewall to protect all of these resources if you happen to have them running anywhere near the internet.
There are still a lot more steps! Follow on to read the rest of the steps.
- Bootstrapping Docker Swarm Part 1: Installing Docker and Enabling Docker Swarm
- Bootstrapping Docker Swarm Part 2: Setting Up Syslog
- Bootstrapping Docker Swarm Part 3: Getting SSL Certificates
- Bootstrapping Docker Swarm Part 4: Configuring the Load Balancers
- Bootstrapping Docker Swarm Part 5: Enabling the Registry
- Bootstrapping Docker Swarm Part 6: Setting up a Firewall