Building docker images on gitlab

Previously, I have made docker images for specific purposes (see docker-deb-pip, docker-pdf, docker-jekyll) using gitlab and storing it in gitlab registry. This is fairly straight forward, since gitlab.com has set up their runners to support this in a good way. It is a nice trick to speed up certain pipelines, so you don’t have to wait for e.g. packages to be installed.

I set such images to be build once per month. That way I have up-to-date images, and get an error, if there are upstream changes that breaks my image.

Docker-in-docker (dind)

In order for the the self hosted runners to allow docker-in-docker the docker container must be run in privileged mode. This is a major security issue, but is needed for docker-in-docker to work.

The trick is to start docker-in-docker as a separate container (ie. a service), that the other build containers connect to, and through that connects to the docker socket on host. This is the reason for the “unable to resolve docker” or similar errors, where the “docker” is actually the name of the dind service, and docker allows dns resolving of the hostname.

I use my own CA internally, so adding the the CA cert to both the dind service and the other containers is done in .gitlab-ci.yml.

Gitlab runner

As mentioned, the docker containers must be run in privileged mode. This is something that must be enabled in the /etc/gitlab/config.toml file on the runner.

I don’t know how gitlab have set up their system to allow users to run arbitrary code on their privileged runners - the risk of abuse is enormous. My solution would be to kill the VM often, and set concurrency to one, so multiple users are not building at the same time (and hence the risk of being able to access each others data). And, obviously, put the host somewhere in the network with very little access, and monitor the traffic.

In my example, TLS is not enabled. That is the TLS of the data going through the docker socket. This is yet another really bad security issue. TLS is enabled by default now in newer docker versions, and that gave rise to a lot of issues related to ports 2375 (unencrypted) and 2376 (encrypted). TLS will be enabled in an updated example.

Registry

This was the easy part. Enable registry in the gitlab server config, and make a DNS entry pointing your registry to the gitlab server.

The example

The code is available here. Please note the security reservation above. I don’t consider this to be production ready.