Just a short post here:

In my previous post I set up this nice and shiny blog using hugo, you can find it here. In that post I set up a quick gitlab runner config to build and deploy my docker images to my Kubernetes cluster. There was a problem with that however, I was using a Docker-in-Docker (dind) image with TLS enabled. Which is not an ideal workflow, as not only does it require extra config on the runners themselves, and a dedicated docker host, It also requires the build container to run in a privileged execution mode - which creates a large amount of unnecessary security holes, by basically disabling all security mechanisms of containers, which can lead to a rather terrifying vulnerability known as host privilege escalation. To summarise this is really bad and could allow a rouge process running in a privileged container, to breakout and execute its own malicious code on the host system.

So in this post, I’m going to fix that - with the gitlab runner for Kubernetes and Kaniko

Installation

Setting up the runner was simple using the helm chart, another nice bit of tech I will do a post about… once I’ve understood it fully myself. Anyway, following the installation instructions for the Kubernetes executor is easy, all that’s needed is helm and the following commands:

# Add the repo
helm repo add gitlab https://charts.gitlab.io
# install gitlab-runner helm chart
helm install --namespace <NAMESPACE> gitlab-runner -f <CONFIG_VALUES_FILE> gitlab/gitlab-runner

I also created a ‘gitlab’ namespace before this through kubectl.

The config values file I passed in looks like this, there are a lot more configurations available but for a basic config this is all that’s needed. Note the use of rbac create as by default roles and service accounts will not be created, this can be omitted if creating the service accounts manually.

gitlabUrl: https://gitlab.com/ # as with all runners, can be a custom instance
runnerRegistrationToken: <TOKEN>
rbac:
  create: true

And that’s it, Gitlab runner is on Kubernetes. Each job will create its own container to execute CI/CD jobs. the number of concurrent jobs can be configured in the values.yml file above. By default, this is 10 which should be plenty for most people and even small teams.

Runner config and Kaniko

Kaniko is a container build tool that’s a part of Google’s container tools, there’s a nice disclaimer at the top though that reads:

NOTE: kaniko is not an officially supported Google product

Presumably this is to keep the lawyers happy as from what I can see, Kaniko started as an internal Google tool and is now fully open source for all to use.

Building on the back of my previous post, all that needs changing is the docker build stage. The main bonus here for Kaniko is that it doesn’t require any elevated security context. So that’s no TLS enabled docker engines and no privileged execution, this means it can reliably run in a Kubernetes cluster with other apps, and not risk container breakout.

Here’s that new docker build stage:

dockerbuild:
  stage: dockerbuild
  tags:
    - k8s-executor
  variables:
    GIT_SUBMODULE_STRATEGY: recursive
  image:
    name: gcr.io/kaniko-project/executor:debug
    entrypoint: [""]
  script:
    - mkdir -p /kaniko/.docker
    - echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
    - /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --destination $CI_REGISTRY_IMAGE:latest

This was taken from the nice and helpful gitlab docs on using kaniko with a gitlab runner, available here It’s a nice and simple process; Create a config.json for Kaniko which includes all the registry info and credentials, then call the Kaniko executor to set the build going and push up to the repo all in one command - neat!

Wrapping up

So a short one really, I didn’t want to edit the previous post, as it will still come in useful for those who want a more hands-on build process. Even without a kubernetes cluster, Kaniko images can be used with the gitlab docker executor to avoid the use of privileged execution - something that should be avoided where possible.