pre.hn/site/posts/2019-04-14-drone-ci-docker-and-kubernetes.md

title
Setting up CI/CD for Docker and Kubernetes Using Drone

I have been a Travis CI user. However, Travis has gotten less reliable for me lately. On top of that, I have qualms about how the acquisition of Travis by Idera, and the subsequent layoffs, were handled. Travis is also a square peg in the octagonal hole of my Kubernetes environment. It is a hosted, external service. Everything else I use to develop my applications is hosted inside of my cluster. My Docker registry and my gitops operator run in my cluster. My databases are in my cluster. My storage provider and object store run in cluster. My apps run in cluster. Why would I run my CI/CD service outside of the cluster?

I came across Drone in my research about alternatives. Drone is a fully container-native, container-loving CI solution. It's Docker all the way down. Since my application is already "Dockerified", my hosting environment is all Docker all day, and my deployments are already in the form of a Docker push, why not do CI/CD in Docker as well?

Drone is basically a small framework for running CI jobs made of docker containers. You build your pipeline as a series of steps, each of which is a docker base image, some configuration, and your test commands. Drone also has all the standard integrations you would expect for a CI service-- it talks to GitHub, GitLab, Bitbucket, and more.

To use Drone, you'll have to embrace the Docker way. Let go of your test scripts that are building everything from a ubuntu image or a language version manager. Let go of the "special case magic" way DBs and other supporting services are handled in other CI platforms. In exchange, you'll find that Drone will let you use any language, any tools, and any languages, so long as they have a Docker image.

Drone Setup on Kubernetes

I followed this official (but experimental) guide for setting up Drone on Kubernetes.

Here's the configuration I used for Kubernetes:

---
apiVersion: v1
kind: Namespace
metadata:
  name: drone
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  labels:
    app: drone
  name: drone
  namespace: drone
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: drone
      name: drone
      namespace: drone
    spec:
      containers:
      - name: drone
        image: drone/drone:1.0.0
        env:
        - name: DRONE_KUBERNETES_ENABLED
          value: \"true\"
        - name: DRONE_KUBERNETES_NAMESPACE
          value: \"drone\"
        - name: DRONE_GITHUB_SERVER
          value: \"https://github.com\"
        - name: DRONE_GITHUB_CLIENT_ID
          value: \"REDACTED\"
        - name: DRONE_GITHUB_CLIENT_SECRET
          value: \"REDACTED\"
        - name: DRONE_RPC_SECRET
          value: \"REDACTED\"
        - name: DRONE_SERVER_HOST
          value: \"REDACTED.example.com\"
        - name: DRONE_SERVER_PROTO
          value: \"https\"
        - name: DRONE_USER_FILTER
          value: \"prehnRA\"
        - name: DRONE_USER_CREATE
          value: username:prehnRA,admin:true
        - name: DRONE_DATABASE_DRIVER
          value: postgres
        - name: DRONE_DATABASE_DATASOURCE
          valueFrom:
            secretKeyRef:
              name: drone-postgres-url
              key: url
        ports:
          - containerPort: 80
          - containerPort: 443
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: drone
  name: drone
  namespace: drone
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: drone
  type: LoadBalancer
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    ingress.kubernetes.io/ssl-redirect: \"true\"
    nginx.ingress.kubernetes.io/ssl-redirect: \"true\"
    kubernetes.io/tls-acme: \"true\" # enable certificates
    certmanager.k8s.io/cluster-issuer: letsencrypt
    kubernetes.io/ingress.class: \"nginx\"
  labels:
    app: drone
  name: drone
  namespace: drone
spec:
  rules:
  - host: REDACTED.example.com
    http:
      paths:
      - backend:
          serviceName: drone
          servicePort: 80
        path: /
  tls: # specify domains to fetch certificates for
  - hosts:
    - REDACTED.example.com
    secretName: drone-tls
---
apiVersion: kubedb.com/v1alpha1
kind: Postgres
metadata:
  name: drone-postgres
  namespace: drone
spec:
  version: \"10.2-v1\"
  storageType: Durable
  storage:
    storageClassName: \"rook-block\"
    accessModes:
    - ReadWriteOnce
    resources:
      requests:
        storage: 256Mi
  terminationPolicy: DoNotTerminate

Note that I use kubedb in my cluster, so I am able to request a new Postgres database through Kubernetes configuration YAML. If you don't use kubedb or similar, you'll have to provide a db to Drone differently. By default, Drone uses a sqlite3 database, but this isn't much good in Kubernetes by default, because if you Drone pods get restarted, you will lose your configuration and job history.

I give the Postgres url to Drone via a Kubernetes secret called drone-postgres-url in the url key. This secret must be in the drone namespace. Here's how you can do that:

echo -n 'YOUR_POSTGRES_URL' > ./url
kubectl create secret generic drone-postgres-url -n drone --from-file=./url

Testing and Deploying An App

The application I wanted to test and deploy through Drone is an Elixir app that is released by semantic-release as a Docker image. I've written previously on the Revelry blog about how to do that.

You configure Drone using a .drone.yml file. The main YAML object defined in that file is a "pipeline", which defines a series of steps and services which Drone will use to run your tests and deploy your app.

A basic pipeline for my app might look like this:

---
kind: pipeline
name: default

steps:
- name: backend
  image: elixir:1.8.0-alpine
  commands:
  - mix local.hex --force && mix local.rebar --force
  - export MIX_ENV=test
  - mix do deps.get, deps.compile, compile, phx.digest, ecto.create, ecto.migrate, test

There's a problem: my tests won't pass without a working database, and we don't have one yet. In Drone, the way to get a supporting container for something like a database or a cache is via a service. Services are also just a Docker container which runs with a certain configuration. Drone will run them before your steps. Other parts of your pipeline can communicate with services over a network (services are given a hostname that matches their service name) or via a shared volume.

Here's what the same pipeline looks like with a mariadb service:

---
kind: pipeline
name: default

steps:
- name: backend
  image: elixir:1.8.0-alpine
  commands:
  - mix local.hex --force && mix local.rebar --force
  - export MIX_ENV=test
  - mix do deps.get, deps.compile, compile, phx.digest, ecto.create, ecto.migrate, test

services:
- name: cms-database
  image: mariadb
  ports:
  - 3306
  environment:
    MYSQL_DATABASE: \"cms_test\"
    MYSQL_USER: \"REDACTED\"
    MYSQL_PASSWORD: \"REDACTED\"
    MYSQL_RANDOM_ROOT_PASSWORD: \"yes\"

I also configured my test suite to use a mariadb database at the host cms-database, port 3306, with the given username and password.

My tests pass!

Next, I need to add my deployment step. As I mentioned before, this app deploys as a docker image, via semantic-release and semantic-release-docker. In order to do that, I need to use a "docker in docker" image-- which is just what it says, a Docker image containing the Docker daemon and Docker CLI.

It's actually best to use two of these. Drone (and Docker) prefer if any long running services, such as the Docker daemon, run in their own containers. In my experience, trying to run the Docker daemon in the background of a container that is also doing other commands overcomplicates things. Running the Docker daemon in an isolated container works better, because Docker has provided out of the box initialization scripts for that scenario.

Here's the pipeline with both "dind" parts added:

---
kind: pipeline
name: default

steps:
- name: backend
  image: elixir:1.8.0-alpine
  commands:
  - mix local.hex --force && mix local.rebar --force
  - export MIX_ENV=test
  - mix do deps.get, deps.compile, compile, phx.digest, ecto.create, ecto.migrate, test
- name: deployment
  image: docker:dind
  volumes:
  - name: dockersock
    path: /var/run
  environment:
    GH_TOKEN:
      from_secret: gh-token
    DOCKER_USERNAME: ci
    DOCKER_PASSWORD:
      from_secret: docker-password
  commands:
  - apk add nodejs nodejs-npm openssl git
  - npm install -g npm
  - (cd assets; npm install; npm run deploy)
  - npm install
  - docker build --network=host . -t REDACTED.example.com/my_repo/my_app
  - npx semantic-release

services:
- name: cms-database
  image: mariadb
  ports:
  - 3306
  environment:
    MYSQL_DATABASE: \"cms_test\"
    MYSQL_USER: \"REDACTED\"
    MYSQL_PASSWORD: \"REDACTED\"
    MYSQL_RANDOM_ROOT_PASSWORD: \"yes\"
- name: docker
  image: docker:dind
  privileged: true
  network: host
  hostNetwork: true
  mtu: 1200
  volumes:
  - name: dockersock
    path: /var/run

volumes:
- name: dockersock
  temp: {}

That's a lot of new pieces. What's going on? Well, we've added a docker:dind step to the pipeline. This will run our Docker commands, for building and pushing the Docker image. We also need npm and NodeJS, because semantic-release is a Node package. We install npm and update it to the latest version for good measure. Then (cd assets; npm install; npm run deploy) builds my assets for production. npm install installs semantic-release and the various plugins I use (see my article from the Revelry blog). I build the Docker image. Then, I run semantic-release. Since I'm using semantic-release-docker, it will login to my Docker repo and push the image.

In order to actually build the image, we need a running Docker daemon. That's where the second dind comes in. This one is a service. It runs in privileged mode (required for the dind daemon), which means my Drone project must be flagged as "trusted." We use a shared volume to allow the Docker CLI running in my deployment step to communicate with the Docker daemon running as a service.

I need to provide some credentials for GitHub and for my Docker registry. I do that by exposing them as environment variables. Since I don't want to check in a sensitive credential to git, I use Drone secrets for passwords and tokens. You can set the value of the secret in the project settings UI.

When you put it all together, Drone will do the following sequence:

• Start a mariadb database and expose it on a network
• Start a docker and expose it on a shared volume
• Prepare and run my test suite in an Elixir docker image
• If my tests pass, move on to a deployment phase:
  • Build my assets
  • Build a Docker image containing my latest code
  • Tag the image based on the proper semantic version
  • Push that tagged version and latest to my Docker repo
  • Also, tag that release in GitHub

Other Drone Features

I had a lightbulb moment while I was working with Drone. I had been wondering things like "I wonder which DBs Drone supports" or "I wonder which languages Drone supports." Then I realized: Drone supports everything that has a Docker image. To me, this makes it a more flexible and powerful tool than a CI architected like Travis or Codeship. In those CIs, you either have to wait for official support for your language (or service), or you have to hack together a working test script from a base intended for a different language or DB. The way Drone does this means that Drone immediately has a "feature list" longer than I can write here.

Beyond this, there are other Drone features worth mentioning:

• Drone supports Agents, which allow you to scale up your CI environment to handle more simultaneous builds. They are small daemons which you deploy to as many servers as you want. Each agent receives orders from the central Drone server which dispatches builds.
• Drone supports multiple secrets plugins. In my example, I used secrets stored in Drone's backing DB, but Drone also supports Vault and Kubernetes secrets, among others.
• In addition to GitHub, Drone supports GitLab, Gitea, gogs, and Bitbucket.
• drone exec is a cool feature on the Drone CLI which lets you run a build locally on your laptop, using the same configuration.

The only real issue that I hit along the way is that I could not get Drone to accept an encrypted secret from the .drone.yml file. Supposedly, Drone supports encrypting secrets via the CLI and including the encrypted version in your .drone.yml. I could not get that to work. I had to use the database secrets method instead.

The Future of Drone

After experimenting with Drone, I think we're going to see it get a lot of traction. It's such a powerful tool, and it so well leverages the Docker ecosystem, that I can't help but think that it is the future of CI.','Setting up CI/CD for Docker and Kubernetes Using Drone | Robert Prehn','Drone is a CI/CD framework that is all in on Docker. In this article, I will describe how I switched my application off of Travis CI, and onto a container-native CI using Drone, Docker, and Kubernetes.