Automate VMware Templates with Packer and GitLab (Part 2)

As we now installed our GitLab Server we can now continue to setup our GitLab Runner and initialize our Packer workflow.

In Part of this series, we will install another Linux VM, install Docker and prepare our GitLab Runner to execute CI/CD pipelines from our GitLab Repository.

Blogseries Overview

Part 1: Installation of GitLab CE
Part 2: Configuring GitLab Runner with Docker executor
Part 3: Configuring CI Pipeline with Packer Templates

Installation of Linux VM & Docker

To automate our VMware templates, we installed and configured our GitLab CE instance. In the next step we will configure our Docker host and connect GitLab-Runner to our GitLab instance.

Install Linux VM

To build our Docker host, we need another Linux host, like Ubuntu or Debian.

Hardware specifications of that server are a bit higher as it will (eventually) run multiple pipelines at the same time:

4 vCPU
8GB RAM
60GB vDisk

Important: Don’t install the GitLab Runner on the GitLab CE server.

Install & configure Docker

After following the installation wizard, we can now configure the Linux Server as our Docker host.

First, we will install the prerequisites of Docker:

sudo apt update && apt upgrade -y

sudo apt install -y apt-transport-https ca-certificates curl software-properties-common

After installing all prerequisites, we also need to add the GPG key from docker to our config files:

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

And also add the Docker Repository to our APT sources:

echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Now we need to update our repository cache to get all recognized:

sudo apt update

We also need to make sure that APT uses the Docker repository instead of the default Debian/Ubuntu repository:

apt-cache policy docker-ce

Now, as everything is configured, we can finally install Docker… :

sudo apt install docker-ce

…and Docker Compose:

mkdir -p ~/.docker/cli-plugins/

curl -SL https://github.com/docker/compose/releases/download/v2.3.3/docker-compose-linux-x86_64 -o ~/.docker/cli-plugins/docker-compose

chmod +x ~/.docker/cli-plugins/docker-compose

To make sure, everything works fine, we can validate Docker as well as Compose with the following commands:

Docker

Input:
sudo systemctl status docker

Output:
docker.service - Docker Application Container Engine
     Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2023-04-12 19:21:25 UTC; 22s ago
TriggeredBy: ● docker.socket
       Docs: https://docs.docker.com
   Main PID: 7854 (dockerd)
      Tasks: 7
     Memory: 38.3M
        CPU: 340ms
     CGroup: /system.slice/docker.service
             └─7854 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

Docker Compose

Input:
docker compose version

Output:
Docker Compose version v2.3.3

Install & configure GitLab Runner

We now successfully prepared our host to run docker containers to execute Packer commands. The next step is to connect our GitLab Runner to our GitLab Server.

Install

Following commands need to be entered to accomplish this:

curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash

Now we can update our repository cache as well as install GitLab Runner:

sudo apt update

sudo install gitlab-runner

Register to GitLab Server

The GitLab Runner is now installed and running. We can now connect it to our existing GitLab CE instance and make it available to run CI/CD pipelines.

To do this, we’re doing following steps:

Obtain a token from GitLab

To obtain a registration token from GitLab, we navigate to following settings page:

Login as root to GitLab CE
Go to the following menu:
- Admin
- CI/CD
- Runners
Click on “Register an instance runner”

Obtain TLS certificate

To eliminate errors with TLS certificates, we will first configure a trust to the server certificate of our GitLab server:

openssl s_client -showcerts -connect gitlab.example.com:443 -servername gitlab.example.com < /dev/null 2>/dev/null | openssl x509 -outform PEM > /etc/gitlab-runner/certs/gitlab.example.com.crt

If you have configured an certificate from your internal PKI you can also place the crt file with the following format in the same directory:

  -----BEGIN CERTIFICATE-----
  (Your primary SSL certificate: your_domain_name.crt)
  -----END CERTIFICATE-----
  -----BEGIN CERTIFICATE-----
  (Your intermediate certificate)
  -----END CERTIFICATE-----
  -----BEGIN CERTIFICATE-----
  (Your root certificate)
  -----END CERTIFICATE-----

For more information see GitLab Docs.

Runner registration

Now we obtained our registration token (hint: registrations tokens are one-time use) and eliminated TLS errors, we can finally register our runner.

With the following command we register our Runner to our GitLab server:

sudo gitlab-runner register --url "https://gitlab.com/" --registration-token "REGISTRATION_TOKEN" --executor "docker" --description "$HOSTNAME$" --non-interactive

Wrap Up

To summarize our work for this part:

Installed another Linux VM
Installed docker-ce & docker compose
Installed & configured GitLab-Runner
Connected GitLab-Runner to our GitLab-CE instance

We have now prepared our GitLab-Runner to run Docker containers – ready for the last Part of this series.

In the next part, we will setup our code repository and configure the CI/CD pipeline to automatically create vSphere templates!

Kevin Bertrand