GitLab CI

16 September, 2023 - Tags: gitlab, ci

GitLab CI

In all the CI providers we want do automate something for example, we want to run the tests on every commit or we want to check the lints whenever we update markdown files or we want to make sure the code compiles when we make changes in the code.

These all can be automated using different CI providers, generally all of them have their own way of defining how you'll write the automation logic so that their software can understand it. The general thought process behind running the CI remains the same that you want to automate something.

From a security perspective, it might be the case that people trust isolated runners more than your laptop. I'll consume an executable more easily that's built via github releases or gitlab pipelines than a tarball that's pushed by a developer via his laptop. It's good to have an automation that'll release things and no doubt, it's fast as well.

Now all the provider claims to be fast and offer unique selling points which are important if you are building something related to CI or working or an organisation that sells CI based products but in general for most of us who consume CI the basics remains the same.

In this blog, I'm going to understand all the basic concepts that are out there in gitlab CI and write about that.

jobs

run_tests:
  image: alpine
  before_script:
    - sudo apt install jq
  script:
    - kubectl get pods -o json | jq '.'
build_image:

The above are two jobs configured for gitlab.

variables and secrets

build_image:
  variables:
    DOCKER_IMAGE: "ttl.sh/$(uuidgen)"
  script:
    - docker build -t $DOCKER_IMAGE .

variables:
  DOCKER_IMAGE: "ttl.sh/something/nginx"
  DOCKER_TAG: "v1"
build_image:
  docker build -t $DOCKER_IMAGE:$DOCKER_TAG .
push_image:
  docker push $DOCKER_IMAGE:$DOCKER_TAG

docker-in-docker scenario

To replicate docker in docker scenario in gitlab use the following syntax

some_random_job_name:
  image: docker:20:10:16
  services:
    - docker:20:10:16-dind
  variables:
    DOCKER_TLS_CERTDIR: "/certs"

The services are like an adjacent container that are connected to the main container. If you want a database then you can start a service container and both will be linked at build and test time for your application. The variable DOCKER_TLS_CERTDIR is required so that both docker client and server can communicate with each other.

stages

Stages are something that we use to group together a number of jobs and run them in sequential order as per our requirement. Say you write three jobs for a simple application.

  1. build the container image
  2. test the container image
  3. sign the image
  4. push the container image

This is a very generic scenario that we deal with on a day to day basis. We build our image and test it in our pipeline and then if all succeeds then we sign and push the image to the registry.

Now if you define all of this as separate jobs in the following manner:

build_job:
  # build something 
test_job:
  # test something
sign_job:
  # sign build artifacts
push_job:
  # push artifacts

Now when you push the changes and all the four jobs will start running immediately and at least few of them will crash for sure because the image itself if not built yet. How can you push something that's not available as of now? You can't.

That's where stages come in.

We define stages and control the order in which different jobs will execute.

To add a stage to a job you use the following syntax.

goodbye:
  stage: last

The above job will run when the last stage will be reached. So, if you've the following stages:

stages:
  - build
  - deploy
  - last

Think for a moment, when will this goodbye job execute? The answer is after build and deploy jobs are completed. Yes, all the jobs under last stage will run in parallel.

pipeline

Everything that's inside .gitlab-ci.yml combined together is called a pipeline. So, in general a pipeline is a collection of stages and jobs.

A pipeline is composed of independent jobs that run scripts, grouped into stages. Stages run in sequential order, but jobs within stages run in parallel.

The above statement is from gitlab pipeline automated template. The above describe the difference between jobs, stages and pipeline in a beautiful manner.

deploying on a Linux host

The syntax will look like this:

deploy:
  stage: deploy
  script:
    - ssh -o StrictHostKeyChecking=no -i $SSH_KEY <username>@<ip-address>

templates

Similar to GitHub actions marketplace, GitLab have templates that you can import within your projects. Some of them are maintained and offered by GitLab and some of them are community developed. To include a template in .gitlab-ci.yml use the following syntax.

job:
  stage: last
include:
- template: <some-remote-reference-to-.gitlab-ci.

conclusion

I'm feeling much better after understanding and reading .gitlab-ci.yml files. I might write a second part of the blog if I find something and advanced or something that ends up wasting a lot of my time. If you're getting started then doing some hands on practice in pipeline editor will help. It's very helpful because you'll be able to visualize all your jobs and stages in the UI right next to the editor which is very helpful.

Reference