GitLab FI

GitLab Continuous Integration

GitLab Continuous Integration (CI) is used to automate some development tasks in a repository, most commonly for automated unit testing. You can configure your own physical or virtual machine to use GitLab CI (see Stratus.FI virtualization). In addition, you can also use the faculty machine gitlab-ci.fi.muni.cz.

Summary

The faculty gitlab-ci.fi uses the official GitLab Runner with container isolation in Docker. When a new task is started (e.g., following git push into the repository), the GitLab Runner asks Docker to create a new container from the image that is declared in the repository in the file .gitlab-ci.yml. The container clones the repository and runs the tasks described in that file. When finished, the container is dropped and the result is returned to GitLab, which displays it in the CI/CD section.

Configuring the project for gitlab-ci.fi.muni.cz

First, read the introductory information for using GitLab CI/CD. Next, you will need the documentation for .gitlab-ci.yml.

Selecting an image

The image to be used is declared in the .gitlab-ci.yml configuration as the value of the image key. The format is either REPOSITORY:TAG or REPOSITORY (the default tag is then latest). If no image is specified, alpine:latest is used.

image: maven:latest

Selecting the version

Docker image tags are not static, i.e., X:3.0 is just a symbolic name for a version of an image, and it may happen at any time that the repository maintainer changes the image to which the tag points. The versioning of images and the meaning of the versions themselves depends on the maintainers of the individual Docker repositories, but in general it is advisable to follow the principles of semantic versioning.

If possible, prefer images in the most generic major version (e.g., prefer X:3 instead of X:3.5.0) so that your project has an image with security patches and bug fixes for the software used.

However, we do not recommend using latest for important projects. This symbolic marker usually points to the latest stable version of the image, but it may move to a newer version that is not backwards compatible without warning.

Marker settings

To prevent the machine from being overwhelmed by tasks from repositories that have their own CIs set, gitlab-ci.fi only accepts tasks from projects that are tagged with the shared-fi tag , which can be set as follows:

• in Settings → General → Permissions, enable the Pipelines option if it is not already enabled
• in the .gitlab-ci.yml settings, add the shared-fi tag to each target, e.g.:

  build:
    tags:
      - shared-fi

Artifact settings

For projects that create artifacts, we recommend setting CIs so that GitLab automatically cleans them up when newer ones are created.

First, ensure that GitLab preserves the most recent artifact in the project:
Project → Settings → CI/CD → Artifacts→ check Keep artifacts from most recent successful jobs

Then add a setting to .gitlab-ci.yml that sets the artifact lifetime to a very small value (less than 2 hours, e.g. 10 minutes). Set this for each job JOB.
With the setting above, the last artifact will be preserved after the lifetime expires.

‹JOB›:
  artifacts:
    …
    expire_in: 10 minutes

Examples

In the FI faculty GitLab you can take a look at the project unix/ci-examples for examples of CI configuration for simple projects.

Container Registry

The Container Registry service gives users the ability to save images to a Docker project that can then be used in CI or other projects.

The images do not need to be related to the project content in any way. However, you will likely have Dockerfile and other dependencies for the image, so we recommend creating a repository for these files that also keeps the image version up to date.

The Container Registry can be accessed from the gitlab.fi.muni.czmachine and the 5050 port.

Service Settings

Creating an image

Create the image locally in your own Docker instance (commands below). You can also set up automatic image builds using CI, but you must set up your own CI Runner for it for security reasons. The name of the image to be placed in the GitLab Container Registry must start with the domain and port in the format gitlab.fi.muni.cz:5050, and continue with the project path.

So, for example, for a project https://gitlab.fi.muni.cz/NAMESPACE/PROJECT.git, you can create images with names of the form

• gitlab.fi.muni.cz:5050/NAMESPACE/PROJECT:TAG
• gitlab.fi.muni.cz:5050/NAMESPACE/PROJECT/IMAGE:TAG
• gitlab.fi.muni.cz:5050/NAMESPACE/PROJECT/NAME/IMAGE:TAG

If you are happy with the image, you can upload it to the Container Registry:

    docker login --username ‹LOGIN› gitlab.fi.muni.cz:5050
    docker push gitlab.fi.muni.cz:5050/NAMESPACE/...

In the first command, use your faculty login instead of ‹LOGIN›. The command will ask for a password when it runs; enter your faculty password or GitLab access token (it must have at least the scopes read_registry and write_registry, even though the documentation says only the latter is sufficient - this is probably a bug).

For an example, you can look at the Makefilefile in the https://gitlab.fi.muni.cz/xlacko1/pb173-perl-image repository .

Use in GitLab CI

The new image can be used both in your own Docker instance and in GitLab CI/CD jobs. Just include the full path to the custom image in the image:settings.

For example, if we want to create an image named perl-5.32:1.0 in the https://gitlab.fi.muni.cz/xlacko1/pb173-perl-image repository, then we execute the commands for that image in the Dockerfile directory:

    $ docker build -t gitlab.fi.muni.cz:5050/xlacko1/pb173-perl-image/perl-5.32:1.0 .
    $ docker login --username LOGIN gitlab.fi.muni.cz:5050
    $ docker push gitlab.fi.muni.cz:5050/xlacko1/pb173-perl-image/perl-5.32:1.0

In the project where we want to use the created image for CI, declare .gitlab-ci.yml in the file:

image: https://gitlab.fi.muni.cz/xlacko1/pb173-perl-image:1.0

# The 'shared-fi' tag is necessary to use the faculty-wide CI
default:
  tags:
    - shared-fi

Automatic image build

GitLab CI can be used to automatically build a Docker image and upload it to the Container Registry, typically using the docker:dind (Docker-in-Docker) image.

However, this operation requires enabling privileged mode for the job container to allow the container to issue commands to the Docker daemon. However, it also allows the container to bypass the system's security mechanisms and take control. Therefore, this option is disabled on the shared GitLab CI.

If you want to build images automatically, you need to set up your own CI Runner. We recommend using the virtual machine in Stratus.FI. Alternatively, you can run the service on your own machine, but keep in mind that doing so may open up space for a Privilege Escalation attack.

Finally, consider whether automatic image builds are really necessary. If you rarely build an image, the safest option is to build the image in a custom Docker instance as described above.

Common problems and solutions

Stuck tasks

If you encounter a Job is stuck error after configuring your project, you probably did not include the shared-fi tag in the job configuration. Check your settings as described above.

Cannot use Docker commands in a job

If you encounter an error like dial tcp: lookup docker on 147.251.48.14:53: no such host, you are probably trying to use the Faculty CI Runner in privileged mode. This is not allowed for security reasons, see Automatic image build above.