Translated using DeepL

Machine-translated page for increased accessibility for English questioners.

GitLab FI

GitLab Continuous Integration

GitLab Continuous Integration (CI) is used to automate some development tasks in a repository, most commonly for automated unit testing. You can configure your own physical or virtual machine to use GitLab CI (see Stratus.FI virtualization). In addition, you can also use the faculty machine gitlab-ci.fi.muni.cz.


Summary

The faculty gitlab-ci.fi uses the official GitLab Runner with container isolation in Docker. When a new task is started (e.g., following git push into the repository), the GitLab Runner asks Docker to create a new container from the image that is declared in the repository in the file .gitlab-ci.yml. The container clones the repository and runs the tasks described in that file. When finished, the container is dropped and the result is returned to GitLab, which displays it in the CI/CD section.


Configuring the project for gitlab-ci.fi.muni.cz

First, read the introductory information for using GitLab CI/CD. Next, you will need the documentation for .gitlab-ci.yml.

Selecting an image

The image to be used is declared in the .gitlab-ci.yml configuration as the value of the image key. The format is either REPOSITORY:TAG or REPOSITORY (the default tag is then latest). If no image is specified, alpine:latest is used.

image: maven:latest

Selecting the version

Docker image tags are not static, i.e., X:3.0 is just a symbolic name for a version of an image, and it may happen at any time that the repository maintainer changes the image to which the tag points. The versioning of images and the meaning of the versions themselves depends on the maintainers of the individual Docker repositories, but in general it is advisable to follow the principles of semantic versioning.

If possible, prefer images in the most generic major version (e.g., prefer X:3 instead of X:3.5.0) so that your project has an image with security patches and bug fixes for the software used.

However, we do not recommend using latest for important projects. This symbolic marker usually points to the latest stable version of the image, but it may move to a newer version that is not backwards compatible without warning.

Marker settings

To prevent the machine from being overwhelmed by tasks from repositories that have their own CIs set, gitlab-ci.fi only accepts tasks from projects that are tagged with the shared-fi tag , which can be set as follows:

  • in SettingsGeneralPermissions, enable the Pipelines option if it is not already enabled
  • in the .gitlab-ci.yml settings, add the shared-fi tag to each target, e.g.:
    build:
      tags:
        - shared-fi

Artifact settings

For projects that create artifacts, we recommend setting CIs so that GitLab automatically cleans them up when newer ones are created.

If you don't set up scavenging and the project artifacts are consuming GitLab's disk space, they will be deleted by the administrators.

First, ensure that GitLab preserves the most recent artifact in the project:
ProjectSettingsCI/CDArtifacts→ check Keep artifacts from most recent successful jobs

Then add a setting to .gitlab-ci.yml that sets the artifact lifetime to a very small value (less than 2 hours, e.g. 10 minutes). Set this for each job JOB.
With the setting above, the last artifact will be preserved after the lifetime expires.

‹JOB›:
  artifacts:
    …
    expire_in: 10 minutes

Examples

In the FI faculty GitLab you can take a look at the project unix/ci-examples for examples of CI configuration for simple projects.


Container Registry

The Container Registry service gives users the ability to save images to a Docker project that can then be used in CI or other projects.

The images do not need to be related to the project content in any way. However, you will likely have Dockerfile and other dependencies for the image, so we recommend creating a repository for these files that also keeps the image version up to date.

The Container Registry can be accessed from the gitlab.fi.muni.czmachine and the 5050 port.

Service Settings

  • Switching on the service

    In the project that is to maintain the build images, turn on
    SettingsGeneralVisibility, project features, permissionsContainer Registry.
    The service does not need to be turned on for projects that are only going to use the image in CI.
  • Limit on the number of tags

    Images in the Container Registry usually take up a lot of space. Frequently changing tags can quickly exhaust disk space, so turn on automatic cleanup for the project:
    In SettingsPackages & Registries, turn on Clean up image tags. We also recommend changing the Keep the most recent: setting to 5 tags per image name.
  • Access to the image

    Access to images is generally governed by the access rights of the parent project:
    • Private - Project members only
    • Internal - Only people logged into GitLab FI
    • Public - Unrestricted
    Additionally, access can be restricted by changing the setting above from Everyone with access to Only project members.

Creating an image

Create the image locally in your own Docker instance (commands below). You can also set up automatic image builds using CI, but you must set up your own CI Runner for it for security reasons. The name of the image to be placed in the GitLab Container Registry must start with the domain and port in the format gitlab.fi.muni.cz:5050, and continue with the project path.

So, for example, for a project https://gitlab.fi.muni.cz/NAMESPACE/PROJECT.git, you can create images with names of the form

  • gitlab.fi.muni.cz:5050/NAMESPACE/PROJECT:TAG
  • gitlab.fi.muni.cz:5050/NAMESPACE/PROJECT/IMAGE:TAG
  • gitlab.fi.muni.cz:5050/NAMESPACE/PROJECT/NAME/IMAGE:TAG

If you are happy with the image, you can upload it to the Container Registry:

    docker login --username ‹LOGIN› gitlab.fi.muni.cz:5050
    docker push gitlab.fi.muni.cz:5050/NAMESPACE/...

In the first command, use your faculty login instead of ‹LOGIN›. The command will ask for a password when it runs; enter your faculty password or GitLab access token (it must have at least the scopes read_registry and write_registry, even though the documentation says only the latter is sufficient - this is probably a bug).

For an example, you can look at the Makefilefile in the https://gitlab.fi.muni.cz/xlacko1/pb173-perl-image repository .

Use in GitLab CI

The new image can be used both in your own Docker instance and in GitLab CI/CD jobs. Just include the full path to the custom image in the image:settings.

For example, if we want to create an image named perl-5.32:1.0 in the https://gitlab.fi.muni.cz/xlacko1/pb173-perl-image repository, then we execute the commands for that image in the Dockerfile directory:

    $ docker build -t gitlab.fi.muni.cz:5050/xlacko1/pb173-perl-image/perl-5.32:1.0 .
    $ docker login --username LOGIN gitlab.fi.muni.cz:5050
    $ docker push gitlab.fi.muni.cz:5050/xlacko1/pb173-perl-image/perl-5.32:1.0

In the project where we want to use the created image for CI, declare .gitlab-ci.yml in the file:

image: https://gitlab.fi.muni.cz/xlacko1/pb173-perl-image:1.0

# The 'shared-fi' tag is necessary to use the faculty-wide CI
default:
  tags:
    - shared-fi

Automatic image build

GitLab CI can be used to automatically build a Docker image and upload it to the Container Registry, typically using the docker:dind (Docker-in-Docker) image.

However, this operation requires enabling privileged mode for the job container to allow the container to issue commands to the Docker daemon. However, it also allows the container to bypass the system's security mechanisms and take control. Therefore, this option is disabled on the shared GitLab CI.

If you want to build images automatically, you need to set up your own CI Runner. We recommend using the virtual machine in Stratus.FI. Alternatively, you can run the service on your own machine, but keep in mind that doing so may open up space for a Privilege Escalation attack.

  • Virtual machine or computer?

    For this task, we strongly recommend using a virtual machine in Stratus.FI. You should be able to use the pre-installed virtual machine.

    Using your own machine is also possible. Note, however, that there is a risk of privilege escalation or taking control of the machine in case of a configuration error. The same risk applies to a virtual machine, but the extent of the problem is smaller in this case.
  • GitLab CI Runner

    Install GitLab Runner (→ Install on GNU/Linux for Linux).
  • Docker and Docker

    Configure CI Runner to build Docker images according to the official instructions. We recommend the Docker-in-Docker option.
  • Registration

    Register the Runner only for the project in which the images are to be built, see Project Runners.
  • Project security

    Make sure that only trusted users in the project (Owner, Maintainer) can run the CI job. In particular, prevent untrusted users from being able to create a Merge Request with custom code for which to run the job.

    See Protected Branches and Rules for .gitlab-ci.yml.
  • Location of .gitlab-ci.yml

    A potential attacker can modify .gitlab-ci.yml at will and bypass the settings above. This is a serious problem especially for public and internal projects.

    This problem can be solved by setting the project to search for .gitlab-ci.yml in another private repository where only trusted users can modify it. The local file will then be ignored. See Custom CI/CD configuration fileCustom CI/CD configuration file examples.

    Note that setting CODEOWNERS is not enough, it will only prevent an unwanted Merge if the protected file has changed, but the CI tasks will still run.

Finally, consider whether automatic image builds are really necessary. If you rarely build an image, the safest option is to build the image in a custom Docker instance as described above.


Common problems and solutions

Stuck tasks

If you encounter a Job is stuck error after configuring your project, you probably did not include the shared-fi tag in the job configuration. Check your settings as described above.

Cannot use Docker commands in a job

If you encounter an error like dial tcp: lookup docker on 147.251.48.14:53: no such host, you are probably trying to use the Faculty CI Runner in privileged mode. This is not allowed for security reasons, see Automatic image build above.