Usage¶

General¶

The general syntax is as following: sqa [OPTIONS] COMMAND [ARGS]. It will exit with status code 1, if a service encountered an issue and exit with status code 0 otherwise.

GitLab CI/CD¶

The following manual describes how to use the Software Quality Assurance with GitLab CI/CD. It is highly advisable to know how to use git and GitLab when using this method.

Note: It is possible to hook the Software Quality Assurance to GitHub Actions and other code hosting platforms.

Decide on way to retrieve the report¶

There are two different ways to get the generated report by the Software Quality Assurance.

Artifact

When you opt for this option, the report is available via the GitLab web-interface. You can simply download. More information available here This option is easier to set up, but services, that make changes to your files are not covered this way (by the documentation).

Commit (recommended)

This option will commit the resulting report (and any other changes that where made) directly back to the branch the pipeline ran for.

Project member permissions¶

To apply certain changes described below, you need at least Maintainer permissions.

Host the project on any GitLab¶

How to do that, is described here

Enable CI/CD¶

In the GitLab web-interface from the page of your repository navigate to Settings > General > Visibility, project features, permissions. Enable Pipelines and set them to Only Project Members.

Add GitLab Runner¶

To have an available Runner is required for the Software Quality Assurance to function. This will not be covered here. Talk to your local admin get one up and running.

Runner requirements

executor: docker

WARNING: Be aware that the runtime for the services can go up to several hours, depending on the project size!

Further information are available here.

Retrieve report by Artifact¶

Set up .gitlab-ci.yml¶

This yaml block is set up for project without prior GitLab Ci/CD setup. If you already have .gitlab-ci.yml in your project, there should be someone withing your project who can help you adapt.

Copy the content of the following code block to a file named .gitlab-ci.yml in the base directory of your project. Commit and push your changes.

stages:
  - sqa

sqa:
  stage: sqa
  image:
    name: "git.gfz-potsdam.de:5000/id2/software/services/fair/software-quality-assurance/software-quality-assurance:latest"
    entrypoint: [""]
  script:
    - sqa --repository_dir "$CI_PROJECT_DIR" run
  only:
    refs:
      - master
  artifacts:
    paths:
      - QUALITY.md
    expire_in: 30 days

Retrieve report by commit¶

This manual focuses on UNIX systems like Linux or MacOS.

Note: for Windows the commands can slightly differ, but is fully compatible.

Set up deploy key¶

First we need to generate a key pair via the terminal. It will function as deploy key. More information on generating a key pair can be found here.

ssh-keygen -t rsa -b 4096 -C "[PROJECT_NAME]_software-quality-assurance" -f [OUTPUTFILE] -q -N ""

In the GitLab web-interface from the page of your repository navigate to Settings > Repository > Deploy Keys. Paste the content of [OUTPUTFILE].pub to the field key and give it any unique name (like PROJECT_NAME_deploy-key). Check the ‘Write access allowed’ checkbox. Save the changes.

Note: it is recommended to either store [OUTPUTFILE].pub securely or delete it.

Set up Variables¶

We need to tell our CI/CD the identity of the GitLab server we are using. To do that, we request it from the server via a terminal command. It returns two lines with the public identity of the GitLab server.

ssh-keyscan -t rsa [GITLAB_DOMAIN]]

# example

ssh-keyscan -t rsa git.gfz-potsdam.de
# git.gfz-potsdam.de:22 SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.3
git.gfz-potsdam.de ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsiq5txhABcG9oePf47SCd+w/tfSs6NWKaMRd2BfBi9DGORQX4wwV4CYvSJbMfJvQO5SMGwhnEAXe0bWihC2V8X7lzcePrpjfP+uH2lmipcFbV9g3iQpM7Fusr96IV65v/qQ6HE4+KkHJLR1vEKj1AOaVgDww7CHhVAHCkvqcSwwtfOjBbqyfQ2Su7O6UNzs1ecQcIXnQNO8ebzimv3c8mKdo9j8i5eiWVqzRivmi/R3XIIc2T695mTLyxUplqfyvZpDL90F9VEjr/3HM9/3dASbonaTH8SoMRCxOy8Fpz0Va1MdpSor0uoemQjbu8aZzGB0+tsE5eWjuFaZib8QDF

Now open the GitLab web-interface and navigate to Settings > CI/CD > Variables from from the page of your repository. We need to fill in two variables here. Click on ‘Add Variable’.

As key type in GITLAB_DEPLOY_KEY. As value paste the content of [OUTPUTFILE] (was generated in 5.2.1). Click on ‘Add Variable’.

For the second one: As key type in SSH_KNOWN_HOSTS_GITLAB. As value paste both output lines of the ssh-keyscan command (see above). Click on ‘Add Variable’.

Set up .gitlab-ci.yml¶

This yaml block is set up for project without prior GitLab Ci/CD setup. If you already have .gitlab-ci.yml in your project, there should be someone withing your project who can help you adapt.

Copy the content of the following code block to a file named .gitlab-ci.yml in the base directory of your project. Commit and push your changes.

stages:
  - sqa

sqa:
  stage: sqa
  image:
    name: "git.gfz-potsdam.de:5000/id2/software/services/fair/software-quality-assurance/software-quality-assurance:latest"
    entrypoint: [""]
  before_script:
    - mkdir -p ~/.ssh
    - chmod 700 ~/.ssh
    - echo "$SSH_KNOWN_HOSTS_GITLAB" > ~/.ssh/known_hosts
    - chmod 644 ~/.ssh/known_hosts
  script:
    - eval $(ssh-agent -s)
    - echo "$GITLAB_DEPLOY_KEY" | tr -d '\r' | ssh-add -
    - git config user.name "GitLab CI:${CI_PROJECT_NAMESPACE}/${CI_PROJECT_NAME}"
    - git config user.email "[PROJECT_NAME]_software-quality-assurance"
    - sqa --repository_dir "$CI_PROJECT_DIR" run
    - git add .
    - git commit -m "[AUTOMATIC] generated meta/software-quality-assurance-report.md by gitlab ci"
    - git push -o ci.skip "git@${CI_SERVER_HOST}:${CI_PROJECT_NAMESPACE}/${CI_PROJECT_NAME}.git" "HEAD:${CI_COMMIT_REF_NAME}"
  only:
    refs:
      - master

Important In the yaml file above fill in the line - git config user.email "[PROJECT_NAME]_software-quality-assurance" The content of user.email must be identical with what was entered in 5.2.1)

Note: it is recommended to either store [OUTPUTFILE] securely or delete it.

Examples¶

An example of both variants applied at the same time can be found here

Local¶

The following manual describes how to use the Software Quality Assurance on a local machine. This manual focuses on UNIX systems like Linux or MacOS.

Note: for Windows the commands can slightly differ, but is fully compatible.

Assure project is a git repository¶

To run the services on your local machine the first thing to do is to assure, that your software is within a git repository with a remote location. If you know, it is a git repository, you can skip this part.

Open a Terminal and change to the project directory. And then check for git.

cd [PATH_TO_PROJECT]
git remote get-url origin

If you see fatal: not a git repository you don’t have a git repository. Please make sure it becomes one. More information on doing that is available here.

If you see fatal: No such remote 'origin' you have a git repository, but there is no remote location named ‘origin’. Please make sure it has one. More information on doing that is available here.

Execute Software Quality Assurance¶

via Docker¶

Start the docker service:

systemctl start docker.service

The general syntax looks like the following:

docker run -v [/ABSOLUTE/PATH/TO/REPO]:/repo \
           -u $(id -u ${USER}):$(id -g ${USER}) \
           git.gfz-potsdam.de:5000/id2/software/services/fair/software-quality-assurance/software-quality-assurance:latest \
           sqa [MODIFIER] run [SERVICE]

-v [/ABSOLUTE/PATH/TO/REPO]:/repo

This binds the directory of your project to the docker container.

-u $(id -u ${USER}):$(id -g ${USER})

This makes the user/group of the container the same as the host system ones.

git.gfz-potsdam.de:5000/id2/software/services/fair/software-quality-assurance/software-quality-assurance:latest

This is the actual path to the image:image-version.

[SERVICE] [MODIFIER]

Here the service (and modifier) to run is specified.

To get a list of available services use:

docker run git.gfz-potsdam.de:5000/id2/software/services/fair/software-quality-assurance/software-quality-assurance:latest \
           sqa run --help

Examples¶

Run all services for the project within the directory ~/Projects/volcano_detection.

docker run -v ~/Projects/volcano_detection:/repo \
           -u $(id -u ${USER}):$(id -g ${USER}) \
           git.gfz-potsdam.de:5000/id2/software/services/fair/software-quality-assurance/software-quality-assurance:latest \
           sqa run

Get the version of the Software Quality Assurance.

docker run git.gfz-potsdam.de:5000/id2/software/services/fair/software-quality-assurance/software-quality-assurance:latest \
           sqa --version

native¶

general syntax:

sqa [OPTIONS] COMMAND [ARGS]