debian-mirror-gitlab/doc/development/integrations/secure.md

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

742 lines
33 KiB
Markdown
Raw Normal View History

2021-01-29 00:20:46 +05:30
---
stage: Protect
group: Container Security
2021-02-22 17:27:13 +05:30
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
2021-01-29 00:20:46 +05:30
---
2020-03-13 15:44:24 +05:30
# Security scanner integration
Integrating a security scanner into GitLab consists of providing end users
2021-09-30 23:02:18 +05:30
with a [CI job definition](../../ci/yaml/index.md)
2020-05-24 23:13:21 +05:30
they can add to their CI configuration files to scan their GitLab projects.
This CI job should then output its results in a GitLab-specified format. These results are then
2022-04-04 11:22:00 +05:30
automatically presented in various places in GitLab, such as the Pipeline view, merge request
2020-05-24 23:13:21 +05:30
widget, and Security Dashboard.
2020-03-13 15:44:24 +05:30
The scanning job is usually based on a [Docker image](https://docs.docker.com/)
that contains the scanner and all its dependencies in a self-contained environment.
2020-05-24 23:13:21 +05:30
This page documents requirements and guidelines for writing CI jobs that implement a security
scanner, as well as requirements and guidelines for the Docker image.
2020-03-13 15:44:24 +05:30
## Job definition
2020-06-23 00:09:42 +05:30
This section describes several important fields to add to the security scanner's job
2020-05-24 23:13:21 +05:30
definition file. Full documentation on these and other available fields can be viewed
2021-09-30 23:02:18 +05:30
in the [CI documentation](../../ci/yaml/index.md#image).
2020-05-24 23:13:21 +05:30
2020-03-13 15:44:24 +05:30
### Name
For consistency, scanning jobs should be named after the scanner, in lower case.
The job name is suffixed after the type of scanning:
2021-09-30 23:02:18 +05:30
- `_dependency_scanning`
- `_cluster_image_scanning`
- `_container_scanning`
- `_dast`
- `_sast`
2020-03-13 15:44:24 +05:30
For instance, the dependency scanning job based on the "MySec" scanner would be named `mysec_dependency_scanning`.
### Image
2021-09-30 23:02:18 +05:30
The [`image`](../../ci/yaml/index.md#image) keyword is used to specify
2020-03-13 15:44:24 +05:30
the [Docker image](../../ci/docker/using_docker_images.md#what-is-an-image)
containing the security scanner.
### Script
2021-09-30 23:02:18 +05:30
The [`script`](../../ci/yaml/index.md#script) keyword
2020-05-24 23:13:21 +05:30
is used to specify the commands to run the scanner.
Because the `script` entry can't be left empty, it must be set to the command that performs the scan.
2020-03-13 15:44:24 +05:30
It is not possible to rely on the predefined `ENTRYPOINT` and `CMD` of the Docker image
to perform the scan automatically, without passing any command.
2021-09-30 23:02:18 +05:30
The [`before_script`](../../ci/yaml/index.md#before_script)
2020-03-13 15:44:24 +05:30
should not be used in the job definition because users may rely on this to prepare their projects before performing the scan.
For instance, it is common practice to use `before_script` to install system libraries
a particular project needs before performing SAST or Dependency Scanning.
2021-09-30 23:02:18 +05:30
Similarly, [`after_script`](../../ci/yaml/index.md#after_script)
2020-04-22 19:07:51 +05:30
should not be used in the job definition, because it may be overridden by users.
2020-03-13 15:44:24 +05:30
### Stage
For consistency, scanning jobs should belong to the `test` stage when possible.
2021-09-30 23:02:18 +05:30
The [`stage`](../../ci/yaml/index.md#stage) keyword can be omitted because `test` is the default value.
2020-03-13 15:44:24 +05:30
### Fail-safe
To be aligned with the [GitLab Security paradigm](https://about.gitlab.com/direction/secure/#security-paradigm),
scanning jobs should not block the pipeline when they fail,
2021-09-30 23:02:18 +05:30
so the [`allow_failure`](../../ci/yaml/index.md#allow_failure) parameter should be set to `true`.
2020-03-13 15:44:24 +05:30
### Artifacts
Scanning jobs must declare a report that corresponds to the type of scanning they perform,
2021-09-30 23:02:18 +05:30
using the [`artifacts:reports`](../../ci/yaml/index.md#artifactsreports) keyword.
Valid reports are:
- `dependency_scanning`
- `container_scanning`
- `cluster_image_scanning`
- `dast`
- `api_fuzzing`
- `coverage_fuzzing`
- `sast`
2020-03-13 15:44:24 +05:30
For example, here is the definition of a SAST job that generates a file named `gl-sast-report.json`,
and uploads it as a SAST report:
```yaml
2020-06-23 00:09:42 +05:30
mysec_sast:
2020-03-13 15:44:24 +05:30
image: registry.gitlab.com/secure/mysec
artifacts:
reports:
sast: gl-sast-report.json
```
2021-02-22 17:27:13 +05:30
Note that `gl-sast-report.json` is an example file path but any other filename can be used. See
2020-05-24 23:13:21 +05:30
[the Output file section](#output-file) for more details. It's processed as a SAST report because
2021-02-22 17:27:13 +05:30
it's declared under the `reports:sast` key in the job definition, not because of the filename.
2020-03-13 15:44:24 +05:30
2020-04-22 19:07:51 +05:30
### Policies
2020-03-13 15:44:24 +05:30
2020-05-24 23:13:21 +05:30
Certain GitLab workflows, such as [AutoDevOps](../../topics/autodevops/customize.md#disable-jobs),
2021-04-17 20:07:23 +05:30
define CI/CD variables to indicate that given scans should be disabled. You can check for this by looking
2021-09-30 23:02:18 +05:30
for variables such as:
- `DEPENDENCY_SCANNING_DISABLED`
- `CONTAINER_SCANNING_DISABLED`
- `CLUSTER_IMAGE_SCANNING_DISABLED`
- `SAST_DISABLED`
- `DAST_DISABLED`
If appropriate based on the scanner type, you should then disable running the custom scanner.
2020-03-13 15:44:24 +05:30
2020-05-24 23:13:21 +05:30
GitLab also defines a `CI_PROJECT_REPOSITORY_LANGUAGES` variable, which provides the list of
2020-06-23 00:09:42 +05:30
languages in the repository. Depending on this value, your scanner may or may not do something different.
2020-03-13 15:44:24 +05:30
Language detection currently relies on the [`linguist`](https://github.com/github/linguist) Ruby gem.
2021-04-17 20:07:23 +05:30
See the [predefined CI/CD variables](../../ci/variables/predefined_variables.md).
2020-03-13 15:44:24 +05:30
2020-05-24 23:13:21 +05:30
#### Policy checking example
This example shows how to skip a custom Dependency Scanning job, `mysec_dependency_scanning`, unless
the project repository contains Java source code and the `dependency_scanning` feature is enabled:
2020-03-13 15:44:24 +05:30
```yaml
mysec_dependency_scanning:
2020-07-28 23:09:34 +05:30
rules:
- if: $DEPENDENCY_SCANNING_DISABLED
when: never
- if: $GITLAB_FEATURES =~ /\bdependency_scanning\b/
exists:
- '**/*.java'
2020-03-13 15:44:24 +05:30
```
2020-04-22 19:07:51 +05:30
Any additional job policy should only be configured by users based on their needs.
For instance, predefined policies should not trigger the scanning job
for a particular branch or when a particular set of files changes.
2020-03-13 15:44:24 +05:30
## Docker image
The Docker image is a self-contained environment that combines
the scanner with all the libraries and tools it depends on.
2020-05-24 23:13:21 +05:30
Packaging your scanner into a Docker image makes its dependencies and configuration always present,
regardless of the individual machine the scanner runs on.
2020-03-13 15:44:24 +05:30
### Image size
Depending on the CI infrastructure,
the CI may have to fetch the Docker image every time the job runs.
2020-06-23 00:09:42 +05:30
For the scanning job to run fast and avoid wasting bandwidth, Docker images should be as small as
possible. You should aim for 50MB or smaller. If that isn't possible, try to keep it below 1.46 GB,
which is the size of a CD-ROM.
2020-03-13 15:44:24 +05:30
If the scanner requires a fully functional Linux environment,
it is recommended to use a [Debian](https://www.debian.org/intro/about) "slim" distribution or [Alpine Linux](https://www.alpinelinux.org/).
If possible, it is recommended to build the image from scratch, using the `FROM scratch` instruction,
and to compile the scanner with all the libraries it needs.
[Multi-stage builds](https://docs.docker.com/develop/develop-images/multistage-build/)
might also help with keeping the image small.
2020-06-23 00:09:42 +05:30
To keep an image size small, consider using [dive](https://github.com/wagoodman/dive#dive) to analyze layers in a Docker image to
identify where additional bloat might be originating from.
In some cases, it might be difficult to remove files from an image. When this occurs, consider using
[Zstandard](https://github.com/facebook/zstd)
to compress files or large directories. Zstandard offers many different compression levels that can
decrease the size of your image with very little impact to decompression speed. It may be helpful to
automatically decompress any compressed directories as soon as an image launches. You can accomplish
this by adding a step to the Docker image's `/etc/bashrc` or to a specific user's `$HOME/.bashrc`.
Remember to change the entry point to launch a bash login shell if you chose the latter option.
Here are some examples to get you started:
- <https://gitlab.com/gitlab-org/security-products/license-management/-/blob/0b976fcffe0a9b8e80587adb076bcdf279c9331c/config/install.sh#L168-170>
- <https://gitlab.com/gitlab-org/security-products/license-management/-/blob/0b976fcffe0a9b8e80587adb076bcdf279c9331c/config/.bashrc#L49>
2020-03-13 15:44:24 +05:30
### Image tag
As documented in the [Docker Official Images](https://github.com/docker-library/official-images#tags-and-aliases) project,
it is strongly encouraged that version number tags be given aliases which allows the user to easily refer to the "most recent" release of a particular series.
2020-06-23 00:09:42 +05:30
See also [Docker Tagging: Best practices for tagging and versioning Docker images](https://docs.microsoft.com/en-us/archive/blogs/stevelasker/docker-tagging-best-practices-for-tagging-and-versioning-docker-images).
2020-03-13 15:44:24 +05:30
## Command line
A scanner is a command line tool that takes environment variables as inputs,
and generates a file that is uploaded as a report (based on the job definition).
It also generates text output on the standard output and standard error streams, and exits with a status code.
### Variables
2021-04-17 20:07:23 +05:30
All CI/CD variables are passed to the scanner as environment variables.
2021-09-30 23:02:18 +05:30
The scanned project is described by the [predefined CI/CD variables](../../ci/variables/index.md).
2020-03-13 15:44:24 +05:30
2020-05-24 23:13:21 +05:30
#### SAST and Dependency Scanning
2020-03-13 15:44:24 +05:30
2021-04-17 20:07:23 +05:30
SAST and Dependency Scanning scanners must scan the files in the project directory, given by the `CI_PROJECT_DIR` CI/CD variable.
2020-03-13 15:44:24 +05:30
#### Container Scanning
In order to be consistent with the official Container Scanning for GitLab,
scanners must scan the Docker image whose name and tag are given by
2021-01-29 00:20:46 +05:30
`CI_APPLICATION_REPOSITORY` and `CI_APPLICATION_TAG`, respectively. If the `DOCKER_IMAGE`
2021-04-17 20:07:23 +05:30
CI/CD variable is provided, then the `CI_APPLICATION_REPOSITORY` and `CI_APPLICATION_TAG` variables
2021-01-29 00:20:46 +05:30
are ignored, and the image specified in the `DOCKER_IMAGE` variable is scanned instead.
2020-03-13 15:44:24 +05:30
If not provided, `CI_APPLICATION_REPOSITORY` should default to
2021-04-17 20:07:23 +05:30
`$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG`, which is a combination of predefined CI/CD variables.
2020-03-13 15:44:24 +05:30
`CI_APPLICATION_TAG` should default to `CI_COMMIT_SHA`.
The scanner should sign in the Docker registry
using the variables `DOCKER_USER` and `DOCKER_PASSWORD`.
If these are not defined, then the scanner should use
`CI_REGISTRY_USER` and `CI_REGISTRY_PASSWORD` as default values.
2021-09-30 23:02:18 +05:30
#### Cluster Image Scanning
To be consistent with the official `cluster_image_scanning` for GitLab, scanners must scan the
Kubernetes cluster whose configuration is given by `KUBECONFIG`.
If you use the `CIS_KUBECONFIG` CI/CD variable, then the
`KUBECONFIG` variable is ignored and the cluster specified in the
`CIS_KUBECONFIG` variable is scanned instead. If you don't provide
the `CIS_KUBECONFIG` CI/CD variable, the value defaults to the value of
`$KUBECONFIG`. `$KUBECONFIG` is a predefined CI/CD variable configured when the project is assigned to a
Kubernetes cluster. When multiple contexts are provided in the `KUBECONFIG` variable, the context
selected as `current-context` will be used to fetch vulnerabilities.
2020-03-13 15:44:24 +05:30
#### Configuration files
While scanners may use `CI_PROJECT_DIR` to load specific configuration files,
2021-04-17 20:07:23 +05:30
it is recommended to expose configuration as CI/CD variables, not files.
2020-03-13 15:44:24 +05:30
### Output file
2021-04-17 20:07:23 +05:30
Like any artifact uploaded to GitLab CI/CD,
2020-03-13 15:44:24 +05:30
the Secure report generated by the scanner must be written in the project directory,
2021-04-17 20:07:23 +05:30
given by the `CI_PROJECT_DIR` CI/CD variable.
2020-03-13 15:44:24 +05:30
It is recommended to name the output file after the type of scanning, and to use `gl-` as a prefix.
Since all Secure reports are JSON files, it is recommended to use `.json` as a file extension.
2021-02-22 17:27:13 +05:30
For instance, a suggested filename for a Dependency Scanning report is `gl-dependency-scanning.json`.
2020-03-13 15:44:24 +05:30
2021-09-30 23:02:18 +05:30
The [`artifacts:reports`](../../ci/yaml/index.md#artifactsreports) keyword
2020-03-13 15:44:24 +05:30
of the job definition must be consistent with the file path where the Security report is written.
For instance, if a Dependency Scanning analyzer writes its report to the CI project directory,
2021-02-22 17:27:13 +05:30
and if this report filename is `depscan.json`,
2020-03-13 15:44:24 +05:30
then `artifacts:reports:dependency_scanning` must be set to `depscan.json`.
### Exit code
2021-02-22 17:27:13 +05:30
Following the POSIX exit code standard, the scanner exits with 0 for success and any number from 1 to 255 for anything else.
2020-03-13 15:44:24 +05:30
Success also includes the case when vulnerabilities are found.
2020-04-08 14:13:33 +05:30
When executing a scanning job using the [Docker-in-Docker privileged mode](../../user/application_security/sast/index.md#requirements),
we reserve the following standard exit codes.
| Orchestrator Exit Code | Description |
|------------------------|----------------------------------|
| 3 | No match, no compatible analyzer |
| 4 | Project directory empty |
| 5 | No compatible Docker image |
2020-03-13 15:44:24 +05:30
### Logging
The scanner should log error messages and warnings so that users can easily investigate
misconfiguration and integration issues by looking at the log of the CI scanning job.
Scanners may use [ANSI escape codes](https://en.wikipedia.org/wiki/ANSI_escape_code#Colors)
to colorize the messages they write to the Unix standard output and standard error streams.
We recommend using red to report errors, yellow for warnings, and green for notices.
Also, we recommend prefixing error messages with `[ERRO]`, warnings with `[WARN]`, and notices with `[INFO]`.
2020-07-28 23:09:34 +05:30
#### Logging level
The scanner should filter out a log message if its log level is lower than the
2021-04-17 20:07:23 +05:30
one set in the `SECURE_LOG_LEVEL` CI/CD variable. For instance, `info` and `warn`
2020-07-28 23:09:34 +05:30
messages should be skipped when `SECURE_LOG_LEVEL` is set to `error`. Accepted
values are as follows, listed from highest to lowest:
- `fatal`
- `error`
- `warn`
- `info`
- `debug`
It is recommended to use the `debug` level for verbose logging that could be
useful when debugging. The default value for `SECURE_LOG_LEVEL` should be set
to `info`.
2021-01-29 00:20:46 +05:30
When executing command lines, scanners should use the `debug` level to log the command line and its output.
For instance, the [bundler-audit](https://gitlab.com/gitlab-org/security-products/analyzers/bundler-audit) scanner
uses the `debug` level to log the command line `bundle audit check --quiet`,
and what `bundle audit` writes to the standard output.
2021-03-11 19:13:27 +05:30
If the command line fails, then it should be logged with the `error` log level;
this makes it possible to debug the problem without having to change the log level to `debug` and rerun the scanning job.
2021-01-29 00:20:46 +05:30
2021-04-17 20:07:23 +05:30
#### common `logutil` package
2020-07-28 23:09:34 +05:30
2022-01-26 12:08:38 +05:30
If you are using [go](https://go.dev/) and
2020-07-28 23:09:34 +05:30
[common](https://gitlab.com/gitlab-org/security-products/analyzers/common),
2021-04-17 20:07:23 +05:30
then it is suggested that you use [Logrus](https://github.com/Sirupsen/logrus)
and [common's `logutil` package](https://gitlab.com/gitlab-org/security-products/analyzers/common/-/tree/master/logutil)
to configure the formatter for [Logrus](https://github.com/Sirupsen/logrus).
See the [`logutil` README](https://gitlab.com/gitlab-org/security-products/analyzers/common/-/tree/master/logutil/README.md)
2020-07-28 23:09:34 +05:30
2020-03-13 15:44:24 +05:30
## Report
The report is a JSON document that combines vulnerabilities with possible remediations.
This documentation gives an overview of the report JSON format,
as well as recommendations and examples to help integrators set its fields.
The format is extensively described in the documentation of
[SAST](../../user/application_security/sast/index.md#reports-json-format),
2020-11-24 15:15:51 +05:30
[DAST](../../user/application_security/dast/#reports),
2020-03-13 15:44:24 +05:30
[Dependency Scanning](../../user/application_security/dependency_scanning/index.md#reports-json-format),
2021-09-30 23:02:18 +05:30
[Container Scanning](../../user/application_security/container_scanning/index.md#reports-json-format),
and [Cluster Image Scanning](../../user/application_security/cluster_image_scanning/index.md#reports-json-format).
2020-03-13 15:44:24 +05:30
2020-11-24 15:15:51 +05:30
You can find the schemas for these scanners here:
- [SAST](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/sast-report-format.json)
- [DAST](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/dast-report-format.json)
- [Dependency Scanning](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/dependency-scanning-report-format.json)
- [Container Scanning](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/container-scanning-report-format.json)
2021-06-08 01:23:25 +05:30
- [Coverage Fuzzing](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/coverage-fuzzing-report-format.json)
- [Secret Detection](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/secret-detection-report-format.json)
2020-03-13 15:44:24 +05:30
2022-06-21 17:19:12 +05:30
### Retention period for vulnerabilities
GitLab has the following retention policies for vulnerabilities on non-default branches. Vulnerabilities are no longer available:
- When the related CI job artifact expires.
- 90 days after the pipeline is created, even if the related CI job artifacts are locked.
To view vulnerabilities, either:
- Re-run the pipeline.
- Download the related CI job artifacts if they are available.
NOTE:
This does not apply for the vulnerabilities existing on the default branch.
2022-03-02 08:16:31 +05:30
### Enable report validation
2020-03-13 15:44:24 +05:30
2022-05-07 20:08:51 +05:30
> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/354928) in GitLab 14.9, and planned for removal in GitLab 15.0.
DISCLAIMER:
This page contains information related to upcoming products, features, and functionality.
It is important to note that the information presented is for informational purposes only.
Please do not rely on this information for purchasing or planning purposes.
As with all projects, the items mentioned on this page are subject to change or delay.
The development, release, and timing of any products, features, or functionality remain at the
sole discretion of GitLab Inc.
In GitLab 15.0 and later, report validation is enabled and enforced. Reports that fail validation
are not ingested, and an error message displays on the corresponding pipeline.
2020-03-13 15:44:24 +05:30
2022-05-07 20:08:51 +05:30
In GitLab 14.10 and later, report validation against the schemas is enabled but not enforced.
Reports that fail validation are ingested but display a warning in the pipeline security tab.
2022-03-02 08:16:31 +05:30
2022-05-07 20:08:51 +05:30
To enforce report validation for GitLab version 14.10 and earlier, set
[`VALIDATE_SCHEMA`](../../user/application_security/#enable-security-report-validation) to `"true"`.
### Report validation
You must ensure that reports generated by the scanner pass validation against the schema version
declared in your reports. Reports that don't pass validation are not ingested by GitLab, and an
error message displays on the corresponding pipeline.
Reports that use a deprecated version of the secure report schema are ingested but cause a warning
message to display on the corresponding pipeline. If you see this warning, update your
analyzer to use the latest available schemas.
After the deprecation period for a schema version, the file is removed from GitLab. Reports that
declare removed versions are rejected, and an error message displays on the corresponding pipeline.
GitLab uses the
2022-03-02 08:16:31 +05:30
[`json_schemer`](https://www.rubydoc.info/gems/json_schemer) gem to perform validation.
2022-05-07 20:08:51 +05:30
Ongoing improvements to report validation are tracked [in this epic](https://gitlab.com/groups/gitlab-org/-/epics/6968).
In the meantime, you can see which versions are supported in the
[source code](https://gitlab.com/gitlab-org/gitlab/-/blob/08dd756429731a0cca1e27ca9d59eea226398a7d/lib/gitlab/ci/parsers/security/validators/schema_validator.rb#L9-27).
2022-03-02 08:16:31 +05:30
### Report Fields
#### Version
This field specifies which [Security Report Schemas](https://gitlab.com/gitlab-org/security-products/security-report-schemas) version you are using. For information about the versions to use, see [releases](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/releases).
In GitLab 14.10 and later, GitLab validates your report against the version of the schema specified by this value.
The versions supported by GitLab can be found in
[`gitlab/ee/lib/ee/gitlab/ci/parsers/security/validators/schemas`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/lib/ee/gitlab/ci/parsers/security/validators/schemas).
#### Vulnerabilities
2020-03-13 15:44:24 +05:30
The `vulnerabilities` field of the report is an array of vulnerability objects.
2022-03-02 08:16:31 +05:30
##### ID
2020-04-22 19:07:51 +05:30
2021-01-29 00:20:46 +05:30
The `id` field is the unique identifier of the vulnerability.
2020-04-22 19:07:51 +05:30
It is used to reference a fixed vulnerability from a [remediation objects](#remediations).
We recommend that you generate a UUID and use it as the `id` field's value.
2022-03-02 08:16:31 +05:30
##### Category
2020-03-13 15:44:24 +05:30
The value of the `category` field matches the report type:
2021-09-30 23:02:18 +05:30
- `dependency_scanning`
- `cluster_image_scanning`
- `container_scanning`
- `sast`
- `dast`
2020-03-13 15:44:24 +05:30
2022-03-02 08:16:31 +05:30
##### Scanner
2020-03-13 15:44:24 +05:30
The `scanner` field is an object that embeds a human-readable `name` and a technical `id`.
The `id` should not collide with any other scanner another integrator would provide.
2022-03-02 08:16:31 +05:30
##### Name, message, and description
2020-03-13 15:44:24 +05:30
2020-05-24 23:13:21 +05:30
The `name` and `message` fields contain a short description of the vulnerability.
The `description` field provides more details.
2020-03-13 15:44:24 +05:30
2020-05-24 23:13:21 +05:30
The `name` field is context-free and contains no information on where the vulnerability has been found,
2020-03-13 15:44:24 +05:30
whereas the `message` may repeat the location.
2020-05-24 23:13:21 +05:30
As a visual example, this screenshot highlights where these fields are used when viewing a
vulnerability as part of a pipeline view.
2021-02-22 17:27:13 +05:30
![Example Vulnerability](img/example_vuln.png)
2020-05-24 23:13:21 +05:30
2020-03-13 15:44:24 +05:30
For instance, a `message` for a vulnerability
reported by Dependency Scanning gives information on the vulnerable dependency,
which is redundant with the `location` field of the vulnerability.
The `name` field is preferred but the `message` field is used
when the context/location cannot be removed from the title of the vulnerability.
To illustrate, here is an example vulnerability object reported by a Dependency Scanning scanner,
and where the `message` repeats the `location` field:
```json
{
"location": {
"dependency": {
"package": {
"name": "debug"
}
}
},
"name": "Regular Expression Denial of Service",
"message": "Regular Expression Denial of Service in debug",
"description": "The debug module is vulnerable to regular expression denial of service
when untrusted user input is passed into the `o` formatter.
It takes around 50k characters to block for 2 seconds making this a low severity issue."
}
```
2020-04-08 14:13:33 +05:30
2020-03-13 15:44:24 +05:30
The `description` might explain how the vulnerability works or give context about the exploit.
It should not repeat the other fields of the vulnerability object.
In particular, the `description` should not repeat the `location` (what is affected)
or the `solution` (how to mitigate the risk).
2022-03-02 08:16:31 +05:30
##### Solution
2020-03-13 15:44:24 +05:30
2020-05-24 23:13:21 +05:30
You can use the `solution` field to instruct users how to fix the identified vulnerability or to mitigate
the risk. End-users interact with this field, whereas GitLab automatically processes the
`remediations` objects.
2020-03-13 15:44:24 +05:30
2022-03-02 08:16:31 +05:30
##### Identifiers
2020-03-13 15:44:24 +05:30
2020-05-24 23:13:21 +05:30
The `identifiers` array describes the detected vulnerability. An identifier object's `type` and
`value` fields are used to tell if two identifiers are the same. The user interface uses the
object's `name` and `url` fields to display the identifier.
2020-03-13 15:44:24 +05:30
It is recommended to reuse the identifiers the GitLab scanners already define:
| Identifier | Type | Example value |
|------------|------|---------------|
| [CVE](https://cve.mitre.org/cve/) | `cve` | CVE-2019-10086 |
| [CWE](https://cwe.mitre.org/data/index.html) | `cwe` | CWE-1026 |
| [OSVD](https://cve.mitre.org/data/refs/refmap/source-OSVDB.html) | `osvdb` | OSVDB-113928 |
2020-10-24 23:57:45 +05:30
| [USN](https://ubuntu.com/security/notices) | `usn` | USN-4234-1 |
2020-03-13 15:44:24 +05:30
| [WASC](http://projects.webappsec.org/Threat-Classification-Reference-Grid) | `wasc` | WASC-19 |
2020-04-22 19:07:51 +05:30
| [RHSA](https://access.redhat.com/errata/#/) | `rhsa` | RHSA-2020:0111 |
2020-03-13 15:44:24 +05:30
| [ELSA](https://linux.oracle.com/security/) | `elsa` | ELSA-2020-0085 |
2020-05-24 23:13:21 +05:30
The generic identifiers listed above are defined in the [common library](https://gitlab.com/gitlab-org/security-products/analyzers/common),
2021-06-08 01:23:25 +05:30
which is shared by some of the analyzers that GitLab maintains. You can [contribute](https://gitlab.com/gitlab-org/security-products/analyzers/common/blob/master/issue/identifier.go)
2020-05-24 23:13:21 +05:30
new generic identifiers to if needed. Analyzers may also produce vendor-specific or product-specific
identifiers, which don't belong in the [common library](https://gitlab.com/gitlab-org/security-products/analyzers/common).
2020-03-13 15:44:24 +05:30
2021-01-03 14:25:43 +05:30
The first item of the `identifiers` array is called the [primary
identifier](../../user/application_security/terminology/#primary-identifier).
2020-03-13 15:44:24 +05:30
The primary identifier is particularly important, because it is used to
2020-05-24 23:13:21 +05:30
[track vulnerabilities](#tracking-and-merging-vulnerabilities) as new commits are pushed to the repository.
Identifiers are also used to [merge duplicate vulnerabilities](#tracking-and-merging-vulnerabilities)
2020-03-13 15:44:24 +05:30
reported for the same commit, except for `CWE` and `WASC`.
2021-01-03 14:25:43 +05:30
Not all vulnerabilities have CVEs, and a CVE can be identified multiple times. As a result, a CVE
isn't a stable identifier and you shouldn't assume it as such when tracking vulnerabilities.
The maximum number of identifiers for a vulnerability is set as 20. If a vulnerability has more than 20 identifiers,
2022-03-02 08:16:31 +05:30
the system saves only the first 20 of them. Note that vulnerabilities in the [Pipeline Security](../../user/application_security/security_dashboard/#view-vulnerabilities-in-a-pipeline)
2021-02-22 17:27:13 +05:30
tab do not enforce this limit and all identifiers present in the report artifact are displayed.
2021-01-03 14:25:43 +05:30
2022-03-02 08:16:31 +05:30
#### Details
2021-11-11 11:23:49 +05:30
The `details` field is an object that supports many different content elements that are displayed when viewing vulnerability information. An example of the various data elements can be seen in the [security-reports repository](https://gitlab.com/gitlab-examples/security/security-reports/-/tree/master/samples/details-example).
2022-03-02 08:16:31 +05:30
#### Location
2020-03-13 15:44:24 +05:30
The `location` indicates where the vulnerability has been detected.
The format of the location depends on the type of scanning.
Internally GitLab extracts some attributes of the `location` to generate the **location fingerprint**,
2020-05-24 23:13:21 +05:30
which is used to track vulnerabilities
2020-03-13 15:44:24 +05:30
as new commits are pushed to the repository.
The attributes used to generate the location fingerprint also depend on the type of scanning.
2022-03-02 08:16:31 +05:30
##### Dependency Scanning
2020-03-13 15:44:24 +05:30
The `location` of a Dependency Scanning vulnerability is composed of a `dependency` and a `file`.
The `dependency` object describes the affected `package` and the dependency `version`.
`package` embeds the `name` of the affected library/module.
`file` is the path of the dependency file that declares the affected dependency.
For instance, here is the `location` object for a vulnerability affecting
version `4.0.11` of npm package [`handlebars`](https://www.npmjs.com/package/handlebars):
```json
{
"file": "client/package.json",
"dependency": {
"package": {
"name": "handlebars"
},
"version": "4.0.11"
}
}
```
This affected dependency is listed in `client/package.json`,
a dependency file processed by npm or yarn.
The location fingerprint of a Dependency Scanning vulnerability
combines the `file` and the package `name`,
so these attributes are mandatory.
All other attributes are optional.
2022-03-02 08:16:31 +05:30
##### Container Scanning
2020-03-13 15:44:24 +05:30
Similar to Dependency Scanning,
the `location` of a Container Scanning vulnerability has a `dependency` and a `file`.
It also has an `operating_system` field.
For instance, here is the `location` object for a vulnerability affecting
version `2.50.3-2+deb9u1` of Debian package `glib2.0`:
```json
{
"dependency": {
"package": {
"name": "glib2.0"
},
},
"version": "2.50.3-2+deb9u1",
"operating_system": "debian:9",
"image": "registry.gitlab.com/example/app:latest"
}
```
The affected package is found when scanning the Docker image `registry.gitlab.com/example/app:latest`.
The Docker image is based on `debian:9` (Debian Stretch).
The location fingerprint of a Container Scanning vulnerability
combines the `operating_system` and the package `name`,
so these attributes are mandatory.
The `image` is also mandatory.
All other attributes are optional.
2022-03-02 08:16:31 +05:30
##### Cluster Image Scanning
2021-09-30 23:02:18 +05:30
The `location` of a `cluster_image_scanning` vulnerability has a `dependency` field. It also has
an `operating_system` field. For example, here is the `location` object for a vulnerability
affecting version `2.50.3-2+deb9u1` of Debian package `glib2.0`:
```json
{
"dependency": {
"package": {
"name": "glib2.0"
},
},
"version": "2.50.3-2+deb9u1",
"operating_system": "debian:9",
2021-11-18 22:05:49 +05:30
"image": "index.docker.io/library/nginx:1.18",
"kubernetes_resource": {
"namespace": "production",
"kind": "Deployment",
"name": "nginx-ingress",
"container_name": "nginx",
"agent_id": "1"
}
2021-09-30 23:02:18 +05:30
}
```
2021-11-18 22:05:49 +05:30
The affected package is found when scanning a deployment using the `index.docker.io/library/nginx:1.18` image.
2021-09-30 23:02:18 +05:30
The location fingerprint of a Cluster Image Scanning vulnerability combines the
2021-11-18 22:05:49 +05:30
`namespace`, `kind`, `name`, and `container_name` fields from the `kubernetes_resource`,
as well as the package `name`, so these fields are required. The `image` field is also mandatory.
The `cluster_id` and `agent_id` are mutually exclusive, and one of them must be present.
All other fields are optional.
2021-09-30 23:02:18 +05:30
2022-03-02 08:16:31 +05:30
##### SAST
2020-03-13 15:44:24 +05:30
The `location` of a SAST vulnerability must have a `file` and a `start_line` field,
giving the path of the affected file, and the affected line number, respectively.
It may also have an `end_line`, a `class`, and a `method`.
For instance, here is the `location` object for a security flaw found
at line `41` of `src/main/java/com/gitlab/example/App.java`,
2020-04-22 19:07:51 +05:30
in the `generateSecretToken` method of the `com.gitlab.security_products.tests.App` Java class:
2020-03-13 15:44:24 +05:30
```json
{
"file": "src/main/java/com/gitlab/example/App.java",
"start_line": 41,
"end_line": 41,
"class": "com.gitlab.security_products.tests.App",
"method": "generateSecretToken1"
}
```
The location fingerprint of a SAST vulnerability
combines `file`, `start_line`, and `end_line`,
so these attributes are mandatory.
All other attributes are optional.
2022-03-02 08:16:31 +05:30
#### Tracking and merging vulnerabilities
2020-03-13 15:44:24 +05:30
Users may give feedback on a vulnerability:
2020-05-24 23:13:21 +05:30
- They may dismiss a vulnerability if it doesn't apply to their projects
- They may create an issue for a vulnerability if there's a possible threat
2020-03-13 15:44:24 +05:30
GitLab tracks vulnerabilities so that user feedback is not lost
when new Git commits are pushed to the repository.
Vulnerabilities are tracked using a combination of three attributes:
- [Report type](#category)
- [Location fingerprint](#location)
- [Primary identifier](#identifiers)
Right now, GitLab cannot track a vulnerability if its location changes
as new Git commits are pushed, and this results in user feedback being lost.
For instance, user feedback on a SAST vulnerability is lost
if the affected file is renamed or the affected line moves down.
2020-06-23 00:09:42 +05:30
This is addressed in [issue #7586](https://gitlab.com/gitlab-org/gitlab/-/issues/7586).
2020-03-13 15:44:24 +05:30
In some cases, the multiple scans executed in the same CI pipeline result in duplicates
that are automatically merged using the vulnerability location and identifiers.
Two vulnerabilities are considered to be the same if they share the same [location fingerprint](#location)
and at least one [identifier](#identifiers). Two identifiers are the same if they share the same `type` and `id`.
CWE and WASC identifiers are not considered because they describe categories of vulnerability flaws,
but not specific security flaws.
2022-03-02 08:16:31 +05:30
##### Severity and confidence
2020-03-13 15:44:24 +05:30
The `severity` field describes how much the vulnerability impacts the software,
whereas the `confidence` field describes how reliable the assessment of the vulnerability is.
The severity is used to sort the vulnerabilities in the security dashboard.
The severity ranges from `Info` to `Critical`, but it can also be `Unknown`.
Valid values are: `Unknown`, `Info`, `Low`, `Medium`, `High`, or `Critical`
The confidence ranges from `Low` to `Confirmed`, but it can also be `Unknown`,
`Experimental` or even `Ignore` if the vulnerability is to be ignored.
Valid values are: `Ignore`, `Unknown`, `Experimental`, `Low`, `Medium`, `High`, or `Confirmed`
2020-06-23 00:09:42 +05:30
`Unknown` values means that data is unavailable to determine it's actual value. Therefore, it may be `high`, `medium`, or `low`,
and needs to be investigated. We have [provided a chart](../../user/application_security/sast/analyzers.md#analyzers-data)
of the available SAST Analyzers and what data is currently available.
2022-03-02 08:16:31 +05:30
#### Remediations
2020-03-13 15:44:24 +05:30
The `remediations` field of the report is an array of remediation objects.
2020-05-24 23:13:21 +05:30
Each remediation describes a patch that can be applied to
2021-09-04 01:27:46 +05:30
[resolve](../../user/application_security/vulnerabilities/index.md#resolve-a-vulnerability)
2020-03-13 15:44:24 +05:30
a set of vulnerabilities.
2020-05-24 23:13:21 +05:30
Here is an example of a report that contains remediations.
```json
{
"vulnerabilities": [
{
"category": "dependency_scanning",
"name": "Regular Expression Denial of Service",
"id": "123e4567-e89b-12d3-a456-426655440000",
"solution": "Upgrade to new versions.",
"scanner": {
"id": "gemnasium",
"name": "Gemnasium"
},
"identifiers": [
{
"type": "gemnasium",
"name": "Gemnasium-642735a5-1425-428d-8d4e-3c854885a3c9",
"value": "642735a5-1425-428d-8d4e-3c854885a3c9"
}
]
}
],
"remediations": [
{
"fixes": [
{
"id": "123e4567-e89b-12d3-a456-426655440000"
}
],
"summary": "Upgrade to new version",
"diff": "ZGlmZiAtLWdpdCBhL3lhcm4ubG9jayBiL3lhcm4ubG9jawppbmRleCAwZWNjOTJmLi43ZmE0NTU0IDEwMDY0NAotLS0gYS95Y=="
}
]
}
```
2022-03-02 08:16:31 +05:30
##### Summary
2020-04-22 19:07:51 +05:30
2020-05-24 23:13:21 +05:30
The `summary` field is an overview of how the vulnerabilities can be fixed. This field is required.
2020-04-22 19:07:51 +05:30
2022-03-02 08:16:31 +05:30
##### Fixed vulnerabilities
2020-04-22 19:07:51 +05:30
The `fixes` field is an array of objects that reference the vulnerabilities fixed by the
2020-05-24 23:13:21 +05:30
remediation. `fixes[].id` contains a fixed vulnerability's [unique identifier](#id). This field is required.
2020-04-22 19:07:51 +05:30
2022-03-02 08:16:31 +05:30
##### Diff
2020-04-22 19:07:51 +05:30
2020-05-24 23:13:21 +05:30
The `diff` field is a base64-encoded remediation code diff, compatible with
[`git apply`](https://git-scm.com/docs/git-format-patch#_discussion). This field is required.