118 lines
6.2 KiB
Markdown
118 lines
6.2 KiB
Markdown
|
---
|
||
|
|
stage: Secure
|
||
|
|
group: Fuzz Testing
|
||
|
|
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers
|
||
|
|
type: reference, howto
|
||
|
|
---
|
||
|
|
|
||
|
|
# Coverage Guided Fuzz Testing **(ULTIMATE)**
|
||
|
|
|
||
|
|
> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3226) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.2 as an [Alpha feature](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha).
|
||
|
|
|
||
|
|
GitLab allows you to add coverage-guided fuzz testing to your pipelines. This helps you discover
|
||
|
|
bugs and potential security issues that other QA processes may miss. Coverage-guided fuzzing sends
|
||
|
|
random inputs to an instrumented version of your application in an effort to cause unexpected
|
||
|
|
behavior, such as a crash. Such behavior indicates a bug that you should address.
|
||
|
|
|
||
|
|
We recommend that you use fuzz testing in addition to the other security scanners in [GitLab Secure](../index.md)
|
||
|
|
and your own test processes. If you're using [GitLab CI/CD](../../../ci/README.md),
|
||
|
|
you can run your coverage guided fuzz tests as part your CI/CD workflow. You can take advantage of
|
||
|
|
Coverage Guided Fuzzing by including the CI job in your existing `.gitlab-ci.yml` file.
|
||
|
|
|
||
|
|
## Supported fuzzing engines and languages
|
||
|
|
|
||
|
|
GitLab supports these languages through the fuzzing engine listed for each. We currently provide a Docker image for apps written in Go, but you can test the other languages below by providing a Docker image with the fuzz engine to run your app.
|
||
|
|
|
||
|
|
| Language | Fuzzing Engine | Example |
|
||
|
|
|----------|---------------------------------------------------------------------------|---------|
|
||
|
|
| C/C++ | [libFuzzer](https://llvm.org/docs/LibFuzzer.html) | |
|
||
|
|
| GoLang | [go-fuzz (libFuzzer support)](https://github.com/dvyukov/go-fuzz) | |
|
||
|
|
| Rust | [cargo-fuzz (libFuzzer support)](https://github.com/rust-fuzz/cargo-fuzz) | |
|
||
|
|
|
||
|
|
## Configuration
|
||
|
|
|
||
|
|
To enable fuzzing, you must
|
||
|
|
[include](../../../ci/yaml/README.md#includetemplate)
|
||
|
|
the [`Coverage-Fuzzing.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/Coverage-Fuzzing.gitlab-ci.yml)
|
||
|
|
provided as part of your GitLab installation.
|
||
|
|
|
||
|
|
To do so, add the following to your `.gitlab-ci.yml` file:
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
include:
|
||
|
|
- template: Coverage-Fuzzing.gitlab-ci.yml
|
||
|
|
```
|
||
|
|
|
||
|
|
The included template makes available the [hidden job](../../../ci/yaml/README.md#hide-jobs)
|
||
|
|
`.fuzz_base`, which you must [extend](../../../ci/yaml/README.md#extends) for each of your fuzz
|
||
|
|
targets. Each fuzz target **must** have a separate job. For example, the
|
||
|
|
[go-fuzzing-example project](https://gitlab.com/gitlab-org/security-products/demos/go-fuzzing-example)
|
||
|
|
contains one job that extends `.fuzz_base` for its single fuzz target.
|
||
|
|
|
||
|
|
The `my_fuzz_target` job (the separate job for your fuzz target) does the following:
|
||
|
|
|
||
|
|
- Extends `.fuzz_base`.
|
||
|
|
- Compiles the fuzz target with [go-fuzz](https://github.com/dvyukov/go-fuzz).
|
||
|
|
- Runs the target with the `gitlab-cov-fuzz` command, which is available to each job that extends
|
||
|
|
`.fuzz_base`.
|
||
|
|
- Runs on a fuzz stage that usually comes after a test stage.
|
||
|
|
|
||
|
|
The `gitlab-cov-fuzz` is a command-line tool that runs the instrumented application. It parses and
|
||
|
|
analyzes the exception information that the fuzzer outputs. It also downloads the [corpus](#glossary)
|
||
|
|
and crash events from previous pipelines automatically. This helps your fuzz targets build on the progress of
|
||
|
|
previous fuzzing jobs. The parsed crash events and data are written to
|
||
|
|
`gl-coverage-fuzzing-report.json`.
|
||
|
|
|
||
|
|
### Artifacts
|
||
|
|
|
||
|
|
Each fuzzing step outputs these artifacts:
|
||
|
|
|
||
|
|
- `gl-coverage-fuzzing-report.json`: This file's format may change in future releases.
|
||
|
|
- `artifacts.zip`: This file contains two directories:
|
||
|
|
- `corpus`: Holds all test cases generated by the current and all previous jobs.
|
||
|
|
- `crashes`: Holds all crash events the current job encountered as well as those not fixed in
|
||
|
|
previous jobs.
|
||
|
|
|
||
|
|
### Types of Fuzzing Jobs
|
||
|
|
|
||
|
|
There are two types of jobs:
|
||
|
|
|
||
|
|
- Fuzzing: Standard fuzzing session. You can configure a long session through a user defined
|
||
|
|
timeout.
|
||
|
|
- Regression: Run the fuzz targets through the accumulated test cases generated by previous fuzzing
|
||
|
|
sessions plus fixed crashes from previous sessions. This is usually very quick.
|
||
|
|
|
||
|
|
Here's our current suggestion for configuring your fuzz target's timeout:
|
||
|
|
|
||
|
|
- Set `COVERAGE_FUZZING_BRANCH` to the branch where you want to run long-running (async) fuzzing
|
||
|
|
jobs. This is `master` by default.
|
||
|
|
- Use regression or short-running fuzzing jobs for other branches or merge requests.
|
||
|
|
|
||
|
|
This suggestion helps find new bugs on the development branch and catch old bugs in merge requests
|
||
|
|
(like unit tests).
|
||
|
|
|
||
|
|
You can configure this by passing `--regression=false/true` to `gitlab-cov-fuzz` as the [Go example](https://gitlab.com/gitlab-org/security-products/demos/go-fuzzing-example/-/blob/master/.gitlab-ci.yml)
|
||
|
|
shows. Also note that `gitlab-cov-fuzz` is a wrapper, so you can pass those arguments to configure
|
||
|
|
any option available in the underlying fuzzing engine.
|
||
|
|
|
||
|
|
### Available variables
|
||
|
|
|
||
|
|
| Environment variable | Description |
|
||
|
|
|---------------------------|--------------------------------------------------------------------|
|
||
|
|
| `COVERAGE_FUZZING_BRANCH` | The branch for long-running fuzzing jobs. The default is `master`. |
|
||
|
|
|
||
|
|
### Additional Configuration
|
||
|
|
|
||
|
|
The `gitlab-cov-fuzz` command passes all arguments it receives to the underlying fuzzing engine. You
|
||
|
|
can therefore use all the options available in that fuzzing engine. For more information on these
|
||
|
|
options, see the underlying fuzzing engine's documentation.
|
||
|
|
|
||
|
|
### Glossary
|
||
|
|
|
||
|
|
- Seed corpus: The set of test cases given as initial input to the fuzz target. This usually speeds
|
||
|
|
up the fuzz target substantially. This can be either manually created test cases or auto-generated
|
||
|
|
with the fuzz target itself from previous runs.
|
||
|
|
- Corpus: The set of meaningful test cases that are generated while the fuzzer is running. Each
|
||
|
|
meaningful test case produces new coverage in the tested program. It's advised to re-use the
|
||
|
|
corpus and pass it to subsequent runs.
|