info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
you can run DAST API tests as part your CI/CD workflow.
## Requirements
- One of the following web API types:
- REST API
- SOAP
- GraphQL
- Form bodies, JSON, or XML
- One of the following assets to provide APIs to test:
- OpenAPI v2 or v3 API definition
- Postman Collection v2.0 or v2.1
- HTTP Archive (HAR) of API requests to test
## When DAST API scans run
When using the `DAST-API.gitlab-ci.yml` template, the defined jobs use the `dast` stage by default. To enable your `.gitlab-ci.yml` file must include the `dast` stage in your `stages` definition. To ensure DAST API scans the latest code, your CI pipeline should deploy changes to a test environment in a stage before the `dast` stage:
```yaml
stages:
- build
- test
- deploy
- dast
```
Note that if your pipeline is configured to deploy to the same web server on each run, running a
pipeline while another is still running could cause a race condition in which one pipeline
overwrites the code from another. The API to scan should be excluded from changes for the duration
of a DAST API scan. The only changes to the API should be from the DAST API scanner. Be aware that
any changes made to the API (for example, by users, scheduled tasks, database changes, code
changes, other pipelines, or other scanners) during a scan could cause inaccurate results.
## Enable DAST API scanning
There are three ways to perform scans. See the configuration section for the one you wish to use:
- [OpenAPI v2 or v3 specification](#openapi-specification)
- [HTTP Archive (HAR)](#http-archive-har)
- [Postman Collection v2.0 or v2.1](#postman-collection)
Examples of various configurations can be found here:
that's provided as part of your GitLab installation. Add the following to your
`.gitlab-ci.yml` file:
```yaml
stages:
- dast
include:
- template: DAST-API.gitlab-ci.yml
```
1. The [configuration file](#configuration-files) has several testing profiles defined with different checks enabled. We recommend that you start with the `Quick` profile.
Testing with this profile completes faster, allowing for easier configuration validation.
Provide the profile by adding the `DAST_API_PROFILE` CI/CD variable to your `.gitlab-ci.yml` file,
substituting `Quick` for the profile you choose:
```yaml
stages:
- dast
include:
- template: DAST-API.gitlab-ci.yml
variables:
DAST_API_PROFILE: Quick
```
1. Provide the location of the OpenAPI specification. You can provide the specification as a file
or URL. Specify the location by adding the `DAST_API_OPENAPI` variable:
```yaml
stages:
- dast
include:
- template: DAST-API.gitlab-ci.yml
variables:
DAST_API_PROFILE: Quick
DAST_API_OPENAPI: test-api-specification.json
```
1. The target API instance's base URL is also required. Provide it by using the `DAST_API_TARGET_URL`
variable or an `environment_url.txt` file.
Adding the URL in an `environment_url.txt` file at your project's root is great for testing in
dynamic environments. To run DAST API against an app dynamically created during a GitLab CI/CD
pipeline, have the app persist its URL in an `environment_url.txt` file. DAST API
automatically parses that file to find its scan target. You can see an
This is a minimal configuration for DAST API. From here you can:
- [Run your first scan](#running-your-first-scan).
- [Add authentication](#authentication).
- Learn how to [handle false positives](#handling-false-positives).
WARNING:
**NEVER** run DAST API testing against a production server. Not only can it perform *any* function that the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting data. Only run DAST API scanning against a test server.
### HTTP Archive (HAR)
The [HTTP Archive format (HAR)](http://www.softwareishard.com/blog/har-12-spec/)
is an archive file format for logging HTTP transactions. When used with the GitLab DAST API scanner, HAR must contain records of calling the web API to test. The DAST API scanner extracts all the requests and
uses them to perform testing.
You can use various tools to generate HAR files:
- [Insomnia Core](https://insomnia.rest/): API client
that's provided as part of your GitLab installation. To do so, add the following to your
`.gitlab-ci.yml` file:
```yaml
stages:
- dast
include:
- template: DAST-API.gitlab-ci.yml
```
1. The [configuration file](#configuration-files) has several testing profiles defined with different checks enabled. We recommend that you start with the `Quick` profile.
Testing with this profile completes faster, allowing for easier configuration validation.
Provide the profile by adding the `DAST_API_PROFILE` CI/CD variable to your `.gitlab-ci.yml` file,
substituting `Quick` for the profile you choose:
```yaml
stages:
- dast
include:
- template: DAST-API.gitlab-ci.yml
variables:
DAST_API_PROFILE: Quick
```
1. Provide the location of the HAR specification. You can provide the specification as a file
or URL. [URL support was introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/285020) in GitLab 13.10 and later. Specify the location by adding the `DAST_API_HAR` variable:
```yaml
stages:
- dast
include:
- template: DAST-API.gitlab-ci.yml
variables:
DAST_API_PROFILE: Quick
DAST_API_HAR: test-api-recording.har
```
1. The target API instance's base URL is also required. Provide it by using the `DAST_API_TARGET_URL`
variable or an `environment_url.txt` file.
Adding the URL in an `environment_url.txt` file at your project's root is great for testing in
dynamic environments. To run DAST API against an app dynamically created during a GitLab CI/CD
pipeline, have the app persist its URL in an `environment_url.txt` file. DAST API
automatically parses that file to find its scan target. You can see an
that's provided as part of your GitLab installation. To do so, add the following to your
`.gitlab-ci.yml` file:
```yaml
stages:
- dast
include:
- template: DAST-API.gitlab-ci.yml
```
1. The [configuration file](#configuration-files) has several testing profiles defined with different checks enabled. We recommend that you start with the `Quick` profile.
Testing with this profile completes faster, allowing for easier configuration validation.
Provide the profile by adding the `DAST_API_PROFILE` CI/CD variable to your `.gitlab-ci.yml` file,
substituting `Quick` for the profile you choose:
```yaml
stages:
- dast
include:
- template: DAST-API.gitlab-ci.yml
variables:
DAST_API_PROFILE: Quick
```
1. Provide the location of the Postman Collection specification. You can provide the specification as a file or URL. Specify the location by adding the `DAST_API_POSTMAN_COLLECTION` variable:
`{"headers":{"Authorization":"Bearer dXNlcm5hbWU6cGFzc3dvcmQ="}}` (substitute your token). You
can create CI/CD variables from the GitLab projects page at **Settings > CI/CD**, in the
**Variables** section.
1. In your `.gitlab-ci.yml` file, set `DAST_API_OVERRIDES_ENV` to the variable you just created:
```yaml
stages:
- dast
include:
- template: DAST-API.gitlab-ci.yml
variables:
DAST_API_PROFILE: Quick
DAST_API_OPENAPI: test-api-specification.json
DAST_API_TARGET_URL: http://test-deployment/
DAST_API_OVERRIDES_ENV: $TEST_API_BEARERAUTH
```
1. To validate that authentication is working, run an DAST API test and review the job logs
and the test API's application logs.
##### Token generated at test runtime
If the bearer token must be generated and doesn't expire during testing, you can provide to DAST API a file containing the token. A prior stage and job, or part of the DAST API job, can
generate this file.
DAST API expects to receive a JSON file with the following structure:
The override engine uses `body-form` when the request body has only form-data content.
Example usage for setting a `body-json` override:
```json
{
"body-json": {
"$.credentials.access-token": "iddqd!42.$"
}
}
```
Note that each JSON property name in the object `body-json` is set to a [JSON Path](https://goessner.net/articles/JsonPath/)
expression. The JSON Path expression `$.credentials.access-token` identifies the node to be
overridden with the value `iddqd!42.$`. The override engine uses `body-json` when the request body
has only [JSON](https://www.json.org/json-en.html) content.
For example, if the body is set to the following JSON:
```json
{
"credentials" : {
"username" :"john.doe",
"access-token" : "non-valid-password"
}
}
```
It is changed to:
```json
{
"credentials" : {
"username" :"john.doe",
"access-token" : "iddqd!42.$"
}
}
```
Here's an example for setting a `body-xml` override. The first entry overrides an XML attribute and
the second entry overrides an XML element:
```json
{
"body-xml" : {
"/credentials/@isEnabled": "true",
"/credentials/access-token/text()" : "iddqd!42.$"
}
}
```
Note that each JSON property name in the object `body-xml` is set to an
[XPath v2](https://www.w3.org/TR/xpath20/)
expression. The XPath expression `/credentials/@isEnabled` identifies the attribute node to override
with the value `true`. The XPath expression `/credentials/access-token/text()` identifies the
element node to override with the value `iddqd!42.$`. The override engine uses `body-xml` when the
request body has only [XML](https://www.w3.org/XML/)
content.
For example, if the body is set to the following XML:
```xml
<credentialsisEnabled="false">
<username>john.doe</username>
<access-token>non-valid-password</access-token>
</credentials>
```
It is changed to:
```xml
<credentialsisEnabled="true">
<username>john.doe</username>
<access-token>iddqd!42.$</access-token>
</credentials>
```
You can provide this JSON document as a file or environment variable. You may also provide a command
to generate the JSON document. The command can run at intervals to support values that expire.
#### Using a file
To provide the overrides JSON as a file, the `DAST_API_OVERRIDES_FILE` CI/CD variable is set. The path is relative to the job current working directory.
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211892) in GitLab 14.0.
When testing an API it can be useful to exclude certain paths. For example, you might exclude testing of an authentication service or an older version of the API. To exclude paths, use the `FUZZAPI_EXCLUDE_PATHS` CI/CD variable . This variable is specified in your `.gitlab-ci.yml` file. To exclude multiple paths, separate entries using the `;` character. In the provided paths you can use a single character wildcard `?` and `*` for a multiple character wildcard.
To verify the paths are excluded, review the `Tested Operations` and `Excluded Operations` portion of the job output. You should not see any excluded paths listed under `Tested Operations`.
```plaintext
2021-05-27 21:51:08 [INF] API Security: --[ Tested Operations ]-------------------------
2021-05-27 21:51:08 [INF] API Security: 201 POST http://target:7777/api/users CREATED
2021-05-27 21:51:08 [INF] API Security: ------------------------------------------------
2021-05-27 21:51:08 [INF] API Security: --[ Excluded Operations ]-----------------------
2021-05-27 21:51:08 [INF] API Security: GET http://target:7777/api/messages
2021-05-27 21:51:08 [INF] API Security: POST http://target:7777/api/messages
2021-05-27 21:51:08 [INF] API Security: ------------------------------------------------
```
#### Examples
This example excludes the `/auth` resource. This does not exclude child resources (`/auth/child`).
```yaml
variables:
DAST_API_EXCLUDE_PATHS=/auth
```
To exclude `/auth`, and child resources (`/auth/child`), we use a wildcard.
```yaml
variables:
DAST_API_EXCLUDE_PATHS=/auth*
```
To exclude multiple paths we use the `;` character. In this example we exclude `/auth*` and `/v1/*`.
When configured correctly, a CI/CD pipeline contains a `dast` stage and an `dast_api` job. The job only fails when an invalid configuration is provided. During normal operation, the job always succeeds even if vulnerabilities are identified during testing.
Vulnerabilities are displayed on the **Security** pipeline tab with the suite name. When testing against the repositories default branch, the DAST API vulnerabilities are also shown on the Security & Compliance's Vulnerability Report page.
To prevent an excessive number of reported vulnerabilities, the DAST API scanner limits the number of vulnerabilities it reports per operation.
## Viewing DAST API vulnerabilities
The DAST API analyzer produces a JSON report that is collected and used
[to populate the vulnerabilities into GitLab vulnerability screens](#view-details-of-a-dast-api-vulnerability).
See [handling false positives](#handling-false-positives) for information about configuration changes you can make to limit the number of false positives reported.
### View details of a DAST API vulnerability
Follow these steps to view details of a vulnerability:
1. You can view vulnerabilities in a project, or a merge request:
- In a project, go to the project's **{shield}** **Security & Compliance > Vulnerability Report**
page. This page shows all vulnerabilities from the default branch only.
- In a merge request, go the merge request's **Security** section and click the **Expand**
button. DAST API vulnerabilities are available in a section labeled
**DAST detected N potential vulnerabilities**. Click the title to display the vulnerability
details.
1. Click the vulnerabilities title to display the details. The table below describes these details.
| Description | Description of the vulnerability including what was modified. |
| Project | Namespace and project in which the vulnerability was detected. |
| Method | HTTP method used to detect the vulnerability. |
| URL | URL at which the vulnerability was detected. |
| Request | The HTTP request that caused the vulnerability. |
| Unmodified Response | Response from an unmodified request. This is what a normal working response looks like. |
| Actual Response | Response received from test request. |
| Evidence | How we determined a vulnerability occurred. |
| Identifiers | The DAST API check used to find this vulnerability. |
| Severity | Severity of the vulnerability. |
| Scanner Type | Scanner used to perform testing. |
### Security Dashboard
The Security Dashboard is a good place to get an overview of all the security vulnerabilities in your groups, projects and
pipelines. For more information, see the [Security Dashboard documentation](../security_dashboard/index.md).
### Interacting with the vulnerabilities
Once a vulnerability is found, you can interact with it. Read more on how to
[address the vulnerabilities](../vulnerabilities/index.md).
## Handling False Positives
False positives can be handled in several ways:
- Dismiss the vulnerability.
- Some checks have several methods of detecting when a vulnerability is identified, called _Assertions_.
Assertions can also be turned off and configured. For example, the DAST API scanner by default uses HTTP
status codes to help identify when something is a real issue. If an API returns a 500 error during
testing, this creates a vulnerability. This isn't always desired, as some frameworks return 500 errors often.
- Turn off the Check producing the false positive. This prevents the check from generating any
vulnerabilities. Example checks are the SQL Injection Check, and JSON Hijacking Check.
### Turn off a Check
Checks perform testing of a specific type and can be turned on and off for specific configuration
profiles. The provided [configuration files](#configuration-files) define several profiles that you
can use. The profile definition in the configuration file lists all the checks that are active
during a scan. To turn off a specific check, remove it from the profile definition in the
configuration file. The profiles are defined in the `Profiles` section of the configuration file.
Example profile definition:
```yaml
Profiles:
- Name: Quick
DefaultProfile: Empty
Routes:
- Route: *Route0
Checks:
- Name: ApplicationInformationCheck
- Name: CleartextAuthenticationCheck
- Name: FrameworkDebugModeCheck
- Name: HtmlInjectionCheck
- Name: InsecureHttpMethodsCheck
- Name: JsonHijackingCheck
- Name: JsonInjectionCheck
- Name: SensitiveInformationCheck
- Name: SessionCookieCheck
- Name: SqlInjectionCheck
- Name: TokenCheck
- Name: XmlInjectionCheck
```
To turn off the JSON Hijacking Check you can remove these lines:
```yaml
- Name: JsonHijackingCheck
```
This results in the following YAML:
```yaml
- Name: Quick
DefaultProfile: Empty
Routes:
- Route: *Route0
Checks:
- Name: ApplicationInformationCheck
- Name: CleartextAuthenticationCheck
- Name: FrameworkDebugModeCheck
- Name: HtmlInjectionCheck
- Name: InsecureHttpMethodsCheck
- Name: JsonInjectionCheck
- Name: SensitiveInformationCheck
- Name: SessionCookieCheck
- Name: SqlInjectionCheck
- Name: TokenCheck
- Name: XmlInjectionCheck
```
### Turn off an Assertion for a Check
Assertions detect vulnerabilities in tests produced by checks. Many checks support multiple Assertions such as Log Analysis, Response Analysis, and Status Code. When a vulnerability is found, the Assertion used is provided. To identify which Assertions are on by default, see the Checks default configuration in the configuration file. The section is called `Checks`.
This example shows the SQL Injection Check:
```yaml
- Name: SqlInjectionCheck
Configuration:
UserInjections: []
Assertions:
- Name: LogAnalysisAssertion
- Name: ResponseAnalysisAssertion
- Name: StatusCodeAssertion
```
Here you can see three Assertions are on by default. A common source of false positives is
`StatusCodeAssertion`. To turn it off, modify its configuration in the `Profiles` section. This
example provides only the other two Assertions (`LogAnalysisAssertion`,
`ResponseAnalysisAssertion`). This prevents `SqlInjectionCheck` from using `StatusCodeAssertion`:
```yaml
Profiles:
- Name: Quick
DefaultProfile: Empty
Routes:
- Route: *Route0
Checks:
- Name: ApplicationInformationCheck
- Name: CleartextAuthenticationCheck
- Name: FrameworkDebugModeCheck
- Name: HtmlInjectionCheck
- Name: InsecureHttpMethodsCheck
- Name: JsonHijackingCheck
- Name: JsonInjectionCheck
- Name: SensitiveInformationCheck
- Name: SessionCookieCheck
- Name: SqlInjectionCheck
Assertions:
- Name: LogAnalysisAssertion
- Name: ResponseAnalysisAssertion
- Name: TokenCheck
- Name: XmlInjectionCheck
```
## Troubleshooting
### Failed to start scanner session (version header not found)
The DAST API engine outputs an error message when it cannot establish a connection with the scanner application component. The error message is shown in the job output window of the `dast_api` job. A common cause of this issue is changing the `DAST_API_API` variable from its default.
**Error message**
- In [GitLab 13.11 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/323939), `Failed to start scanner session (version header not found).`
- In GitLab 13.10 and earlier, `API Security version header not found. Are you sure that you are connecting to the API Security server?`.
**Solution**
- Remove the `DAST_API_API` variable from the `.gitlab-ci.yml` file. The value will be inherited from the DAST API CI/CD template. We recommend this method instead of manually setting a value.
- If removing the variable is not possible, check to see if this value has changed in the latest version of the [DAST API CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST-API.gitlab-ci.yml). If so, update the value in the `.gitlab-ci.yml` file.
### Application cannot determine the base URL for the target API
The DAST API engine outputs an error message when it cannot determine the target API after inspecting the OpenAPI document. This error message is shown when the target API has not been set in the `.gitlab-ci.yml` file, it is not available in the `environment_url.txt` file, and it could not be computed using the OpenAPI document.
There is a order of precedence in which the DAST API engine tries to get the target API when checking the different sources. First, it will try to use the `DAST_API_TARGET_URL`. If the environment variable has not been set, then the DAST API engine will attempt to use the `environment_url.txt` file. If there is no file `environment_url.txt`, then the DAST API engine will use the OpenAPI document contents and the URL provided in `DAST_API_OPENAPI` (if a URL is provided) to try to compute the target API.
The best-suited solution will depend on whether or not your target API changes for each deployment. In static environments, the target API is the same for each deployment, in this case please refer to the [static environment solution](#static-environment-solution). If the target API changes for each deployment a [dynamic environment solution](#dynamic-environment-solutions) should be applied.
For environments where the target API remains the same, we recommend you specify the target URL by using the `DAST_API_TARGET_URL` environment variable. In your `.gitlab-ci.yml`, add a variable `DAST_API_TARGET_URL`. The variable must be set to the base URL of API testing target. For example:
In a dynamic environment your target API changes for each different deployment. In this case, there is more than one possible solution, we recommend you use the `environment_url.txt` file when dealing with dynamic environments.
To support dynamic environments in which the target API URL changes during each pipeline, DAST API engine supports the use of an `environment_url.txt` file that contains the URL to use. This file is not checked into the repository, instead it's created during the pipeline by the job that deploys the test target and collected as an artifact that can be used by later jobs in the pipeline. The job that creates the `environment_url.txt` file must run before the DAST API engine job.
1. Modify the test target deployment job adding the base URL in an `environment_url.txt` file at the root of your project.
1. Modify the test target deployment job collecting the `environment_url.txt` as an artifact.
- Assert: Assertions are detection modules used by checks to trigger a vulnerability. Many assertions have
configurations. A check can use multiple Assertions. For example, Log Analysis, Response Analysis,
and Status Code are common Assertions used together by checks. Checks with multiple Assertions
allow them to be turned on and off.
- Check: Performs a specific type of test, or performed a check for a type of vulnerability. For
example, the SQL Injection Check performs DAST testing for SQL Injection vulnerabilities. The DAST API scanner is comprised of several checks. Checks can be turned on and off in a profile.
- Profile: A configuration file has one or more testing profiles, or sub-configurations. You may
have a profile for feature branches and another with extra testing for a main branch.
