debian-mirror-gitlab/doc/development/go_guide/index.md

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

568 lines
23 KiB
Markdown
Raw Normal View History

2021-01-29 00:20:46 +05:30
---
stage: none
group: unassigned
2022-11-25 23:54:43 +05:30
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
2021-01-29 00:20:46 +05:30
---
2019-03-02 22:35:43 +05:30
# Go standards and style guidelines
This document describes various guidelines and best practices for GitLab
2022-01-26 12:08:38 +05:30
projects using the [Go language](https://go.dev/).
2019-03-02 22:35:43 +05:30
## Overview
GitLab is built on top of [Ruby on Rails](https://rubyonrails.org/), but we're
also using Go for projects where it makes sense. Go is a very powerful
language, with many advantages, and is best suited for projects with a lot of
2021-09-30 23:02:18 +05:30
IO (disk/network access), HTTP requests, parallel processing, and so on. Since we
2019-03-02 22:35:43 +05:30
have both Ruby on Rails and Go at GitLab, we should evaluate carefully which of
the two is best for the job.
This page aims to define and organize our Go guidelines, based on our various
experiences. Several projects were started with different standards and they
2021-02-22 17:27:13 +05:30
can still have specifics. They are described in their respective
2019-03-02 22:35:43 +05:30
`README.md` or `PROCESS.md` files.
2021-11-18 22:05:49 +05:30
## Go language versions
The Go upgrade documentation [provides an overview](go_upgrade.md#overview)
of how GitLab manages and ships Go binary support.
If a GitLab component requires a newer version of Go, please
follow the [upgrade process](go_upgrade.md#updating-go-version) to ensure no customer, team, or component is adversely impacted.
Sometimes, individual projects must also [manage builds with multiple versions of Go](go_upgrade.md#supporting-multiple-go-versions).
2020-06-23 00:09:42 +05:30
## Dependency Management
Go uses a source-based strategy for dependency management. Dependencies are
downloaded as source from their source repository. This differs from the more
common artifact-based strategy where dependencies are downloaded as artifacts
from a package repository that is separate from the dependency's source
repository.
Go did not have first-class support for version management prior to 1.11. That
version introduced Go modules and the use of semantic versioning. Go 1.12
introduced module proxies, which can serve as an intermediate between clients
and source version control systems, and checksum databases, which can be used to
verify the integrity of dependency downloads.
See [Dependency Management in Go](dependencies.md) for more details.
2019-03-02 22:35:43 +05:30
## Code Review
We follow the common principles of
[Go Code Review Comments](https://github.com/golang/go/wiki/CodeReviewComments).
Reviewers and maintainers should pay attention to:
- `defer` functions: ensure the presence when needed, and after `err` check.
- Inject dependencies as parameters.
2019-07-07 11:18:12 +05:30
- Void structs when marshaling to JSON (generates `null` instead of `[]`).
2019-03-02 22:35:43 +05:30
### Security
Security is our top priority at GitLab. During code reviews, we must take care
of possible security breaches in our code:
- XSS when using text/template
- CSRF Protection using Gorilla
- Use a Go version without known vulnerabilities
- Don't leak secret tokens
- SQL injections
Remember to run
2020-03-13 15:44:24 +05:30
[SAST](../../user/application_security/sast/index.md) and [Dependency Scanning](../../user/application_security/dependency_scanning/index.md)
2021-03-11 19:13:27 +05:30
**(ULTIMATE)** on your project (or at least the
[`gosec` analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/gosec)),
2021-11-11 11:23:49 +05:30
and to follow our [Security requirements](../code_review.md#security).
2019-03-02 22:35:43 +05:30
Web servers can take advantages of middlewares like [Secure](https://github.com/unrolled/secure).
### Finding a reviewer
Many of our projects are too small to have full-time maintainers. That's why we
have a shared pool of Go reviewers at GitLab. To find a reviewer, use the
2020-03-13 15:44:24 +05:30
["Go" section](https://about.gitlab.com/handbook/engineering/projects/#gitlab_reviewers_go)
of the "GitLab" project on the Engineering Projects
page in the handbook.
2019-03-02 22:35:43 +05:30
To add yourself to this list, add the following to your profile in the
2022-07-23 23:45:48 +05:30
[`team.yml`](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/team.yml)
2019-03-02 22:35:43 +05:30
file and ask your manager to review and merge.
```yaml
projects:
2019-12-21 20:55:43 +05:30
gitlab: reviewer go
2019-03-02 22:35:43 +05:30
```
## Code style and format
2021-02-22 17:27:13 +05:30
- Avoid global variables, even in packages. By doing so you introduce side
2019-03-02 22:35:43 +05:30
effects if the package is included multiple times.
2020-05-24 23:13:21 +05:30
- Use `goimports` before committing.
2021-04-17 20:07:23 +05:30
[`goimports`](https://pkg.go.dev/golang.org/x/tools/cmd/goimports)
2020-05-24 23:13:21 +05:30
is a tool that automatically formats Go source code using
2022-01-26 12:08:38 +05:30
[`Gofmt`](https://pkg.go.dev/cmd/gofmt), in addition to formatting import lines,
2020-05-24 23:13:21 +05:30
adding missing ones and removing unreferenced ones.
2021-02-22 17:27:13 +05:30
Most editors/IDEs allow you to run commands before/after saving a file, you can set it
2020-05-24 23:13:21 +05:30
up to run `goimports` so that it's applied to every file when saving.
2020-03-13 15:44:24 +05:30
- Place private methods below the first caller method in the source file.
2019-03-02 22:35:43 +05:30
### Automatic linting
All Go projects should include these GitLab CI/CD jobs:
```yaml
2020-03-13 15:44:24 +05:30
lint:
image: registry.gitlab.com/gitlab-org/gitlab-build-images:golangci-lint-alpine
stage: test
2019-03-02 22:35:43 +05:30
script:
2020-03-13 15:44:24 +05:30
# Use default .golangci.yml file from the image if one is not present in the project root.
- '[ -e .golangci.yml ] || cp /golangci/.golangci.yml .'
# Write the code coverage report to gl-code-quality-report.json
# and print linting issues to stdout in the format: path/to/file:line description
2020-10-24 23:57:45 +05:30
# remove `--issues-exit-code 0` or set to non-zero to fail the job if linting issues are detected
- golangci-lint run --issues-exit-code 0 --out-format code-climate | tee gl-code-quality-report.json | jq -r '.[] | "\(.location.path):\(.location.lines.begin) \(.description)"'
2020-03-13 15:44:24 +05:30
artifacts:
reports:
codequality: gl-code-quality-report.json
paths:
- gl-code-quality-report.json
2019-03-02 22:35:43 +05:30
```
2020-03-13 15:44:24 +05:30
Including a `.golangci.yml` in the root directory of the project allows for
configuration of `golangci-lint`. All options for `golangci-lint` are listed in
this [example](https://github.com/golangci/golangci-lint/blob/master/.golangci.example.yml).
2020-06-23 00:09:42 +05:30
Once [recursive includes](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/56836)
2021-02-22 17:27:13 +05:30
become available, you can share job templates like this
2019-03-02 22:35:43 +05:30
[analyzer](https://gitlab.com/gitlab-org/security-products/ci-templates/raw/master/includes-dev/analyzer.yml).
2021-02-22 17:27:13 +05:30
Go GitLab linter plugins are maintained in the [`gitlab-org/language-tools/go/linters`](https://gitlab.com/gitlab-org/language-tools/go/linters/) namespace.
2019-03-02 22:35:43 +05:30
## Dependencies
Dependencies should be kept to the minimum. The introduction of a new
2022-10-11 01:57:18 +05:30
dependency should be argued in the merge request, as per our [Approval Guidelines](../code_review.md#approval-guidelines).
2022-08-27 11:52:29 +05:30
Both [License Scanning](../../user/compliance/license_compliance/index.md)
and [Dependency Scanning](../../user/application_security/dependency_scanning/index.md)
should be activated on all projects to ensure new dependencies
2019-03-02 22:35:43 +05:30
security status and license compatibility.
### Modules
2022-10-11 01:57:18 +05:30
In Go 1.11 and later, a standard dependency system is available behind the name
2022-08-27 11:52:29 +05:30
[Go Modules](https://github.com/golang/go/wiki/Modules). It provides a way to
2019-03-02 22:35:43 +05:30
define and lock dependencies for reproducible builds. It should be used
whenever possible.
2019-10-12 21:52:04 +05:30
When Go Modules are in use, there should not be a `vendor/` directory. Instead,
2021-02-22 17:27:13 +05:30
Go automatically downloads dependencies when they are needed to build the
2019-10-12 21:52:04 +05:30
project. This is in line with how dependencies are handled with Bundler in Ruby
projects, and makes merge requests easier to review.
In some cases, such as building a Go project for it to act as a dependency of a
CI run for another project, removing the `vendor/` directory means the code must
be downloaded repeatedly, which can lead to intermittent problems due to rate
2022-10-11 01:57:18 +05:30
limiting or network failures. In these circumstances, you should
2022-08-27 11:52:29 +05:30
[cache the downloaded code between](../../ci/caching/index.md#cache-go-dependencies).
2019-10-12 21:52:04 +05:30
2021-03-11 19:13:27 +05:30
There was a
[bug on modules checksums](https://github.com/golang/go/issues/29278) in Go versions earlier than v1.11.4, so make
2019-03-02 22:35:43 +05:30
sure to use at least this version to avoid `checksum mismatch` errors.
### ORM
We don't use object-relational mapping libraries (ORMs) at GitLab (except
[ActiveRecord](https://guides.rubyonrails.org/active_record_basics.html) in
Ruby on Rails). Projects can be structured with services to avoid them.
2021-03-11 19:13:27 +05:30
[`pgx`](https://github.com/jackc/pgx) should be enough to interact with PostgreSQL
2019-03-02 22:35:43 +05:30
databases.
### Migrations
In the rare event of managing a hosted database, it's necessary to use a
migration system like ActiveRecord is providing. A simple library like
[Journey](https://github.com/db-journey/journey), designed to be used in
2021-02-22 17:27:13 +05:30
`postgres` containers, can be deployed as long-running pods. New versions
2019-03-02 22:35:43 +05:30
deploy a new pod, migrating the data automatically.
## Testing
2019-09-30 21:07:59 +05:30
### Testing frameworks
2019-03-02 22:35:43 +05:30
We should not use any specific library or framework for testing, as the
2022-01-26 12:08:38 +05:30
[standard library](https://pkg.go.dev/std) provides already everything to get
2019-09-30 21:07:59 +05:30
started. If there is a need for more sophisticated testing tools, the following
external dependencies might be worth considering in case we decide to use a specific
library or framework:
2019-03-02 22:35:43 +05:30
- [Testify](https://github.com/stretchr/testify)
2021-03-11 19:13:27 +05:30
- [`httpexpect`](https://github.com/gavv/httpexpect)
2019-03-02 22:35:43 +05:30
2019-09-30 21:07:59 +05:30
### Subtests
2021-11-11 11:23:49 +05:30
Use [subtests](https://go.dev/blog/subtests) whenever possible to improve
2019-03-02 22:35:43 +05:30
code readability and test output.
2019-09-30 21:07:59 +05:30
### Better output in tests
When comparing expected and actual values in tests, use
2021-04-17 20:07:23 +05:30
[`testify/require.Equal`](https://pkg.go.dev/github.com/stretchr/testify/require#Equal),
[`testify/require.EqualError`](https://pkg.go.dev/github.com/stretchr/testify/require#EqualError),
[`testify/require.EqualValues`](https://pkg.go.dev/github.com/stretchr/testify/require#EqualValues),
2019-09-30 21:07:59 +05:30
and others to improve readability when comparing structs, errors,
large portions of text, or JSON documents:
2020-04-08 14:13:33 +05:30
```golang
2019-09-30 21:07:59 +05:30
type TestData struct {
// ...
}
func FuncUnderTest() TestData {
// ...
}
func Test(t *testing.T) {
t.Run("FuncUnderTest", func(t *testing.T) {
want := TestData{}
got := FuncUnderTest()
require.Equal(t, want, got) // note that expected value comes first, then comes the actual one ("diff" semantics)
})
}
```
2019-10-12 21:52:04 +05:30
### Table-Driven Tests
Using [Table-Driven Tests](https://github.com/golang/go/wiki/TableDrivenTests)
is generally good practice when you have multiple entries of
inputs/outputs for the same function. Below are some guidelines one can
follow when writing table-driven test. These guidelines are mostly
extracted from Go standard library source code. Keep in mind it's OK not
to follow these guidelines when it makes sense.
#### Defining test cases
Each table entry is a complete test case with inputs and expected
results, and sometimes with additional information such as a test name
to make the test output easily readable.
- [Define a slice of anonymous struct](https://github.com/golang/go/blob/50bd1c4d4eb4fac8ddeb5f063c099daccfb71b26/src/encoding/csv/reader_test.go#L16)
inside of the test.
- [Define a slice of anonymous struct](https://github.com/golang/go/blob/55d31e16c12c38d36811bdee65ac1f7772148250/src/cmd/go/internal/module/module_test.go#L9-L66)
outside of the test.
- [Named structs](https://github.com/golang/go/blob/2e0cd2aef5924e48e1ceb74e3d52e76c56dd34cc/src/cmd/go/internal/modfetch/coderepo_test.go#L54-L69)
for code reuse.
- [Using `map[string]struct{}`](https://github.com/golang/go/blob/6d5caf38e37bf9aeba3291f1f0b0081f934b1187/src/cmd/trace/annotations_test.go#L180-L235).
#### Contents of the test case
- Ideally, each test case should have a field with a unique identifier
to use for naming subtests. In the Go standard library, this is commonly the
`name string` field.
2020-04-22 19:07:51 +05:30
- Use `want`/`expect`/`actual` when you are specifying something in the
2021-02-22 17:27:13 +05:30
test case that is used for assertion.
2019-10-12 21:52:04 +05:30
#### Variable names
- Each table-driven test map/slice of struct can be named `tests`.
- When looping through `tests` the anonymous struct can be referred
to as `tt` or `tc`.
- The description of the test can be referred to as
`name`/`testName`/`tn`.
2019-03-02 22:35:43 +05:30
### Benchmarks
Programs handling a lot of IO or complex operations should always include
2022-01-26 12:08:38 +05:30
[benchmarks](https://pkg.go.dev/testing#hdr-Benchmarks), to ensure
2019-03-02 22:35:43 +05:30
performance consistency over time.
2020-05-24 23:13:21 +05:30
## Error handling
### Adding context
Adding context before you return the error can be helpful, instead of
just returning the error. This allows developers to understand what the
program was trying to do when it entered the error state making it much
easier to debug.
For example:
2020-06-23 00:09:42 +05:30
```golang
2020-05-24 23:13:21 +05:30
// Wrap the error
return nil, fmt.Errorf("get cache %s: %w", f.Name, err)
// Just add context
return nil, fmt.Errorf("saving cache %s: %v", f.Name, err)
```
A few things to keep in mind when adding context:
- Decide if you want to expose the underlying error
to the caller. If so, use `%w`, if not, you can use `%v`.
- Don't use words like `failed`, `error`, `didn't`. As it's an error,
the user already knows that something failed and this might lead to
having strings like `failed xx failed xx failed xx`. Explain _what_
failed instead.
- Error strings should not be capitalized or end with punctuation or a
newline. You can use `golint` to check for this.
### Naming
- When using sentinel errors they should always be named like `ErrXxx`.
- When creating a new error type they should always be named like
`XxxError`.
### Checking Error types
- To check error equality don't use `==`. Use
[`errors.Is`](https://pkg.go.dev/errors?tab=doc#Is) instead (for Go
versions >= 1.13).
- To check if the error is of a certain type don't use type assertion,
use [`errors.As`](https://pkg.go.dev/errors?tab=doc#As) instead (for
Go versions >= 1.13).
### References for working with errors
2021-11-11 11:23:49 +05:30
- [Go 1.13 errors](https://go.dev/blog/go1.13-errors).
2022-08-27 11:52:29 +05:30
- [Programing with errors](https://peter.bourgon.org/blog/2019/09/11/programming-with-errors.html).
- [Don't just check errors, handle them gracefully](https://dave.cheney.net/2016/04/27/dont-just-check-errors-handle-them-gracefully).
2020-05-24 23:13:21 +05:30
2019-03-02 22:35:43 +05:30
## CLIs
Every Go program is launched from the command line.
2021-03-11 19:13:27 +05:30
[`cli`](https://github.com/urfave/cli) is a convenient package to create command
line apps. It should be used whether the project is a daemon or a simple CLI
2022-08-27 11:52:29 +05:30
tool. Flags can be mapped to [environment variables](https://github.com/urfave/cli#values-from-the-environment) directly,
2019-03-02 22:35:43 +05:30
which documents and centralizes at the same time all the possible command line
interactions with the program. Don't use `os.GetEnv`, it hides variables deep
in the code.
## Daemons
### Logging
2019-07-07 11:18:12 +05:30
The usage of a logging library is strongly recommended for daemons. Even
though there is a `log` package in the standard library, we generally use
[Logrus](https://github.com/sirupsen/logrus). Its plugin ("hooks") system
2019-03-02 22:35:43 +05:30
makes it a powerful logging library, with the ability to add notifiers and
formatters at the logger level directly.
2019-07-07 11:18:12 +05:30
#### Structured (JSON) logging
Every binary ideally must have structured (JSON) logging in place as it helps
with searching and filtering the logs. At GitLab we use structured logging in
JSON format, as all our infrastructure assumes that. When using
[Logrus](https://github.com/sirupsen/logrus) you can turn on structured
2022-11-25 23:54:43 +05:30
logging by using the build in [JSON formatter](https://github.com/sirupsen/logrus#formatters). This follows the
2021-12-11 22:18:48 +05:30
same logging type we use in our [Ruby applications](../logging.md#use-structured-json-logging).
2019-07-07 11:18:12 +05:30
#### How to use Logrus
There are a few guidelines one should follow when using the
[Logrus](https://github.com/sirupsen/logrus) package:
- When printing an error use
2021-04-17 20:07:23 +05:30
[WithError](https://pkg.go.dev/github.com/sirupsen/logrus#WithError). For
2019-07-07 11:18:12 +05:30
example, `logrus.WithError(err).Error("Failed to do something")`.
- Since we use [structured logging](#structured-json-logging) we can log
fields in the context of that code path, such as the URI of the request using
2021-04-17 20:07:23 +05:30
[`WithField`](https://pkg.go.dev/github.com/sirupsen/logrus#WithField) or
[`WithFields`](https://pkg.go.dev/github.com/sirupsen/logrus#WithFields). For
2020-03-13 15:44:24 +05:30
example, `logrus.WithField("file", "/app/go").Info("Opening dir")`. If you
2019-07-07 11:18:12 +05:30
have to log multiple keys, always use `WithFields` instead of calling
`WithField` more than once.
2019-03-02 22:35:43 +05:30
### Tracing and Correlation
[LabKit](https://gitlab.com/gitlab-org/labkit) is a place to keep common
libraries for Go services. Currently it's vendored into two projects:
Workhorse and Gitaly, and it exports two main (but related) pieces of
functionality:
- [`gitlab.com/gitlab-org/labkit/correlation`](https://gitlab.com/gitlab-org/labkit/tree/master/correlation):
for propagating and extracting correlation ids between services.
- [`gitlab.com/gitlab-org/labkit/tracing`](https://gitlab.com/gitlab-org/labkit/tree/master/tracing):
for instrumenting Go libraries for distributed tracing.
This gives us a thin abstraction over underlying implementations that is
consistent across Workhorse, Gitaly, and, in future, other Go servers. For
example, in the case of `gitlab.com/gitlab-org/labkit/tracing` we can switch
2021-03-11 19:13:27 +05:30
from using `Opentracing` directly to using `Zipkin` or Gokit's own tracing wrapper
2019-03-02 22:35:43 +05:30
without changes to the application code, while still keeping the same
2021-09-30 23:02:18 +05:30
consistent configuration mechanism (that is, the `GITLAB_TRACING` environment
2019-03-02 22:35:43 +05:30
variable).
### Context
Since daemons are long-running applications, they should have mechanisms to
manage cancellations, and avoid unnecessary resources consumption (which could
2022-07-23 23:45:48 +05:30
lead to DDoS vulnerabilities). [Go Context](https://github.com/golang/go/wiki/CodeReviewComments#contexts)
should be used in functions that can block and passed as the first parameter.
2019-03-02 22:35:43 +05:30
## Dockerfiles
Every project should have a `Dockerfile` at the root of their repository, to
build and run the project. Since Go program are static binaries, they should
not require any external dependency, and shells in the final image are useless.
2022-08-27 11:52:29 +05:30
We encourage [Multistage builds](https://docs.docker.com/develop/develop-images/multistage-build/):
2019-03-02 22:35:43 +05:30
- They let the user build the project with the right Go version and
dependencies.
- They generate a small, self-contained image, derived from `Scratch`.
2020-06-23 00:09:42 +05:30
Generated Docker images should have the program at their `Entrypoint` to create
2019-03-02 22:35:43 +05:30
portable commands. That way, anyone can run the image, and without parameters
2021-02-22 17:27:13 +05:30
it displays its help message (if `cli` has been used).
2019-03-02 22:35:43 +05:30
2020-04-22 19:07:51 +05:30
## Secure Team standards and style guidelines
The following are some style guidelines that are specific to the Secure Team.
### Code style and format
Use `goimports -local gitlab.com/gitlab-org` before committing.
2021-04-17 20:07:23 +05:30
[`goimports`](https://pkg.go.dev/golang.org/x/tools/cmd/goimports)
2020-04-22 19:07:51 +05:30
is a tool that automatically formats Go source code using
2022-01-26 12:08:38 +05:30
[`Gofmt`](https://pkg.go.dev/cmd/gofmt), in addition to formatting import lines,
2020-04-22 19:07:51 +05:30
adding missing ones and removing unreferenced ones.
2021-02-22 17:27:13 +05:30
By using the `-local gitlab.com/gitlab-org` option, `goimports` groups locally referenced
2020-04-22 19:07:51 +05:30
packages separately from external ones. See
[the imports section](https://github.com/golang/go/wiki/CodeReviewComments#imports)
of the Code Review Comments page on the Go wiki for more details.
2021-02-22 17:27:13 +05:30
Most editors/IDEs allow you to run commands before/after saving a file, you can set it
2020-04-22 19:07:51 +05:30
up to run `goimports -local gitlab.com/gitlab-org` so that it's applied to every file when saving.
2022-11-25 23:54:43 +05:30
### Naming branches
Only use the characters `a-z`, `0-9` or `-` in branch names. This restriction is due to the fact that `go get` doesn't work as expected when a branch name contains certain characters, such as a slash `/`:
```shell
$ go get -u gitlab.com/gitlab-org/security-products/analyzers/report/v3@some-user/some-feature
go get: gitlab.com/gitlab-org/security-products/analyzers/report/v3@some-user/some-feature: invalid version: version "some-user/some-feature" invalid: disallowed version string
```
If a branch name contains a slash, it forces us to refer to the commit SHA instead, which is less flexible. For example:
```shell
$ go get -u gitlab.com/gitlab-org/security-products/analyzers/report/v3@5c9a4279fa1263755718cf069d54ba8051287954
go: downloading gitlab.com/gitlab-org/security-products/analyzers/report/v3 v3.15.3-0.20221012172609-5c9a4279fa12
...
```
2022-08-13 15:12:31 +05:30
### Initializing slices
If initializing a slice, provide a capacity where possible to avoid extra
allocations.
2022-08-27 11:52:29 +05:30
**Don't:**
2022-08-13 15:12:31 +05:30
2022-08-27 11:52:29 +05:30
```golang
var s2 []string
for _, val := range s1 {
s2 = append(s2, val)
}
```
2022-08-13 15:12:31 +05:30
2022-08-27 11:52:29 +05:30
**Do:**
2022-08-13 15:12:31 +05:30
2022-08-27 11:52:29 +05:30
```golang
s2 := make([]string, 0, size)
for _, val := range s1 {
s2 = append(s2, val)
}
```
2022-08-13 15:12:31 +05:30
If no capacity is passed to `make` when creating a new slice, `append`
will continuously resize the slice's backing array if it cannot hold
the values. Providing the capacity ensures that allocations are kept
2022-08-27 11:52:29 +05:30
to a minimum. It's recommended that the [`prealloc`](https://github.com/alexkohler/prealloc)
2022-08-13 15:12:31 +05:30
golanci-lint rule automatically check for this.
2021-04-29 21:17:54 +05:30
### Analyzer Tests
2021-06-08 01:23:25 +05:30
The conventional Secure [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/) has a [`convert` function](https://gitlab.com/gitlab-org/security-products/analyzers/command/-/blob/main/convert.go#L15-17) that converts SAST/DAST scanner reports into [GitLab Security Reports](https://gitlab.com/gitlab-org/security-products/security-report-schemas). When writing tests for the `convert` function, we should make use of [test fixtures](https://dave.cheney.net/2016/05/10/test-fixtures-in-go) using a `testdata` directory at the root of the analyzer's repository. The `testdata` directory should contain two subdirectories: `expect` and `reports`. The `reports` directory should contain sample SAST/DAST scanner reports which are passed into the `convert` function during the test setup. The `expect` directory should contain the expected GitLab Security Report that the `convert` returns. See Secret Detection for an [example](https://gitlab.com/gitlab-org/security-products/analyzers/secrets/-/blob/160424589ef1eed7b91b59484e019095bc7233bd/convert_test.go#L13-66).
2021-04-29 21:17:54 +05:30
If the scanner report is small, less than 35 lines, then feel free to [inline the report](https://gitlab.com/gitlab-org/security-products/analyzers/sobelow/-/blob/8bd2428a/convert/convert_test.go#L13-77) rather than use a `testdata` directory.
2021-12-11 22:18:48 +05:30
#### Test Diffs
2022-01-26 12:08:38 +05:30
The [go-cmp](https://github.com/google/go-cmp) package should be used when comparing large structs in tests. It makes it possible to output a specific diff where the two structs differ, rather than seeing the whole of both structs printed out in the test logs. Here is a small example:
2021-12-11 22:18:48 +05:30
```golang
package main
import (
"reflect"
"testing"
"github.com/google/go-cmp/cmp"
)
type Foo struct {
Desc Bar
Point Baz
}
type Bar struct {
A string
B string
}
type Baz struct {
X int
Y int
}
func TestHelloWorld(t *testing.T) {
want := Foo{
Desc: Bar{A: "a", B: "b"},
Point: Baz{X: 1, Y: 2},
}
got := Foo{
Desc: Bar{A: "a", B: "b"},
Point: Baz{X: 2, Y: 2},
}
t.Log("reflect comparison:")
if !reflect.DeepEqual(got, want) {
t.Errorf("Wrong result. want:\n%v\nGot:\n%v", want, got)
}
t.Log("cmp comparison:")
if diff := cmp.Diff(want, got); diff != "" {
t.Errorf("Wrong result. (-want +got):\n%s", diff)
}
}
```
The output demonstrates why `go-cmp` is far superior when comparing large structs. Even though you could spot the difference with this small difference, it quickly gets unwieldy as the data grows.
```plaintext
main_test.go:36: reflect comparison:
main_test.go:38: Wrong result. want:
{{a b} {1 2}}
Got:
{{a b} {2 2}}
main_test.go:41: cmp comparison:
main_test.go:43: Wrong result. (-want +got):
main.Foo{
Desc: {A: "a", B: "b"},
Point: main.Baz{
- X: 1,
+ X: 2,
Y: 2,
},
}
```
2019-03-02 22:35:43 +05:30
---
2021-09-30 23:02:18 +05:30
[Return to Development documentation](../index.md).