debian-mirror-gitlab/doc/development/shell_scripting_guide/index.md

131 lines
4.9 KiB
Markdown
Raw Normal View History

2021-01-29 00:20:46 +05:30
---
stage: none
group: unassigned
2021-02-22 17:27:13 +05:30
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
2021-01-29 00:20:46 +05:30
---
2019-10-12 21:52:04 +05:30
# Shell scripting standards and style guidelines
GitLab consists of many various services and sub-projects. The majority of
their backend code is written in [Ruby](https://www.ruby-lang.org) and
[Go](https://golang.org). However, some of them use shell scripts for
automation of routine system administration tasks like deployment,
installation, etc. It's being done either for historical reasons or as an effort
to minimize the dependencies, for instance, for Docker images.
This page aims to define and organize our shell scripting guidelines,
based on our various experiences. All shell scripts across GitLab project
should be eventually harmonized with this guide. If there are any per-project
deviations from this guide, they should be described in the
`README.md` or `PROCESS.md` file for such a project.
2021-01-03 14:25:43 +05:30
## Avoid using shell scripts
2019-10-12 21:52:04 +05:30
2021-02-22 17:27:13 +05:30
WARNING:
2019-10-12 21:52:04 +05:30
This is a must-read section.
Having said all of the above, we recommend staying away from shell scripts
as much as possible. A language like Ruby or Python (if required for
consistency with codebases that we leverage) is almost always a better choice.
The high-level interpreted languages have more readable syntax, offer much more
2019-12-04 20:38:33 +05:30
mature capabilities for unit-testing, linting, and error reporting.
2019-10-12 21:52:04 +05:30
Use shell scripts only if there's a strong restriction on project's
dependencies size or any other requirements that are more important
in a particular case.
## Scope of this guide
According to the [GitLab installation requirements](../../install/requirements.md),
this guide covers only those shells that are used by
[supported Linux distributions](../../install/requirements.md#supported-linux-distributions),
that is:
- [POSIX Shell](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html)
- [Bash](https://www.gnu.org/software/bash/)
## Shell language choice
- When you need to reduce the dependencies list, use what's provided by the environment. For example, for Docker images it's `sh` from `alpine` which is the base image for most of our tool images.
- Everywhere else, use `bash` if possible. It's more powerful than `sh` but still a widespread shell.
## Code style and format
This section describes the tools that should be made a mandatory part of
a project's CI pipeline if it contains shell scripts. These tools
2019-12-04 20:38:33 +05:30
automate shell code formatting, checking for errors or vulnerabilities, etc.
2019-10-12 21:52:04 +05:30
### Linting
We're using the [ShellCheck](https://www.shellcheck.net/) utility in its default configuration to lint our
2019-12-04 20:38:33 +05:30
shell scripts.
2019-10-12 21:52:04 +05:30
All projects with shell scripts should use this GitLab CI/CD job:
```yaml
shell check:
2020-01-01 13:55:28 +05:30
image: koalaman/shellcheck-alpine:stable
2019-10-12 21:52:04 +05:30
stage: test
before_script:
- shellcheck --version
script:
2020-11-24 15:15:51 +05:30
- shellcheck scripts/**/*.sh # path to your shell scripts
2019-10-12 21:52:04 +05:30
```
2021-02-22 17:27:13 +05:30
NOTE:
By default, ShellCheck uses the [shell detection](https://github.com/koalaman/shellcheck/wiki/SC2148#rationale)
2019-10-12 21:52:04 +05:30
to determine the shell dialect in use. If the shell file is out of your control and ShellCheck cannot
detect the dialect, use `-s` flag to specify it: `-s sh` or `-s bash`.
### Formatting
It's recommended to use the [shfmt](https://github.com/mvdan/sh#shfmt) tool to maintain consistent formatting.
We format shell scripts according to the [Google Shell Style Guide](https://google.github.io/styleguide/shell.xml),
so the following `shfmt` invocation should be applied to the project's script files:
2020-03-13 15:44:24 +05:30
```shell
2020-05-24 23:13:21 +05:30
shfmt -i 2 -ci -w scripts/**/*.sh
```
In addition to the [Linting](#linting) GitLab CI/CD job, all projects with shell scripts should also
use this job:
```yaml
shfmt:
2021-02-22 17:27:13 +05:30
image: mvdan/shfmt:v3.2.0-alpine
2020-05-24 23:13:21 +05:30
stage: test
before_script:
- shfmt -version
script:
2020-11-24 15:15:51 +05:30
- shfmt -i 2 -ci -d scripts # path to your shell scripts
2019-10-12 21:52:04 +05:30
```
2021-02-22 17:27:13 +05:30
NOTE:
By default, shfmt uses the [shell detection](https://github.com/mvdan/sh#shfmt) similar to one of ShellCheck
2019-10-12 21:52:04 +05:30
and ignore files starting with a period. To override this, use `-ln` flag to specify the shell dialect:
`-ln posix` or `-ln bash`.
## Testing
2021-02-22 17:27:13 +05:30
NOTE:
2019-10-12 21:52:04 +05:30
This is a work in progress.
2020-06-23 00:09:42 +05:30
It is an [ongoing effort](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/64016) to evaluate different tools for the
2020-03-13 15:44:24 +05:30
automated testing of shell scripts (like [BATS](https://github.com/bats-core/bats-core)).
2019-10-12 21:52:04 +05:30
## Code Review
The code review should be performed according to:
- [ShellCheck Checks list](https://github.com/koalaman/shellcheck/wiki/Checks)
- [Google Shell Style Guide](https://google.github.io/styleguide/shell.xml)
- [Shfmt formatting caveats](https://github.com/mvdan/sh#caveats)
However, the recommended course of action is to use the aforementioned
tools and address reported offenses. This should eliminate the need
for code review.
---
[Return to Development documentation](../README.md).