debian-mirror-gitlab/doc/development/testing_guide/testing_levels.md

549 lines
23 KiB
Markdown
Raw Normal View History

2021-01-29 00:20:46 +05:30
---
stage: none
group: unassigned
2022-11-25 23:54:43 +05:30
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
2021-01-29 00:20:46 +05:30
---
2018-03-17 18:26:18 +05:30
# Testing levels
![Testing priority triangle](img/testing_triangle.png)
_This diagram demonstrates the relative priority of each test type we use. `e2e` stands for end-to-end._
2019-09-04 21:01:54 +05:30
As of 2019-05-01, we have the following distribution of tests per level:
2019-07-31 22:56:46 +05:30
2019-09-04 21:01:54 +05:30
| Test level | Community Edition | Enterprise Edition | Community + Enterprise Edition |
| --------- | ---------- | -------------- | ----- |
| Black-box tests at the system level (aka end-to-end or QA tests) | 68 (0.14%) | 31 (0.2%) | 99 (0.17%) |
| White-box tests at the system level (aka system or feature tests) | 5,471 (11.9%) | 969 (7.4%) | 6440 (10.9%) |
| Integration tests | 8,333 (18.2%) | 2,244 (17.2%) | 10,577 (17.9%) |
| Unit tests | 32,031 (69.7%) | 9,778 (75.1%) | 41,809 (71%) |
2019-07-31 22:56:46 +05:30
2018-03-17 18:26:18 +05:30
## Unit tests
2019-03-02 22:35:43 +05:30
Formal definition: <https://en.wikipedia.org/wiki/Unit_testing>
2018-03-17 18:26:18 +05:30
These kind of tests ensure that a single unit of code (a method) works as
expected (given an input, it has a predictable output). These tests should be
isolated as much as possible. For example, model methods that don't do anything
with the database shouldn't need a DB record. Classes that don't need database
records should use stubs/doubles as much as possible.
| Code path | Tests path | Testing engine | Notes |
| --------- | ---------- | -------------- | ----- |
2021-11-11 11:23:49 +05:30
| `app/assets/javascripts/` | `spec/frontend/` | Jest | More details in the [Frontend Testing guide](frontend_testing.md) section. |
2018-03-17 18:26:18 +05:30
| `app/finders/` | `spec/finders/` | RSpec | |
2019-07-31 22:56:46 +05:30
| `app/graphql/` | `spec/graphql/` | RSpec | |
2018-03-17 18:26:18 +05:30
| `app/helpers/` | `spec/helpers/` | RSpec | |
2019-07-31 22:56:46 +05:30
| `app/models/` | `spec/models/` | RSpec | |
2018-03-17 18:26:18 +05:30
| `app/policies/` | `spec/policies/` | RSpec | |
| `app/presenters/` | `spec/presenters/` | RSpec | |
| `app/serializers/` | `spec/serializers/` | RSpec | |
| `app/services/` | `spec/services/` | RSpec | |
| `app/uploaders/` | `spec/uploaders/` | RSpec | |
2019-07-31 22:56:46 +05:30
| `app/validators/` | `spec/validators/` | RSpec | |
2018-03-17 18:26:18 +05:30
| `app/views/` | `spec/views/` | RSpec | |
| `app/workers/` | `spec/workers/` | RSpec | |
2019-07-31 22:56:46 +05:30
| `bin/` | `spec/bin/` | RSpec | |
| `config/` | `spec/config/` | RSpec | |
| `config/initializers/` | `spec/initializers/` | RSpec | |
| `config/routes.rb`, `config/routes/` | `spec/routing/` | RSpec | |
2021-09-04 01:27:46 +05:30
| `config/puma.example.development.rb` | `spec/rack_servers/` | RSpec | |
2019-07-31 22:56:46 +05:30
| `db/` | `spec/db/` | RSpec | |
2019-12-21 20:55:43 +05:30
| `db/{post_,}migrate/` | `spec/migrations/` | RSpec | More details in the [Testing Rails migrations guide](testing_migrations_guide.md). |
2019-07-31 22:56:46 +05:30
| `Gemfile` | `spec/dependencies/`, `spec/sidekiq/` | RSpec | |
| `lib/` | `spec/lib/` | RSpec | |
| `lib/tasks/` | `spec/tasks/` | RSpec | |
| `rubocop/` | `spec/rubocop/` | RSpec | |
2018-03-17 18:26:18 +05:30
2020-03-13 15:44:24 +05:30
### Frontend unit tests
Unit tests are on the lowest abstraction level and typically test functionality
that is not directly perceivable by a user.
```mermaid
graph RL
plain[Plain JavaScript];
Vue[Vue Components];
feature-flags[Feature Flags];
license-checks[License Checks];
plain---Vuex;
plain---GraphQL;
Vue---plain;
Vue---Vuex;
Vue---GraphQL;
browser---plain;
browser---Vue;
plain---backend;
Vuex---backend;
GraphQL---backend;
Vue---backend;
backend---database;
backend---feature-flags;
backend---license-checks;
class plain tested;
class Vuex tested;
classDef node color:#909090,fill:#f0f0f0,stroke-width:2px,stroke:#909090
classDef label stroke-width:0;
classDef tested color:#000000,fill:#a0c0ff,stroke:#6666ff,stroke-width:2px,stroke-dasharray: 5, 5;
subgraph " "
tested;
mocked;
class tested tested;
end
```
#### When to use unit tests
- **Exported functions and classes**:
Anything exported can be reused at various places in ways you have no control over.
You should document the expected behavior of the public interface with tests.
- **Vuex actions**:
Any Vuex action must work in a consistent way, independent of the component it is triggered from.
- **Vuex mutations**:
For complex Vuex mutations, you should separate the tests from other parts of the Vuex store to simplify problem-solving.
#### When *not* to use unit tests
2020-04-08 14:13:33 +05:30
2020-03-13 15:44:24 +05:30
- **Non-exported functions or classes**:
Anything not exported from a module can be considered private or an implementation detail, and doesn't need to be tested.
- **Constants**:
Testing the value of a constant means copying it, resulting in extra effort without additional confidence that the value is correct.
- **Vue components**:
Computed properties, methods, and lifecycle hooks can be considered an implementation detail of components, are implicitly covered by component tests, and don't need to be tested.
2022-08-27 11:52:29 +05:30
For more information, see the [official Vue guidelines](https://v1.test-utils.vuejs.org/guides/#getting-started).
2020-03-13 15:44:24 +05:30
#### What to mock in unit tests
- **State of the class under test**:
Modifying the state of the class under test directly rather than using methods of the class avoids side effects in test setup.
- **Other exported classes**:
Every class must be tested in isolation to prevent test scenarios from growing exponentially.
- **Single DOM elements if passed as parameters**:
For tests only operating on single DOM elements, rather than a whole page, creating these elements is cheaper than loading an entire HTML fixture.
- **All server requests**:
When running frontend unit tests, the backend may not be reachable, so all outgoing requests need to be mocked.
- **Asynchronous background operations**:
2021-02-22 17:27:13 +05:30
Background operations cannot be stopped or waited on, so they continue running in the following tests and cause side effects.
2020-03-13 15:44:24 +05:30
#### What *not* to mock in unit tests
- **Non-exported functions or classes**:
2021-02-22 17:27:13 +05:30
Everything that is not exported can be considered private to the module, and is implicitly tested through the exported classes and functions.
2020-03-13 15:44:24 +05:30
- **Methods of the class under test**:
2021-02-22 17:27:13 +05:30
By mocking methods of the class under test, the mocks are tested and not the real methods.
2020-03-13 15:44:24 +05:30
- **Utility functions (pure functions, or those that only modify parameters)**:
If a function has no side effects because it has no state, it is safe to not mock it in tests.
- **Full HTML pages**:
Avoid loading the HTML of a full page in unit tests, as it slows down tests.
### Frontend component tests
Component tests cover the state of a single component that is perceivable by a user depending on external signals such as user input, events fired from other components, or application state.
```mermaid
graph RL
plain[Plain JavaScript];
Vue[Vue Components];
feature-flags[Feature Flags];
license-checks[License Checks];
plain---Vuex;
plain---GraphQL;
Vue---plain;
Vue---Vuex;
Vue---GraphQL;
browser---plain;
browser---Vue;
plain---backend;
Vuex---backend;
GraphQL---backend;
Vue---backend;
backend---database;
backend---feature-flags;
backend---license-checks;
class Vue tested;
classDef node color:#909090,fill:#f0f0f0,stroke-width:2px,stroke:#909090
classDef label stroke-width:0;
classDef tested color:#000000,fill:#a0c0ff,stroke:#6666ff,stroke-width:2px,stroke-dasharray: 5, 5;
subgraph " "
tested;
mocked;
class tested tested;
end
```
#### When to use component tests
- **Vue components**
#### When *not* to use component tests
- **Vue applications**:
Vue applications may contain many components.
Testing them on a component level requires too much effort.
Therefore they are tested on frontend integration level.
- **HAML templates**:
HAML templates contain only Markup and no frontend-side logic.
Therefore they are not complete components.
#### What to mock in component tests
- **DOM**:
Operating on the real DOM is significantly slower than on the virtual DOM.
- **Properties and state of the component under test**:
Similar to testing classes, modifying the properties directly (rather than relying on methods of the component) avoids side effects.
- **Vuex store**:
To avoid side effects and keep component tests simple, Vuex stores are replaced with mocks.
- **All server requests**:
Similar to unit tests, when running component tests, the backend may not be reachable, so all outgoing requests need to be mocked.
- **Asynchronous background operations**:
2021-02-22 17:27:13 +05:30
Similar to unit tests, background operations cannot be stopped or waited on. This means they continue running in the following tests and cause side effects.
2020-03-13 15:44:24 +05:30
- **Child components**:
Every component is tested individually, so child components are mocked.
2022-08-27 11:52:29 +05:30
See also [`shallowMount()`](https://v1.test-utils.vuejs.org/api/#shallowmount)
2020-03-13 15:44:24 +05:30
#### What *not* to mock in component tests
- **Methods or computed properties of the component under test**:
2021-02-22 17:27:13 +05:30
By mocking part of the component under test, the mocks are tested and not the real component.
2020-03-13 15:44:24 +05:30
- **Functions and classes independent from Vue**:
All plain JavaScript code is already covered by unit tests and needs not to be mocked in component tests.
2018-03-17 18:26:18 +05:30
## Integration tests
2019-03-02 22:35:43 +05:30
Formal definition: <https://en.wikipedia.org/wiki/Integration_testing>
2018-03-17 18:26:18 +05:30
2019-02-15 15:39:39 +05:30
These kind of tests ensure that individual parts of the application work well
2022-01-26 12:08:38 +05:30
together, without the overhead of the actual app environment (such as the browser).
2019-02-15 15:39:39 +05:30
These tests should assert at the request/response level: status code, headers,
body.
2022-01-26 12:08:38 +05:30
They're useful, for example, to test permissions, redirections, API endpoints, what view is rendered, and so forth.
2018-03-17 18:26:18 +05:30
| Code path | Tests path | Testing engine | Notes |
| --------- | ---------- | -------------- | ----- |
2022-01-26 12:08:38 +05:30
| `app/controllers/` | `spec/requests/`, `spec/controllers` | RSpec | Request specs are preferred over legacy controller specs. Request specs are encouraged for API endpoints. |
2018-03-17 18:26:18 +05:30
| `app/mailers/` | `spec/mailers/` | RSpec | |
| `lib/api/` | `spec/requests/api/` | RSpec | |
2021-11-11 11:23:49 +05:30
| `app/assets/javascripts/` | `spec/frontend/` | Jest | [More details below](#frontend-integration-tests) |
2020-03-13 15:44:24 +05:30
### Frontend integration tests
Integration tests cover the interaction between all components on a single page.
Their abstraction level is comparable to how a user would interact with the UI.
```mermaid
graph RL
plain[Plain JavaScript];
Vue[Vue Components];
feature-flags[Feature Flags];
license-checks[License Checks];
plain---Vuex;
plain---GraphQL;
Vue---plain;
Vue---Vuex;
Vue---GraphQL;
browser---plain;
browser---Vue;
plain---backend;
Vuex---backend;
GraphQL---backend;
Vue---backend;
backend---database;
backend---feature-flags;
backend---license-checks;
class plain tested;
class Vue tested;
class Vuex tested;
class GraphQL tested;
class browser tested;
linkStyle 0,1,2,3,4,5,6 stroke:#6666ff,stroke-width:2px,stroke-dasharray: 5, 5;
classDef node color:#909090,fill:#f0f0f0,stroke-width:2px,stroke:#909090
classDef label stroke-width:0;
classDef tested color:#000000,fill:#a0c0ff,stroke:#6666ff,stroke-width:2px,stroke-dasharray: 5, 5;
subgraph " "
tested;
mocked;
class tested tested;
end
```
#### When to use integration tests
- **Page bundles (`index.js` files in `app/assets/javascripts/pages/`)**:
Testing the page bundles ensures the corresponding frontend components integrate well.
- **Vue applications outside of page bundles**:
Testing Vue applications as a whole ensures the corresponding frontend components integrate well.
#### What to mock in integration tests
- **HAML views (use fixtures instead)**:
Rendering HAML views requires a Rails environment including a running database, which you cannot rely on in frontend tests.
- **All server requests**:
Similar to unit and component tests, when running component tests, the backend may not be reachable, so all outgoing requests must be mocked.
- **Asynchronous background operations that are not perceivable on the page**:
Background operations that affect the page must be tested on this level.
2021-02-22 17:27:13 +05:30
All other background operations cannot be stopped or waited on, so they continue running in the following tests and cause side effects.
2020-03-13 15:44:24 +05:30
#### What *not* to mock in integration tests
- **DOM**:
Testing on the real DOM ensures your components work in the intended environment.
2020-06-23 00:09:42 +05:30
Part of DOM testing is delegated to [cross-browser testing](https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/45).
2020-03-13 15:44:24 +05:30
- **Properties or state of components**:
On this level, all tests can only perform actions a user would do.
For example: to change the state of a component, a click event would be fired.
- **Vuex stores**:
When testing the frontend code of a page as a whole, the interaction between Vue components and Vuex stores is covered as well.
2018-03-17 18:26:18 +05:30
### About controller tests
2021-03-08 18:12:59 +05:30
GitLab is [transitioning from controller specs to request specs](https://gitlab.com/groups/gitlab-org/-/epics/5076).
2018-03-17 18:26:18 +05:30
In an ideal world, controllers should be thin. However, when this is not the
2020-03-13 15:44:24 +05:30
case, it's acceptable to write a system or feature test without JavaScript instead
of a controller test. Testing a fat controller usually involves a lot of stubbing, such as:
2018-03-17 18:26:18 +05:30
```ruby
controller.instance_variable_set(:@user, user)
```
2021-03-08 18:12:59 +05:30
and use methods [deprecated in Rails 5](https://gitlab.com/gitlab-org/gitlab/-/issues/16260).
2018-03-17 18:26:18 +05:30
2019-02-15 15:39:39 +05:30
## White-box tests at the system level (formerly known as System / Feature tests)
2018-03-17 18:26:18 +05:30
2019-02-15 15:39:39 +05:30
Formal definitions:
2018-03-17 18:26:18 +05:30
2019-03-02 22:35:43 +05:30
- <https://en.wikipedia.org/wiki/System_testing>
- <https://en.wikipedia.org/wiki/White-box_testing>
2019-02-15 15:39:39 +05:30
2020-03-13 15:44:24 +05:30
These kind of tests ensure the GitLab *Rails* application (for example,
2019-12-21 20:55:43 +05:30
`gitlab-foss`/`gitlab`) works as expected from a *browser* point of view.
2019-02-15 15:39:39 +05:30
Note that:
- knowledge of the internals of the application are still required
- data needed for the tests are usually created directly using RSpec factories
- expectations are often set on the database or objects state
These tests should only be used when:
- the functionality/component being tested is small
- the internal state of the objects/database *needs* to be tested
- it cannot be tested at a lower level
For instance, to test the breadcrumbs on a given page, writing a system test
makes sense since it's a small component, which cannot be tested at the unit or
controller level.
Only test the happy path, but make sure to add a test case for any regression
2020-03-13 15:44:24 +05:30
that couldn't have been caught at lower levels with better tests (for example, if a
regression is found, regression tests should be added at the lowest level
2018-03-17 18:26:18 +05:30
possible).
| Tests path | Testing engine | Notes |
| ---------- | -------------- | ----- |
2021-02-22 17:27:13 +05:30
| `spec/features/` | [Capybara](https://github.com/teamcapybara/capybara) + [RSpec](https://github.com/rspec/rspec-rails#feature-specs) | If your test has the `:js` metadata, the browser driver is [Poltergeist](https://github.com/teamcapybara/capybara#poltergeist), otherwise it's using [RackTest](https://github.com/teamcapybara/capybara#racktest). |
2018-03-17 18:26:18 +05:30
2020-03-13 15:44:24 +05:30
### Frontend feature tests
In contrast to [frontend integration tests](#frontend-integration-tests), feature
tests make requests against the real backend instead of using fixtures.
This also implies that database queries are executed which makes this category significantly slower.
See also:
- The [RSpec testing guidelines](../testing_guide/best_practices.md#rspec).
- System / Feature tests in the [Testing Best Practices](best_practices.md#system--feature-tests).
2020-06-23 00:09:42 +05:30
- [Issue #26159](https://gitlab.com/gitlab-org/gitlab/-/issues/26159) which aims at combining those guidelines with this page.
2020-03-13 15:44:24 +05:30
```mermaid
graph RL
plain[Plain JavaScript];
Vue[Vue Components];
feature-flags[Feature Flags];
license-checks[License Checks];
plain---Vuex;
plain---GraphQL;
Vue---plain;
Vue---Vuex;
Vue---GraphQL;
browser---plain;
browser---Vue;
plain---backend;
Vuex---backend;
GraphQL---backend;
Vue---backend;
backend---database;
backend---feature-flags;
backend---license-checks;
class backend tested;
class plain tested;
class Vue tested;
class Vuex tested;
class GraphQL tested;
class browser tested;
linkStyle 0,1,2,3,4,5,6,7,8,9,10 stroke:#6666ff,stroke-width:2px,stroke-dasharray: 5, 5;
classDef node color:#909090,fill:#f0f0f0,stroke-width:2px,stroke:#909090
classDef label stroke-width:0;
classDef tested color:#000000,fill:#a0c0ff,stroke:#6666ff,stroke-width:2px,stroke-dasharray: 5, 5;
subgraph " "
tested;
mocked;
class tested tested;
end
```
#### When to use feature tests
- Use cases that require a backend, and cannot be tested using fixtures.
- Behavior that is not part of a page bundle, but defined globally.
#### Relevant notes
A `:js` flag is added to the test to make sure the full environment is loaded:
```ruby
scenario 'successfully', :js do
sign_in(create(:admin))
end
```
The steps of each test are written using ([capybara methods](https://www.rubydoc.info/gems/capybara)).
XHR (XMLHttpRequest) calls might require you to use `wait_for_requests` in between steps, such as:
```ruby
find('.form-control').native.send_keys(:enter)
wait_for_requests
expect(page).not_to have_selector('.card')
```
2019-12-04 20:38:33 +05:30
### Consider **not** writing a system test
2018-03-17 18:26:18 +05:30
If we're confident that the low-level components work well (and we should be if
we have enough Unit & Integration tests), we shouldn't need to duplicate their
thorough testing at the System test level.
It's very easy to add tests, but a lot harder to remove or improve tests, so one
2019-02-15 15:39:39 +05:30
should take care of not introducing too many (slow and duplicated) tests.
2018-03-17 18:26:18 +05:30
The reasons why we should follow these best practices are as follows:
2021-02-22 17:27:13 +05:30
- System tests are slow to run because they spin up the entire application stack
2018-03-17 18:26:18 +05:30
in a headless browser, and even slower when they integrate a JS driver
- When system tests run with a JavaScript driver, the tests are run in a
different thread than the application. This means it does not share a
2021-02-22 17:27:13 +05:30
database connection and your test must commit the transactions in
2018-03-17 18:26:18 +05:30
order for the running application to see the data (and vice-versa). In that
2021-02-22 17:27:13 +05:30
case we need to truncate the database after each spec instead of
2018-03-17 18:26:18 +05:30
rolling back a transaction (the faster strategy that's in use for other kind
of tests). This is slower than transactions, however, so we want to use
truncation only when necessary.
2019-02-15 15:39:39 +05:30
## Black-box tests at the system level, aka end-to-end tests
Formal definitions:
2019-03-02 22:35:43 +05:30
- <https://en.wikipedia.org/wiki/System_testing>
- <https://en.wikipedia.org/wiki/Black-box_testing>
2018-03-17 18:26:18 +05:30
2020-05-24 23:13:21 +05:30
GitLab consists of [multiple pieces](../architecture.md#components) such as [GitLab Shell](https://gitlab.com/gitlab-org/gitlab-shell), [GitLab Workhorse](https://gitlab.com/gitlab-org/gitlab-workhorse),
[Gitaly](https://gitlab.com/gitlab-org/gitaly), [GitLab Pages](https://gitlab.com/gitlab-org/gitlab-pages), [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner), and GitLab Rails. All theses pieces
are configured and packaged by [Omnibus GitLab](https://gitlab.com/gitlab-org/omnibus-gitlab).
2018-03-17 18:26:18 +05:30
2020-05-24 23:13:21 +05:30
The QA framework and instance-level scenarios are [part of GitLab Rails](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/qa) so that
2019-02-15 15:39:39 +05:30
they're always in-sync with the codebase (especially the views).
2018-03-17 18:26:18 +05:30
2019-02-15 15:39:39 +05:30
Note that:
2018-03-17 18:26:18 +05:30
2019-02-15 15:39:39 +05:30
- knowledge of the internals of the application are not required
- data needed for the tests can only be created using the GUI or the API
- expectations can only be made against the browser page and API responses
2018-11-20 20:47:30 +05:30
2021-09-04 01:27:46 +05:30
Every new feature should come with a [test plan](https://gitlab.com/gitlab-org/gitlab/-/tree/master/.gitlab/issue_templates/Test%20plan.md).
2018-11-20 20:47:30 +05:30
2019-02-15 15:39:39 +05:30
| Tests path | Testing engine | Notes |
| ---------- | -------------- | ----- |
2021-02-22 17:27:13 +05:30
| `qa/qa/specs/features/` | [Capybara](https://github.com/teamcapybara/capybara) + [RSpec](https://github.com/rspec/rspec-rails#feature-specs) + Custom QA framework | Tests should be placed under their corresponding [Product category](https://about.gitlab.com/handbook/product/categories/) |
2019-01-03 12:48:30 +05:30
2019-09-04 21:01:54 +05:30
> See [end-to-end tests](end_to_end/index.md) for more information.
2018-03-17 18:26:18 +05:30
2019-07-07 11:18:12 +05:30
Note that `qa/spec` contains unit tests of the QA framework itself, not to be
confused with the application's [unit tests](#unit-tests) or
[end-to-end tests](#black-box-tests-at-the-system-level-aka-end-to-end-tests).
2019-02-15 15:39:39 +05:30
### Smoke tests
Smoke tests are quick tests that may be run at any time (especially after the
pre-deployment migrations).
These tests run against the UI and ensure that basic functionality is working.
> See [Smoke Tests](smoke.md) for more information.
### GitLab QA orchestrator
2022-01-26 12:08:38 +05:30
[GitLab QA orchestrator](https://gitlab.com/gitlab-org/gitlab-qa) is a tool that allows you to test that all these pieces integrate well together by building a Docker image for a given version of GitLab Rails and running end-to-end tests (using Capybara) against it.
2019-02-15 15:39:39 +05:30
2020-05-24 23:13:21 +05:30
Learn more in the [GitLab QA orchestrator README](https://gitlab.com/gitlab-org/gitlab-qa/tree/master/README.md).
2018-03-17 18:26:18 +05:30
2018-03-27 19:54:05 +05:30
## EE-specific tests
EE-specific tests follows the same organization, but under the `ee/spec` folder.
2018-03-17 18:26:18 +05:30
## How to test at the correct level?
As many things in life, deciding what to test at each level of testing is a
trade-off:
- Unit tests are usually cheap, and you should consider them like the basement
of your house: you need them to be confident that your code is behaving
correctly. However if you run only unit tests without integration / system
2020-05-24 23:13:21 +05:30
tests, you might [miss](https://twitter.com/ThePracticalDev/status/850748070698651649) the
[big](https://twitter.com/timbray/status/822470746773409794) /
[picture](https://twitter.com/withzombies/status/829716565834752000) !
2018-03-17 18:26:18 +05:30
- Integration tests are a bit more expensive, but don't abuse them. A system test
is often better than an integration test that is stubbing a lot of internals.
- System tests are expensive (compared to unit tests), even more if they require
2019-07-07 11:18:12 +05:30
a JavaScript driver. Make sure to follow the guidelines in the [Speed](best_practices.md#test-speed)
2018-03-17 18:26:18 +05:30
section.
Another way to see it is to think about the "cost of tests", this is well
2020-05-24 23:13:21 +05:30
explained [in this article](https://medium.com/table-xi/high-cost-tests-and-high-value-tests-a86e27a54df#.2ulyh3a4e)
and the basic idea is that the cost of a test includes:
2018-03-17 18:26:18 +05:30
- The time it takes to write the test
- The time it takes to run the test every time the suite runs
- The time it takes to understand the test
- The time it takes to fix the test if it breaks and the underlying code is OK
- Maybe, the time it takes to change the code to make the code testable.
### Frontend-related tests
2020-05-24 23:13:21 +05:30
There are cases where the behavior you are testing is not worth the time spent
2018-03-17 18:26:18 +05:30
running the full application, for example, if you are testing styling, animation,
edge cases or small actions that don't involve the backend,
2021-11-11 11:23:49 +05:30
you should write an integration test using [Frontend integration tests](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/frontend_integration/README.md).
2018-03-17 18:26:18 +05:30
---
[Return to Testing documentation](index.md)