2.3 KiB
2.3 KiB
GitLab tests in the Continuous Integration (CI) context
Test suite parallelization on the CI
Our current CI parallelization setup is as follows:
- The
knapsackjob in the prepare stage that is supposed to ensure we have aknapsack/${CI_PROJECT_NAME}/rspec_report-master.jsonfile:
- The
knapsack/${CI_PROJECT_NAME}/rspec_report-master.jsonfile is fetched from S3, if it's not here we initialize the file with{}.
- Each
rspec x yjob are run withknapsack rspecand should have an evenly distributed share of tests:
- It works because the jobs have access to the
knapsack/${CI_PROJECT_NAME}/rspec_report-master.jsonsince the "artifacts from all previous stages are passed by default". [^1] - the jobs set their own report path to
KNAPSACK_REPORT_PATH=knapsack/${CI_PROJECT_NAME}/${JOB_NAME[0]}_node_${CI_NODE_INDEX}_${CI_NODE_TOTAL}_report.json. - if knapsack is doing its job, test files that are run should be listed under
Report specs, not underLeftover specs.
- The
update-knapsackjob takes all theknapsack/${CI_PROJECT_NAME}/${JOB_NAME[0]}_node_${CI_NODE_INDEX}_${CI_NODE_TOTAL}_report.jsonfiles from therspec x yjobs and merge them all together into a singleknapsack/${CI_PROJECT_NAME}/rspec_report-master.jsonfile that is then uploaded to S3.
After that, the next pipeline will use the up-to-date
knapsack/${CI_PROJECT_NAME}/rspec_report-master.json file. The same strategy
is used for Spinach tests as well.
Monitoring
The GitLab test suite is monitored for the master branch, and any branch
that includes rspec-profile in their name.
A public dashboard is available for everyone to see. Feel free to look at the slowest test files and try to improve them.
CI setup
- On CE and EE, the test suite runs both PostgreSQL and MySQL.
- Rails logging to
log/test.logis disabled by default in CI for performance reasons. To override this setting, provide theRAILS_ENABLE_TEST_LOGenvironment variable.