9.8 KiB
type |
---|
reference, howto |
Browser Performance Testing (PREMIUM)
Introduced in GitLab Premium 10.3.
If your application offers a web interface and you're using GitLab CI/CD, you can quickly determine the performance impact of pending code changes.
Overview
GitLab uses Sitespeed.io, a free and open source
tool, for measuring the performance of web sites. GitLab has built a simple
Sitespeed plugin which outputs
the performance score for each page analyzed in a file called performance.json
.
The Sitespeed.io performance score
is a composite value based on best practices.
GitLab can show the Performance report in the merge request widget area.
Use cases
Consider the following workflow:
- A member of the marketing team is attempting to track engagement by adding a new tool.
- With browser performance metrics, they see how their changes are impacting the usability of the page for end users.
- The metrics show that after their changes, the performance score of the page has gone down.
- When looking at the detailed report, they see the new JavaScript library was
included in
<head>
, which affects loading page speed. - They ask for help from a front end developer, who sets the library to load asynchronously.
- The frontend developer approves the merge request, and authorizes its deployment to production.
How browser performance testing works
First, define a job in your .gitlab-ci.yml
file that generates the
Performance report artifact.
GitLab then checks this report, compares key performance metrics for each page
between the source and target branches, and shows the information in the merge request.
For an example Performance job, see Configuring Browser Performance Testing.
NOTE: Note:
If the Performance report has no data to compare, such as when you add the
Performance job in your .gitlab-ci.yml
for the very first time, no information
displays in the merge request widget area. Consecutive merge requests will have data for
comparison, and the Performance report will be shown properly.
Configuring Browser Performance Testing
This example shows how to run the sitespeed.io container on your code by using GitLab CI/CD and sitespeed.io using Docker-in-Docker.
-
First, set up GitLab Runner with a docker-in-docker build.
-
After configuring the Runner, add a new job to
.gitlab-ci.yml
that generates the expected report. -
Define the
performance
job according to your version of GitLab:- For GitLab 12.4 and later - include the
Browser-Performance.gitlab-ci.yml
template provided as a part of your GitLab installation. - For GitLab versions earlier than 12.4 - Copy and use the job as defined in the
Browser-Performance.gitlab-ci.yml
template.
CAUTION: Caution: The job definition provided by the template does not support Kubernetes yet. For a complete example of a more complex setup that works in Kubernetes, see
Browser-Performance-Testing.gitlab-ci.yml
. - For GitLab 12.4 and later - include the
-
Add the following to your
.gitlab-ci.yml
file:include: template: Verify/Browser-Performance.gitlab-ci.yml performance: variables: URL: https://example.com
CAUTION: Caution: The job definition provided by the template is supported in GitLab 11.5 and later versions. It also requires GitLab Runner 11.5 or later. For earlier versions, use the previous job definitions.
The above example creates a performance
job in your CI/CD pipeline and runs
sitespeed.io against the webpage you defined in URL
to gather key metrics.
The GitLab plugin for sitespeed.io
is downloaded to save the report as a Performance report artifact
that you can later download and analyze. Due to implementation limitations, we always
take the latest Performance artifact available.
The full HTML sitespeed.io report is saved as an artifact, and if GitLab Pages is enabled, it can be viewed directly in your browser.
You can also customize options by setting the SITESPEED_OPTIONS
variable.
For example, you can override the number of runs sitespeed.io
makes on the given URL:
include:
template: Verify/Browser-Performance.gitlab-ci.yml
performance:
variables:
URL: https://example.com
SITESPEED_OPTIONS: -n 5
For further customization options for sitespeed.io, including the ability to provide a list of URLs to test, please see the Sitespeed.io Configuration documentation.
TIP: Tip: Key metrics are automatically extracted and shown in the merge request widget.
Configuring degradation threshold
Introduced in GitLab 13.0.
You can configure the sensitivity of degradation alerts to avoid getting alerts for minor drops in metrics.
This is done by setting the DEGRADATION_THRESHOLD
variable. In the example below, the alert will only show up
if the Total Score
metric degrades by 5 points or more:
include:
template: Verify/Browser-Performance.gitlab-ci.yml
performance:
variables:
URL: https://example.com
DEGRADATION_THRESHOLD: 5
The Total Score
metric is based on sitespeed.io's coach performance score. There is more information in the coach documentation.
Performance testing on Review Apps
The above CI YAML configuration is great for testing against static environments, and it can be extended for dynamic environments, but a few extra steps are required:
- The
performance
job should run after the dynamic environment has started. - In the
review
job, persist the hostname and upload it as an artifact so it's available to theperformance
job. The same can be done for static environments like staging and production to unify the code path. You can save it as an artifact withecho $CI_ENVIRONMENT_URL > environment_url.txt
in your job'sscript
. - In the
performance
job, read the previous artifact into an environment variable. In this case, use$URL
because the sitespeed.io command uses it for the URL parameter. Because Review App URLs are dynamic, define theURL
variable throughbefore_script
instead ofvariables
. - You can now run the sitespeed.io container against the desired hostname and paths.
Your .gitlab-ci.yml
file would look like:
stages:
- deploy
- performance
include:
template: Verify/Browser-Performance.gitlab-ci.yml
review:
stage: deploy
environment:
name: review/$CI_COMMIT_REF_SLUG
url: http://$CI_COMMIT_REF_SLUG.$APPS_DOMAIN
script:
- run_deploy_script
- echo $CI_ENVIRONMENT_URL > environment_url.txt
artifacts:
paths:
- environment_url.txt
only:
- branches
except:
- master
performance:
dependencies:
- review
before_script:
- export URL=$(cat environment_url.txt)
Previous job definitions
CAUTION: Caution:
Before GitLab 11.5, the Performance job and artifact had to be named specifically
to automatically extract report data and show it in the merge request widget.
While these old job definitions are still maintained, they have been deprecated
and may be removed in next major release, GitLab 12.0.
GitLab recommends you update your current .gitlab-ci.yml
configuration to reflect that change.
For GitLab 11.4 and earlier, the job should look like:
performance:
stage: performance
image: docker:git
variables:
URL: https://example.com
services:
- docker:stable-dind
script:
- mkdir gitlab-exporter
- wget -O ./gitlab-exporter/index.js https://gitlab.com/gitlab-org/gl-performance/raw/master/index.js
- mkdir sitespeed-results
- docker run --shm-size=1g --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io:6.3.1 --plugins.add ./gitlab-exporter --outputFolder sitespeed-results $URL
- mv sitespeed-results/data/performance.json performance.json
artifacts:
paths:
- performance.json
- sitespeed-results/