4.1 KiB
| stage | group | info |
|---|---|---|
| Growth | Adoption | To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments |
Experiment rollouts and feature flags
Experiment rollout issue
Each experiment should have an experiment rollout issue to track the experiment from rollout through to cleanup and removal. The rollout issue is similar to a feature flag rollout issue, and is also used to track the status of an experiment.
When an experiment is deployed, the due date of the issue should be set (this depends on the experiment but can be up to a few weeks in the future). After the deadline, the issue must be resolved and either:
- It was successful and the experiment becomes the new default.
- It was not successful and all code related to the experiment is removed.
In either case, an outcome of the experiment should be posted to the issue with the reasoning for the decision.
Turn off all experiments
When there is a case on GitLab.com (SaaS) that necessitates turning off all experiments, we have this control.
You can toggle experiments on SaaS on and off using the gitlab_experiment feature flag.
This can be done via ChatOps:
- disable:
/chatops run feature set gitlab_experiment false - enable:
/chatops run feature delete gitlab_experiment - This allows the
default_enabledvalue of true in the YAML to be honored.
Notes on feature flags
NOTE: We use the terms "enabled" and "disabled" here, even though it's against our documentation style guide recommendations because these are the terms that the feature flag documentation uses.
You may already be familiar with the concept of feature flags in GitLab, but using
feature flags in experiments is a bit different. While in general terms, a feature flag
is viewed as being either on or off, this isn't accurate for experiments.
Generally, off means that when we ask if a feature flag is enabled, it always
returns false, and on means that it always returns true. An interim state,
considered conditional, also exists. We take advantage of this trinary state of
feature flags. To understand this conditional aspect: consider that either of these
settings puts a feature flag into this state:
- Setting a
percentage_of_actorsof any percent greater than 0%. - Enabling it for a single user or group.
Conditional means that it returns true in some situations, but not all situations.
When a feature flag is disabled (meaning the state is off), the experiment is
considered inactive. You can visualize this in the decision tree diagram
as reaching the first Running? node, and traversing the negative path.
When a feature flag is rolled out to a percentage_of_actors or similar (meaning the
state is conditional) the experiment is considered to be running
where sometimes the control is assigned, and sometimes the candidate is assigned.
We don't refer to this as being enabled, because that's a confusing and overloaded
term here. In the experiment terms, our experiment is running, and the feature flag is
conditional.
When a feature flag is enabled (meaning the state is on), the candidate is always
assigned.
We should try to be consistent with our terms, and so for experiments, we have an
inactive experiment until we set the feature flag to conditional. After which,
our experiment is then considered running. If you choose to "enable" your feature flag,
you should consider the experiment to be resolved, because everyone is assigned
the candidate unless they've opted out of experimentation.
As of GitLab 13.10, work is being done to improve this process and how we communicate about it.