## What Track the status of an experiment through to removal. 1. Experiment key: `` 1. Framework: `experimentation.rb` | `gitlab_experiment` 1. Feature flag name: _experiment_percentage` | `` This is an experiment tracking issue for: `` using the scoped [experiment label](https://about.gitlab.com/handbook/engineering/development/growth/#experiment-tracking-issue). As well as defining the experiment rollout and cleanup, this issue incorporates the relevant [`Feature Flag Roll Out`](https://gitlab.com/gitlab-org/gitlab/-/edit/master/.gitlab/issue_templates/Feature%20Flag%20Roll%20Out.md) steps. ## Owners - Team: `group::TEAM_NAME` - Most appropriate slack channel to reach out to: `#g_TEAM_NAME` - Best individual to reach out to: NAME ## Expectations ### What are we expecting to happen? ### What might happen if this goes wrong? ### What can we monitor to detect problems with this? ### Tracked data Note: you can utilize [CXL calculator](https://cxl.com/ab-test-calculator/) to determine if your experiment has reached signifigance, it also includes an estimate for how much longer an experiment will need to run for before reaching signifigance. ### Staging Test ### Experiment tracking log ### Experiment Results ## Roll Out Steps - [ ] Confirm that QA tests pass with the feature flag enabled (if you're unsure how, contact the relevant [stable counterpart in the Quality department](https://about.gitlab.com/handbook/engineering/quality/#individual-contributors)) - [ ] Enable on staging (`/chatops run feature set feature_name true --staging`) - [ ] Test on staging - [ ] Ensure that documentation has been updated - [ ] Enable on GitLab.com for individual groups/projects listed above and verify behaviour (`/chatops run feature set --project=gitlab-org/gitlab feature_name true`) - [ ] Coordinate a time to enable the flag with the SRE oncall and release managers - In `#production` mention `@sre-oncall` and `@release-managers`. Once an SRE on call and Release Manager on call confirm, you can proceed with the rollout - [ ] Announce on the issue an estimated time this will be enabled on GitLab.com - [ ] Enable on GitLab.com by running chatops command in `#production` (`/chatops run feature set feature_name true`) - [ ] Cross post chatops Slack command to `#support_gitlab-com` ([more guidance when this is necessary in the dev docs](https://docs.gitlab.com/ee/development/feature_flags/controls.html#where-to-run-commands)) and in your team channel - [ ] Announce on the issue that the flag has been enabled - [ ] Remove experiment code and feature flag and add changelog entry - a separate [cleanup issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Experiment%20Successful%20Cleanup) might be required - [ ] After the flag removal is deployed, [clean up the feature flag](https://docs.gitlab.com/ee/development/feature_flags/controls.html#cleaning-up) by running chatops command in `#production` channel - [ ] Assign to the product manager to update the [knowledge base](https://about.gitlab.com/direction/growth/#growth-insights-knowledge-base) (if applicable) ## Rollback Steps - [ ] This feature can be disabled by running the following Chatops command: ``` /chatops run feature set feature_name false ``` /label ~"feature flag" ~"devops::growth" ~"growth experiment" ~"experiment tracking" ~Engineering ~"workflow::scheduling" ~"experiment::pending"