debian-mirror-gitlab/.gitlab/issue_templates/Experimentation.md

26 lines
1.8 KiB
Markdown
Raw Normal View History

2021-04-17 20:07:23 +05:30
<!-- Title suggestion: Experiment: [description] -->
# Experiment Summary
<!-- Quick rundown of what is being done -->
2021-03-11 19:13:27 +05:30
2021-04-17 20:07:23 +05:30
# Design
<!-- This should include the contexts that determine the reproducibility (stickiness) of an experiment. This means that if you want the same behavior for a user, the context would be user, or if you want all users when viewing a specific project, the context would be the project being viewed, etc. -->
2021-03-11 19:13:27 +05:30
2021-04-17 20:07:23 +05:30
# Rollout strategy
2021-03-11 19:13:27 +05:30
<!-- This is currently called A/B test, which isn't accurate for multi-variants. Let's call this rollout strategy. It should outline the percentages for variants and if there's more than one step to this, each of those steps and the timing for those steps (e.g. 30 days after initial rollout). -->
2021-04-17 20:07:23 +05:30
# Inclusions and exclusions
2021-03-11 19:13:27 +05:30
<!-- These would be the rules for which given context (and are limited to context or resolvable at experiment time details) is included or excluded from the test. An example of this would be to only run an experiment on groups less than N number of days old. -->
2021-04-17 20:07:23 +05:30
# Segmentation
2021-03-11 19:13:27 +05:30
<!-- Rules for always saying context with these criteria always get this variant. For instance, if you want to always give groups less than N number of days old the experiment experience, they are specified here. This is different from the exclusion rules above. -->
2021-04-17 20:07:23 +05:30
# Tracking Details
- [json schema](https://gitlab.com/gitlab-org/iglu/-/blob/master/public/schemas/com.gitlab/gitlab_experiment/jsonschema/0-3-0) used in `gitlab-experiment` tracking.
2021-04-29 21:17:54 +05:30
- see [taxonomy](https://docs.gitlab.com/ee/development/snowplow/index.html#structured-event-taxonomy) for a guide.
2021-04-17 20:07:23 +05:30
| activity | category | action | label | context | property | value |
| -------- | -------- | ------ | ----- | ------- | -------- | ----- |
| | | | | json schema | | |