Compare commits

..

314 commits

Author SHA1 Message Date
Pirate Praveen
8da987f0d1 Upload to buster-fasttrack 2021-08-11 22:45:14 +05:30
Pirate Praveen
0c4769bcfc Merge tag 'debian/13.12.9+ds1-3' into buster-fasttrack
gitlab Debian release 13.12.9+ds1-3
2021-08-11 22:43:09 +05:30
Pirate Praveen
78834fd8c4 Upload to buster-fasttrack 2021-08-05 00:36:42 +05:30
Pirate Praveen
e6d56a972b Lock yarn version to 2.4.2 as berry now points to 3.0.0 and need node 12 2021-08-05 00:36:42 +05:30
Pirate Praveen
62ff05978b Merge tag 'debian/13.12.9+ds1-1' into buster-fasttrack
gitlab Debian release 13.12.9+ds1-1
2021-08-04 23:02:39 +05:30
Pirate Praveen
badbc4a115 Upload to buster-fasttrack 2021-07-09 00:43:28 +05:30
Pirate Praveen
0e116ca35e Merge tag 'debian/13.12.8+ds1-1' into buster-fasttrack
gitlab Debian release 13.12.8+ds1-1
2021-07-09 00:42:48 +05:30
Pirate Praveen
21afb2fcaa Upload to buster-fasttrack 2021-07-02 21:06:17 +05:30
Pirate Praveen
d283892edb Merge tag 'debian/13.12.6+ds1-1' into buster-fasttrack
gitlab Debian release 13.12.6+ds1-1
2021-07-02 21:04:16 +05:30
Pirate Praveen
faa7997001 Upload to buster-fasttrack 2021-06-22 00:47:47 +05:30
Pirate Praveen
8c3d9e263a Merge tag 'debian/13.12.4+ds1-1' into buster-fasttrack
gitlab Debian release 13.12.4+ds1-1
2021-06-22 00:47:19 +05:30
Pirate Praveen
c86c35eb39 Upload to buster-fasttrack 2021-06-21 21:44:12 +05:30
Pirate Praveen
3bf1fde4d6 Update minimum version of golang-any (>= 2:1.15~) 2021-06-21 21:44:12 +05:30
Pirate Praveen
d8c2b47392 Merge tag 'debian/13.12.3+ds1-7' into buster-fasttrack
gitlab Debian release 13.12.3+ds1-7
2021-06-21 19:26:33 +05:30
Pirate Praveen
b2ff9664f1 Upload to buster-fasttrack 2021-06-21 14:55:36 +05:30
Pirate Praveen
801565bca1 Merge tag 'debian/13.12.3+ds1-6' into buster-fasttrack
gitlab Debian release 13.12.3+ds1-6
2021-06-21 14:54:14 +05:30
Pirate Praveen
ef3db33685 Upload to buster-fasttrack 2021-06-18 23:03:13 +05:30
Pirate Praveen
7d9810bee3 Merge tag 'debian/13.12.3+ds1-4' into buster-fasttrack
gitlab Debian release 13.12.3+ds1-4
2021-06-18 23:02:44 +05:30
Pirate Praveen
a96fb1d432 Upload to buster-fasttrack 2021-06-17 22:24:20 +05:30
Pirate Praveen
48d850471f Merge tag 'debian/13.12.3+ds1-3' into buster-fasttrack
gitlab Debian release 13.12.3+ds1-3
2021-06-17 22:23:28 +05:30
Pirate Praveen
82485912da Upload to buster-fasttrack 2021-06-02 18:21:07 +05:30
Pirate Praveen
453e9bf939 Merge tag 'debian/13.11.5+ds1-1' into buster-fasttrack
gitlab Debian release 13.11.5+ds1-1
2021-06-02 18:19:36 +05:30
Pirate Praveen
f738593d87 Upload to buster-fasttrack 2021-04-30 17:26:05 +05:30
Pirate Praveen
1528c4dcc8 Relax dependency of gon in Gemfile and update minimum version in d/control 2021-04-30 17:25:10 +05:30
Pirate Praveen
b18fbb44ef Upload to buster-fasttrack 2021-04-30 11:12:19 +05:30
Pirate Praveen
2bb9b69925 Merge tag 'debian/13.11.2+ds1-1' into buster-fasttrack
gitlab Debian release 13.11.2+ds1-1
2021-04-30 11:11:56 +05:30
Pirate Praveen
c06d604b85 Upload to buster-fasttrack 2021-04-28 20:08:55 +05:30
Pirate Praveen
8def421912 Merge tag 'debian/13.10.4+ds1-1' into buster-fasttrack
gitlab Debian release 13.10.4+ds1-1
2021-04-28 20:08:32 +05:30
Pirate Praveen
238ef05840 Upload to buster-fasttrack 2021-04-25 19:06:50 +05:30
Pirate Praveen
02e8861154 Merge tag 'debian/13.10.3+ds2-3' into buster-fasttrack
gitlab Debian release 13.10.3+ds2-3
2021-04-25 19:06:24 +05:30
Pirate Praveen
da4bb4072d Upload to buster-fasttrack 2021-04-24 00:27:56 +05:30
Pirate Praveen
4812699f85 Merge tag 'debian/13.10.3+ds2-2' into buster-fasttrack
gitlab Debian release 13.10.3+ds2-2
2021-04-24 00:27:24 +05:30
Pirate Praveen
9eae96aef8 Upload to buster-fasttrack 2021-04-16 14:16:28 +05:30
Pirate Praveen
b02afa4f89 Merge tag 'debian/13.9.6+ds1-1' into buster-fasttrack
gitlab Debian release 13.9.6+ds1-1
2021-04-16 14:16:04 +05:30
Pirate Praveen
afe9c542cf Upload to buster-fasttrack 2021-04-03 00:21:08 +05:30
Pirate Praveen
85efb61598 Merge tag 'debian/13.9.5+ds1-1' into buster-fasttrack
gitlab Debian release 13.9.5+ds1-1
2021-04-03 00:20:39 +05:30
Pirate Praveen
389d1ed7e0 Upload to buster-fasttrack 2021-03-30 14:13:22 +05:30
Pirate Praveen
07faab1c68 Merge tag 'debian/13.9.4+ds1-3' into buster-fasttrack
gitlab Debian release 13.9.4+ds1-3
2021-03-30 14:12:07 +05:30
Pirate Praveen
bb8bc5f025 Merge changelog from buster-fasttrack-13.8 branch 2021-03-30 14:10:10 +05:30
Pirate Praveen
5d08035ff2 Upload to buster-fasttrack 2021-03-24 00:20:02 +05:30
Pirate Praveen
54ca8e9b34 Switch back to ruby-rugged 0.28 in Gemfile 2021-03-24 00:19:11 +05:30
Pirate Praveen
970af3fbc8 Upload to buster-fasttrack 2021-03-23 23:44:23 +05:30
Pirate Praveen
94f269c75b Merge tag 'debian/13.9.4+ds1-2' into buster-fasttrack
gitlab Debian release 13.9.4+ds1-2
2021-03-23 23:43:58 +05:30
Pirate Praveen
47201e85b9 Upload to buster-fasttrack 2021-03-12 00:13:44 +05:30
Pirate Praveen
d8213aad6d Merge tag 'debian/13.9.3+ds1-2' into buster-fasttrack
gitlab Debian release 13.9.3+ds1-2
2021-03-12 00:13:18 +05:30
Pirate Praveen
9fa08a5a75 Upload to buster-fasttrack 2021-03-09 17:16:39 +05:30
Pirate Praveen
95ac6d2e76 Merge tag 'debian/13.8.5+ds1-2' into buster-fasttrack
gitlab Debian release 13.8.5+ds1-2
2021-03-09 17:16:16 +05:30
Pirate Praveen
b7cd407f4a Upload to buster-fasttrack 2021-03-09 16:54:15 +05:30
Pirate Praveen
6566dafcbe Merge tag 'debian/13.8.5+ds1-1' into buster-fasttrack
gitlab Debian release 13.8.5+ds1-1
2021-03-09 16:53:41 +05:30
Pirate Praveen
ea99bee680 Upload to buster-fasttrack 2021-03-05 22:26:23 +05:30
Pirate Praveen
13d55acb2f Merge tag 'debian/13.7.8+ds1-1' into buster-fasttrack
gitlab Debian release 13.7.8+ds1-1
2021-03-05 22:25:56 +05:30
Pirate Praveen
0f9ac85fd5 Upload to buster-fasttrack 2021-03-03 00:28:10 +05:30
Pirate Praveen
083733eb08 Merge tag 'debian/13.7.7-2' into buster-fasttrack
gitlab Debian release 13.7.7-2
2021-03-03 00:27:40 +05:30
Pirate Praveen
4e935c048a Remove more obsolete configuration files 2021-03-01 19:15:21 +05:30
Pirate Praveen
3dc1227098 Update minimum versions to ruby-commonmarker and ruby-progressbar 2021-02-28 19:07:54 +05:30
Pirate Praveen
caf0a8742a Update changelog 2021-02-28 19:07:44 +05:30
Pirate Praveen
5bdcc8c88a Merge tag 'debian/13.7.7-1' into buster-fasttrack
gitlab Debian release 13.7.7-1
2021-02-28 00:02:25 +05:30
Pirate Praveen
57a876bb64 Upload to experimental 2021-02-27 22:44:55 +05:30
Pirate Praveen
b0a1be2410 Upload to buster-fasttrack 2021-02-25 19:06:46 +05:30
Pirate Praveen
043cc55d80 Add resolutions: typescript 4.1.5 in package.json 2021-02-25 17:17:37 +05:30
Pirate Praveen
e899d82336 Add --quiet to gitlab-sidekiq.service running bundle install --local 2021-02-20 12:52:00 +05:30
Pirate Praveen
8818323049 Upload to buster-fasttrack 2021-02-19 14:50:35 +05:30
Pirate Praveen
9e74fe3316 Update minimum version of gitaly (for gitaly-git2go command) 2021-02-19 14:50:35 +05:30
Pirate Praveen
5d5509ebc2 Regenerate Gemfile.lock before gitlab-sidekiq service start 2021-02-19 14:50:35 +05:30
Pirate Praveen
141852e541 Upload to buster-fasttrack 2021-02-12 00:54:37 +05:30
Pirate Praveen
ab1ab22510 Change dependency on ruby to ruby2.7 2021-02-12 00:54:37 +05:30
Pirate Praveen
86b89756c6 Merge tag 'debian/13.6.7-1' into buster-fasttrack
gitlab Debian release 13.6.7-1
2021-02-12 00:43:43 +05:30
Pirate Praveen
f6d26205c3 Upload to buster-fasttrack 2021-02-04 23:23:36 +05:30
Pirate Praveen
930d576360 Merge tag 'debian/13.6.6-1' into buster-fasttrack
gitlab Debian release 13.6.6-1
2021-02-04 23:23:05 +05:30
Pirate Praveen
4c5e61eb8d Update minimum version of ruby-attr-encrypted 2021-02-04 15:26:42 +05:30
Pirate Praveen
ccb8d1dc7a Document how to activate a new user from rails console 2021-02-01 15:48:44 +05:30
Pirate Praveen
c0b590a24e Use gitlab-rails-console command in README.Debian 2021-02-01 15:46:34 +05:30
Pirate Praveen
c31bdba3c2 Update upstream source from tag 'upstream/13.6.5'
Update to upstream version '13.6.5'
with Debian dir c236b94d07
2021-02-01 13:23:47 +05:30
Pirate Praveen
40dd71157d Revert "FIx the merge conflict by a patch"
This reverts commit 33cece4870.
2021-02-01 13:18:56 +05:30
Pirate Praveen
1ba4a2d0dc Remove patch that relaxed rugged version 2021-02-01 13:18:31 +05:30
Pirate Praveen
4b415d66a2 Switch back ruby-licensee dependency to 8.9 2021-01-30 21:49:47 +05:30
Pirate Praveen
33cece4870 FIx the merge conflict by a patch 2021-01-30 21:45:13 +05:30
Pirate Praveen
5017262b02 Remove files modified during merge 2021-01-30 21:42:02 +05:30
Pirate Praveen
10cf31694e Upload to buster-fasttrack 2021-01-30 21:35:25 +05:30
Pirate Praveen
d65af223ef Merge tag 'debian/13.6.5-1' into buster-fasttrack
gitlab Debian release 13.6.5-1
2021-01-30 21:34:36 +05:30
Pirate Praveen
986be72bc1 Update path for katex fonts and fix type in path for babel modules 2021-01-26 22:00:57 +05:30
Pirate Praveen
0b8e7e76e3 Update minimum version of node-vue 2021-01-19 23:03:50 +05:30
Pirate Praveen
c120569225 Update minimum version of node-popper.js 2021-01-19 23:03:50 +05:30
Pirate Praveen
6fc5dfe24c Use more packaged modules 2021-01-19 23:03:50 +05:30
Pirate Praveen
fba8257bfc Use packaged versions of mermaid, d3* 2021-01-18 21:19:33 +05:30
Pirate Praveen
5572715e33 Update minimum version of ruby-mini-magick 2021-01-18 18:42:21 +05:30
Pirate Praveen
c69c7012b9 Upload to buster-fasttrack 2021-01-18 16:20:22 +05:30
Pirate Praveen
b105a8b37a Update upstream source from tag 'upstream/13.5.7'
Update to upstream version '13.5.7'
with Debian dir 8b5f6fd9e6
2021-01-18 16:18:16 +05:30
Pirate Praveen
a47ae995f6 Switch to using all node modules via yarn install to fix regression 2021-01-18 15:42:10 +05:30
Pirate Praveen
0139ac7e71 Use a node_modules symlink in /etc/gitlab instead of setting NODE_PATH 2021-01-17 18:26:29 +05:30
Pirate Praveen
c16076696d Switch remaining portal urls to link urls in package.json 2021-01-17 18:26:00 +05:30
Pirate Praveen
a90fb1b0b8 Upload to buster-fasttrack 2021-01-11 17:36:07 +05:30
Pirate Praveen
ba0909160e Fix syntax of nginx.ssl.config.example 2021-01-10 21:06:18 +05:30
Pirate Praveen
3c0cd17605 Upload to buster-fasttrack 2021-01-10 14:00:43 +05:30
Pirate Praveen
c5181aeb77 Relax dependencies to use rugged 0.28 (1.x breaks gitlab) 2021-01-10 14:00:43 +05:30
Pirate Praveen
0730289d8c Merge tag 'debian/13.5.6-1' into buster-fasttrack
gitlab Debian release 13.5.6-1
2021-01-09 13:27:59 +05:30
Pirate Praveen
3b6657b07e Upload to experimental 2021-01-08 16:49:48 +05:30
Pirate Praveen
a15020e7f6 Update upstream source from tag 'upstream/13.5.6'
Update to upstream version '13.5.6'
with Debian dir 4a61941b36
2021-01-08 16:18:43 +05:30
Pirate Praveen
da2143818a Switch back to using lodash from npmjs.com (see #979531) 2021-01-08 16:05:41 +05:30
Pirate Praveen
f9af389e35 Update changelog 2021-01-07 18:47:39 +05:30
Pirate Praveen
2ebba9b3d5 Don't install yarn.lock (we use some dependencies from system) 2021-01-07 18:47:39 +05:30
Pirate Praveen
6de2200cdc Update nginx configuration using upstream templates 2021-01-07 18:47:39 +05:30
Pirate Praveen
6f87dba2f7 Refresh patches (create new patches to fix dependency mismatch) 2021-01-07 18:47:39 +05:30
Pirate Praveen
b832c2c768 Update minimum versions of dependencies 2021-01-07 18:47:39 +05:30
Pirate Praveen
0dc8238efe Install or ignore files newly added in this release 2021-01-04 16:45:18 +05:30
Pirate Praveen
2ff18e2b9f Update upstream source from tag 'upstream/13.5.5'
Update to upstream version '13.5.5'
with Debian dir 30daad7736
2021-01-03 14:30:06 +05:30
Pirate Praveen
e72e22c117 Remove examples that no longer exist in source 2020-12-17 16:51:32 +05:30
Pirate Praveen
f552ba6b2a Upload to buster-fasttrack 2020-12-17 16:41:07 +05:30
Pirate Praveen
693ebb2925 Merge tag 'debian/13.4.7-2' into buster-fasttrack
gitlab Debian release 13.4.7-2
2020-12-17 16:39:44 +05:30
Sruthi Chandran
39503b0376 Update changelog 2020-12-09 23:07:25 +05:30
Sruthi Chandran
a18796fc31 Add canvas as externals in webpack.config.js 2020-12-09 23:02:53 +05:30
Sruthi Chandran
045e956fd3 Update changelog for fasttrack 2020-12-09 16:45:24 +05:30
Sruthi Chandran
0fad823dbb Merge tag 'debian/13.4.7-1' into buster-fasttrack
gitlab Debian release 13.4.7-1
2020-12-09 16:44:26 +05:30
Sruthi Chandran
7d088d5bde Update changelog for fasttrack 2020-12-08 01:31:17 +05:30
Sruthi Chandran
a4587e1bf4 Merge tag 'debian/13.4.6-3' into buster-fasttrack
gitlab Debian release 13.4.6-3
2020-12-08 01:30:26 +05:30
Pirate Praveen
570bd6818a
Upload to buster-fasttrack 2020-12-02 23:39:44 +05:30
Pirate Praveen
de911deaa7
Merge tag 'debian/13.4.6-1' into buster-fasttrack
gitlab Debian release 13.4.6-1
2020-12-02 23:39:07 +05:30
Pirate Praveen
19ec17ded0
Upload to buster-fasttrack 2020-11-07 18:44:44 +05:30
Pirate Praveen
1a2f2fc702
Merge tag 'debian/13.3.9-1' into buster-fasttrack
gitlab Debian release 13.3.9-1
2020-11-07 18:44:24 +05:30
Pirate Praveen
723f89810f
Upload to buster-fasttrack 2020-11-05 14:33:05 +05:30
Pirate Praveen
7c6fc2c353
Update minimum version of ruby-redis-namespace to 1.7 2020-11-05 14:33:04 +05:30
Pirate Praveen
b2708fc26d
Merge tag 'debian/13.3.8-1' into buster-fasttrack
gitlab Debian release 13.3.8-1
2020-11-05 10:37:43 +05:30
Pirate Praveen
224264af66
Fix merge conflict manually 2020-10-12 14:29:18 +05:30
Pirate Praveen
c38d9f301a
Upload to buster-fasttrack 2020-10-12 14:24:10 +05:30
Pirate Praveen
d627340054
Merge tag 'debian/13.2.10-1' into buster-fasttrack
gitlab Debian release 13.2.10-1
2020-10-12 14:23:33 +05:30
Pirate Praveen
88efd90ab3 Upload to buster-fasttrack 2020-09-03 15:24:03 +05:30
Pirate Praveen
72f4ebe592 Upload to buster-fasttrack 2020-09-03 14:48:54 +05:30
Pirate Praveen
40a11540a7 Update minimum version of ruby-websocket-extensions 2020-09-03 14:48:54 +05:30
Pirate Praveen
0664877716 Merge tag 'debian/13.2.8-1' into buster-fasttrack
gitlab Debian release 13.2.8-1
2020-09-03 13:22:22 +05:30
Pirate Praveen
cb38ddd706 Upload to unstable 2020-09-03 11:44:11 +05:30
Pirate Praveen
bb1945acd1 Remove faraday-middleware-aws-signers-v4 from upstream-file-list 2020-09-03 11:44:11 +05:30
Pirate Praveen
5c0409becc Update minimum version of node-jquery to 3.5 2020-09-03 11:42:46 +05:30
Pirate Praveen
da45eefcbf Refresh patches 2020-09-03 11:42:46 +05:30
Pirate Praveen
115c05ed8f New upstream version 13.2.8 2020-09-03 11:42:46 +05:30
Pirate Praveen
5812e0eb52 Drop faraday-middleware-aws-signers-v4 from components list 2020-09-03 11:15:37 +05:30
Pirate Praveen
e98f27b46d Upload to buster-fasttrack 2020-08-29 21:02:33 +05:30
Pirate Praveen
246fd3d9a4 Merge tag 'debian/13.2.6-3' into buster-fasttrack
gitlab Debian release 13.2.6-3
2020-08-29 21:01:44 +05:30
Pirate Praveen
5e81a76aae Upload to buster-fasttrack 2020-08-19 17:03:41 +05:30
Pirate Praveen
8596463c10 Merge tag 'debian/13.2.6-1' into buster-fasttrack
gitlab Debian release 13.2.6-1
2020-08-19 17:03:07 +05:30
Pirate Praveen
a71a29d28f Upload to buster-fasttrack 2020-08-19 00:45:56 +05:30
Pirate Praveen
463b215bb5 Merge tag 'debian/13.2.5-1' into buster-fasttrack
gitlab Debian release 13.2.5-1
2020-08-19 00:45:04 +05:30
Pirate Praveen
492c8befdf Upload to buster-fasttrack 2020-08-15 18:43:03 +05:30
Pirate Praveen
e9dfcf6c1f Merge tag 'debian/13.2.3-2' into buster-fasttrack
gitlab Debian release 13.2.3-2
2020-08-15 18:40:44 +05:30
Pirate Praveen
f88c3a10a4 Bump minimum version of webpack to 4.43 2020-08-15 18:36:40 +05:30
Pirate Praveen
24050d4920 Upload to buster-fasttrack 2020-08-11 01:25:23 +05:30
Pirate Praveen
179fa2deaa Tighten dependency on ruby-gitaly and ruby-acme-client 2020-08-11 01:25:23 +05:30
Pirate Praveen
1ca465c8fe Merge tag 'debian/13.2.3-1' into buster-fasttrack
gitlab Debian release 13.2.3-1
2020-08-10 15:59:56 +05:30
Pirate Praveen
99df46efad Upload to buster-fasttrack 2020-08-09 16:28:48 +05:30
Pirate Praveen
7483c79c80 Merge tag 'debian/13.1.6-1' into buster-fasttrack
gitlab Debian release 13.1.6-1
2020-08-09 16:26:49 +05:30
Pirate Praveen
18e3186139 Upload to buster-fasttrack 2020-07-16 19:42:14 +05:30
Pirate Praveen
7bc7cabff2 Merge tag 'debian/13.1.4-2' into buster-fasttrack
gitlab Debian release 13.1.4-2
2020-07-16 19:41:45 +05:30
Pirate Praveen
2787fbd1f1 Upload to buster-fasttrack 2020-07-14 13:18:43 +05:30
Pirate Praveen
cbc0b1e267 Drop ruby 2.5 compat patches, we have ruby 2.7 in fasttrack now 2020-07-14 13:17:52 +05:30
Pirate Praveen
9ea06aadb1 Upload to buster-fasttrack 2020-07-11 21:28:02 +05:30
Pirate Praveen
cc38475d45 Update minimum version of ruby to 2.7 2020-07-11 21:27:20 +05:30
Pirate Praveen
d41a45e8f9 Upload to buster-fasttrack 2020-07-11 13:15:16 +05:30
Pirate Praveen
9c3338d303 Merge tag 'debian/13.1.4-1' into buster-fasttrack
gitlab Debian release 13.1.4-1
2020-07-11 13:14:37 +05:30
Pirate Praveen
03be9d8b48 Upload to buster-fasttrack 2020-07-09 17:01:43 +05:30
Pirate Praveen
a35072182a Merge tag 'debian/13.1.3-1' into buster-fasttrack
gitlab Debian release 13.1.3-1
2020-07-09 16:56:22 +05:30
Pirate Praveen
e6b762e584 Upload to buster-fasttrack 2020-07-02 16:06:34 +05:30
Pirate Praveen
94d8eaaec5 Merge tag 'debian/13.1.2-1' into buster-fasttrack
gitlab Debian release 13.1.2-1
2020-07-02 16:05:58 +05:30
Pirate Praveen
234950d498 Upload to buster-fasttrack 2020-07-02 01:18:39 +05:30
Pirate Praveen
c37b12dff7 Add more ruby 2.5 compat fixes 2020-07-02 01:17:14 +05:30
Pirate Praveen
eff5e491e4 Upload to buster-fasttrack 2020-07-01 19:17:24 +05:30
Pirate Praveen
4915a9b22c Merge tag 'debian/13.1.1-1' into buster-fasttrack
gitlab Debian release 13.1.1-1
2020-07-01 19:16:51 +05:30
Pirate Praveen
c34c73eafa Upload to buster-fasttrack 2020-06-30 17:46:16 +05:30
Pirate Praveen
0dfa45f101 Add patch to fix ruby 2.5 compatibility issue 2020-06-30 17:44:52 +05:30
Pirate Praveen
d387b65fa2 Update minimum version of ruby-oauth2 to 1.4.4 2020-06-30 01:52:15 +05:30
Pirate Praveen
1a8395f96e Drop patches (changes already included in other patches) 2020-06-30 01:52:15 +05:30
Pirate Praveen
4aae01e459 Update changelog 2020-06-30 01:52:05 +05:30
Pirate Praveen
d6bb874986 Merge tag 'debian/13.1.0-1' into buster-fasttrack
gitlab Debian release 13.1.0-1
2020-06-30 00:41:08 +05:30
Pirate Praveen
f73fa2d1b1 Upload to buster-fasttrack 2020-06-28 15:43:03 +05:30
Pirate Praveen
492ea4810c Merge tag 'debian/13.0.6-2' into buster-fasttrack
gitlab Debian release 13.0.6-2
2020-06-28 15:41:42 +05:30
Pirate Praveen
8680a174b6 Upload to experimental 2020-06-27 23:08:08 +05:30
Pirate Praveen
290aa15f4e Relax dependency on ruby-grape-entity 2020-06-27 23:07:23 +05:30
Pirate Praveen
e23b02efe9 Add rack-timeout gem to Gemfile 2020-06-27 23:05:49 +05:30
Pirate Praveen
645c713439 Switch to using structure.sql from schema.rb for database initialization 2020-06-27 22:49:49 +05:30
Pirate Praveen
abc0d8f89b Upload to buster-fasttrack 2020-06-13 14:36:34 +05:30
Pirate Praveen
aa128a8c0f Merge tag 'debian/13.0.6-1' into buster-fasttrack
gitlab Debian release 13.0.6-1
2020-06-13 14:35:54 +05:30
Pirate Praveen
8819babaeb Upload to buster-fasttrack 2020-06-04 23:35:31 +05:30
Pirate Praveen
a232f5c0c2 Merge tag 'debian/13.0.4-1' into buster-fasttrack
gitlab Debian release 13.0.4-1
2020-06-04 23:34:37 +05:30
Pirate Praveen
163b24d00a Upload to buster-fasttrack 2020-05-31 21:30:41 +05:30
Pirate Praveen
54aa708174 Update minimum version for rm_conffile to include 12.10.7 2020-05-31 21:30:41 +05:30
Pirate Praveen
3050621a11 Tighten dependency on ruby-gitaly and yarnpkg 2020-05-31 21:30:41 +05:30
Pirate Praveen
21bfb841c1 Fix dpkg-source failing to apply patch 2020-05-31 02:17:09 +05:30
Pirate Praveen
b3d6b97966 Merge tag 'debian/13.0.3-1' into buster-fasttrack
gitlab Debian release 13.0.3-1
2020-05-31 01:56:36 +05:30
Pirate Praveen
d11626294f Upload to buster-fasttrack 2020-05-29 22:30:04 +05:30
Pirate Praveen
c3652828f4 Merge tag 'debian/12.10.7-1' into buster-fasttrack
gitlab Debian release 12.10.7-1
2020-05-29 22:29:18 +05:30
Pirate Praveen
406c71498e Upload to experimental 2020-05-28 23:31:58 +05:30
Pirate Praveen
1c9886d0ee Refresh patches (remove fuzz) 2020-05-28 23:30:46 +05:30
Pirate Praveen
1cfa946ec8 Update upstream source from tag 'upstream/12.10.7'
Update to upstream version '12.10.7'
with Debian dir cbc7381234
2020-05-28 23:00:39 +05:30
Pirate Praveen
33a3b36430 New upstream version 12.10.7 2020-05-28 22:55:37 +05:30
Pirate Praveen
196473dfda Upload to buster-fasttrack 2020-05-09 18:03:57 +05:30
Pirate Praveen
b5e44695ef Merge tag 'debian/12.10.3-1' into buster-fasttrack
gitlab Debian release 12.10.3-1
2020-05-09 18:03:20 +05:30
Pirate Praveen
4bc305fae7 Upload to buster-fasttrack 2020-05-01 14:06:29 +05:30
Pirate Praveen
4992f2cf12 Merge tag 'debian/12.10.2-1' into buster-fasttrack
gitlab Debian release 12.10.2-1
2020-05-01 14:05:38 +05:30
Pirate Praveen
c8e819d1e7 Upload to buster-fasttrack 2020-04-27 15:09:57 +05:30
Pirate Praveen
916b97853d Merge tag 'debian/12.10.1-1' into buster-fasttrack
gitlab Debian release 12.10.1-1
2020-04-27 15:08:56 +05:30
Pirate Praveen
b01f44a0f9 Upload to buster-fasttrack 2020-04-23 18:47:39 +05:30
Pirate Praveen
f50b703e68 Merge tag 'debian/12.10.0-1' into buster-fasttrack
gitlab Debian release 12.10.0-1
2020-04-23 18:22:19 +05:30
Pirate Praveen
ca8817506a Upload to buster-fasttrack 2020-04-15 21:53:17 +05:30
Pirate Praveen
5950f6f711 Merge tag 'debian/12.9.3+dfsg-1' into buster-fasttrack
gitlab Debian release 12.9.3+dfsg-1
2020-04-15 21:29:50 +05:30
Pirate Praveen
df49f228a4 Upload to buster-fasttrack 2020-04-12 18:53:21 +05:30
Pirate Praveen
a7bb45eb22 Merge tag 'debian/12.9.2-5' into buster-fasttrack
gitlab Debian release 12.9.2-5
2020-04-12 18:52:32 +05:30
Pirate Praveen
c2995daa06 Upload to buster-fasttrack 2020-04-11 20:31:17 +05:30
Pirate Praveen
1ca72cd760 Merge tag 'debian/12.9.2-4' into buster-fasttrack
gitlab Debian release 12.9.2-4
2020-04-11 20:30:45 +05:30
Pirate Praveen
50accf9085 Exclude packaged modules from transpiling (webpack/babel-loader) 2020-04-11 19:51:50 +05:30
Pirate Praveen
d428561424 Remove --max-old-path option to webpack (correctly passed in NODE_OPTIONS) 2020-04-11 19:47:36 +05:30
Pirate Praveen
778cc57fa7 Upload to buster-fasttrack 2020-04-11 17:02:14 +05:30
Pirate Praveen
014a34d1c1 Merge tag 'debian/12.9.2-3' into buster-fasttrack
gitlab Debian release 12.9.2-3
2020-04-11 17:01:31 +05:30
Pirate Praveen
93f8f729e2 Update changelog and upload to buster-fasttrack 2020-04-11 00:48:41 +05:30
Pirate Praveen
1bdbd8e754 Pass NODE_OPTIONS="--max-old-space-size=2048" to webpack to work with nodejs 10 2020-04-11 00:48:06 +05:30
Pirate Praveen
067a3f7c5c Tighten dependency on ruby-graphql (for backports) 2020-04-10 22:19:32 +05:30
Pirate Praveen
389424f753 Upload to buster-fasttrack 2020-04-10 22:18:57 +05:30
Pirate Praveen
298f1c9f45 Merge tag 'debian/12.9.2-1' into buster-fasttrack
gitlab Debian release 12.9.2-1
2020-04-10 20:08:32 +05:30
Pirate Praveen
114ca03098 Upload to fasttrack 2020-04-04 16:20:17 +05:30
Pirate Praveen
199c725982 Merge tag 'debian/12.8.8-6' into buster-fasttrack
gitlab Debian release 12.8.8-6
2020-04-04 16:19:28 +05:30
Pirate Praveen
7fc5c902d0 Upload to buster-fasttrack 2020-04-03 20:01:35 +05:30
Pirate Praveen
a6158837ca Drop dependency on libjs-pdf (not needed anymore) 2020-04-03 20:00:26 +05:30
Pirate Praveen
12cf7d8c8f Upload to buster-fasttrack 2020-04-03 18:02:52 +05:30
Pirate Praveen
3c9f59f379 Merge tag 'debian/12.8.8-5' into buster-fasttrack
gitlab Debian release 12.8.8-5
2020-04-03 18:02:09 +05:30
Pirate Praveen
be8a4cfeb8 Tighten dependency on ruby-grape-path-helpers 2020-04-01 20:24:18 +05:30
Pirate Praveen
2576d2ad65 Tighten dependencies for dependencies in buster-backports 2020-04-01 13:06:27 +05:30
Pirate Praveen
089937af1c Upload to buster-fasttrack 2020-04-01 09:31:39 +05:30
Pirate Praveen
520f3ea139 Merge tag 'debian/12.8.8-2' into buster-fasttrack
gitlab Debian release 12.8.8-2
2020-04-01 09:30:35 +05:30
Pirate Praveen
6746c2f57c Upload to buster-fasttrack 2020-03-08 10:45:42 +05:30
Pirate Praveen
2fd88176e8 Merge tag 'debian/12.6.8-1' into buster-fasttrack
gitlab Debian release 12.6.8-1
2020-03-08 10:44:36 +05:30
Pirate Praveen
c112c9e6de Upload to buster-fasttrack 2020-02-22 14:37:26 +05:30
Pirate Praveen
b399c68f5c Merge tag 'debian/12.6.7-1' into buster-fasttrack
gitlab Debian release 12.6.7-1
2020-02-22 14:35:53 +05:30
Pirate Praveen
507653d6b6 Upload to buster-fasttrack 2020-01-18 21:32:58 +05:30
Pirate Praveen
99c236f9a8 gitlab Debian release 12.6.4-1
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEsclPZxif+sAmSPvz1N6yL8C5bhUFAl4czhMACgkQ1N6yL8C5
 bhV+Cg/8DgVtoUfIMoipeKqp5H4f2HeVS4OME3ys543Wuxkh6erqedhZVFDv0nmk
 kINjSo7hU6dMRWImeZWWWFFJ+4wl4z629u0JXancD/0fJgaZplWPhR0RbjRIIRRZ
 flzFBJzsyhh+Ata1hP80q0+MasMjSIiCEJr1ocH19W4UcPc4gZbCQJuY8CA//RPz
 25bqsVwW8hWrPxFQCVfJmshQQViSLz3/IBJJLwo+uQwcpEbNCafg8Ux3Ulv9QSB2
 +7sTttI8oDGgjZvCpfH7ekP2yyw8J7OXgy0z3G5f0Dr53hUJlGPibovAz6mgwagN
 353tB6Nm53W15LiUHHf2XbmV34r7HrQPnXysuN/HAskbMRK/7WcHdbrm5LFeOUIO
 CBp7jQE9+aU6QIyMoUQpu8uxKfO5MuYPKbL1I6j1K99ERQqBaecWxnRmGw87v0ZL
 irVm/TFUkeVX7w3K5BrMjez3B8+9l1Z6YcMVy1k4BlUtDzHrsZg6JC02JLchEHEI
 v+USQZs2r3Gr0oUdoqEKB5Hf7NVdEzWAdUUDyQkPM6qd9FZ6f2OcoM34k5VIJ9B5
 WAfKuL4KxLTjNoYy3NHbmcQF+GRI5BuKT8Kifijt1sLa2uYwt93OPfnmE/fP0nnY
 IxLwby5/K3feyEjta3ktlJNpn+S1Hou62nhG8dBF56ZmQY8Z99g=
 =Oo87
 -----END PGP SIGNATURE-----

Merge tag 'debian/12.6.4-1' into buster-fasttrack

gitlab Debian release 12.6.4-1
2020-01-18 21:31:59 +05:30
Pirate Praveen
b39cc878df Fix last version of obsolete config file 2020-01-08 20:45:48 +05:30
Pirate Praveen
d4d38e6549 Update changelog 2020-01-08 19:27:22 +05:30
Pirate Praveen
1e0a27c6cd gitlab Debian release 12.6.2-2
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEsclPZxif+sAmSPvz1N6yL8C5bhUFAl4VtNwACgkQ1N6yL8C5
 bhWAZQ//fnm5InpjIq8ujEuKOJ+PhhOn1mh38eeADK6kC4Pcw7yjQIrCJU3R0bdZ
 7+pSjdRQ8Qy+cuCKFGjCoESai2xsHnJvo18GWRmfa1JrNEjpulCCFU6GbIOQtwcs
 O+dhCwbbKPNEpPjLX9nMZbDi2TQONP4msjluYFRn/kceN8io0BOAgkzhxzlKX2o6
 10hQLPXhIaWaM7ePIupgIshHURIcno7wRV/JdK4ZIp8/aS1OcJpASzpnqJqwAQ5I
 J4XgPdGVtwFX1kE9GV2DD5v9iBQJ2T42lIVEIhQuAqYrZsrP+Ln7O12CInWYnvEV
 4leWSHbRS0PzRJF+iC5HhY0pr92LDrko1uiRC6o5VGzzGdpTfHD19hVHZQp6lkAL
 WDE4qzDUt9VcEscbHR54s0gd1NEHLbrDiqkoXCu3dkAFAz5zos8CzA0g2vsrkfK1
 1wZbGyim7yjZoHbQZcYY4LWPms9t2gfIaFVEo12AIQXcZJLsBtNzBcLvaIGTRTyM
 96jkDzrr0+BKjfRBCddH+OgfXi5HxW1sv5EPjjuf5yzu9GuJ7oliaFpqVoUtScpn
 Rh8cgunzpTEBZs/hPoVmDlCvsANMZwGJQthGcSaCd4KuIahnfg+Xu+TMEOsBTGJr
 Rm3l11sAUBj8vBO8M6exGaKMV4bB226QCjNA92TUTiGQHDg26WU=
 =KxY2
 -----END PGP SIGNATURE-----

Merge tag 'debian/12.6.2-2' into buster-fasttrack

gitlab Debian release 12.6.2-2
2020-01-08 19:26:21 +05:30
Pirate Praveen
817faa72c9 Update changelog 2020-01-08 19:25:20 +05:30
Pirate Praveen
361b18f17d Update changelog 2019-12-31 16:00:04 +05:30
Pirate Praveen
5d4267de19 Merge tag 'debian/12.5.4-2' into buster-fasttrack
gitlab Debian release 12.5.4-2
2019-12-31 15:59:07 +05:30
Pirate Praveen
f1fbb90e0c Update minimum version of ruby-gpgme to 2.0.19 2019-12-31 15:35:09 +05:30
Pirate Praveen
0f12edca41 Update changelog 2019-12-31 14:27:51 +05:30
Pirate Praveen
349431e7aa Merge tag 'debian/12.5.4-1' into buster-fasttrack
gitlab Debian release 12.5.4-1
2019-12-31 14:26:51 +05:30
Pirate Praveen
e48bb38abb Update minimum version of ruby-rouge to 3.11 2019-12-24 00:12:02 +05:30
Pirate Praveen
7d28908c0d Update changelog 2019-12-23 23:23:44 +05:30
Pirate Praveen
99e79781ea Merge branch 'master' into buster-fasttrack 2019-12-23 23:21:02 +05:30
Pirate Praveen
ddd3ad69ba Update changelog 2019-12-20 20:05:41 +05:30
Pirate Praveen
296afc5841 Merge branch 'master' into buster-fasttrack 2019-12-20 20:04:05 +05:30
Pirate Praveen
e170d45429 Update changelog 2019-11-29 20:57:39 +05:30
Pirate Praveen
b9f1850e6c Merge branch 'master' into buster-fasttrack 2019-11-29 20:56:59 +05:30
Pirate Praveen
8f0e03c6c8 Update changelog for fasttrack release 2019-11-29 14:13:58 +05:30
Pirate Praveen
dece181ffb Merge tag 'debian/12.2.9-5' into buster-fasttrack
gitlab Debian release 12.2.9-5
2019-11-29 14:05:00 +05:30
Pirate Praveen
727e607415 Update changelog 2019-11-15 19:15:28 +05:30
Pirate Praveen
e772c39290 Update minimum version for rm_confifile maintscript option 2019-11-15 19:11:24 +05:30
Utkarsh Gupta
0031db7120 Update d/ch 2019-10-31 02:33:23 +05:30
Utkarsh Gupta
21f7e2d18a gitlab Debian release 12.2.9-1
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEbJ0QSEqa5Mw4X3xxgj6WdgbDS5YFAl258+oACgkQgj6WdgbD
 S5aFXRAAtWqHzYQFavQ23NXZn5QkFGKuUKqq31EtHrdgVyCW1wdTif36es4IpBfK
 8zJ+WFllW+cOUnfkVfmbG4ygEVJbEhRpwaYN1gO4QrVvsumbtq55N90mJvYsD2Fy
 JdgbB0g0vz/qUxyAxZW2zmQAOHdi3egfrHBL9L04oZgfXq0dpTrL+YllADFRJrgH
 y2jvU6W0zZ/HPdpoKg3ejaGI1P5rmxFVzX9/jEyPhwpHGvBfChPCE0g0sKn9isk9
 151QE0T+0fbHWez4cQ723vMv55+J3GO5HNNkeUWzuXzogga1piQ4C0CtqP3fpu8+
 a8nf6IqQ7ToA0g5nQeMIXsUnux8muA66zxjiyWqyln7rssCvNUy/OOHKeC9JeoRJ
 tyuBZrU5XoLHKXgxwPLJKFNe+DuMRGn6m1IQmTOgaEc7feyZQeS0e2QQ7bYd46Zp
 SHkUx7luIEu2B44t/ioFzCbXj6Vb3ybKY+n1CKOymb1OSyibpgFKxe/1m5D6sbSX
 aFvt44OZWpaTAbWK7YYE7QDDGK2dXU1VnFxxgE1wEqNv92qk0bOFLp2mgsJTfD1H
 RtRV06JsVPsZo0pnnpedi1MSgmzham9O/hG8I71f4xlRofN+Tjq2sTYZtlDi43MD
 rMhkKd9OgaNUWpIZHNefiqhRzZEiJsZe/8DDmti8TGpRAeDP0SI=
 =ebh2
 -----END PGP SIGNATURE-----

Merge tag 'debian/12.2.9-1' into buster-fasttrack

gitlab Debian release 12.2.9-1
2019-10-31 02:17:46 +05:30
Dmitry Smirnov
cc29107cef CI: re-factored [skip ci] 2019-10-25 02:15:23 +11:00
Pirate Praveen
ef492edd14 Update changelog 2019-10-22 19:16:00 +05:30
Pirate Praveen
e5533d897c Update minimum version of gitaly to 1.53.3 2019-10-22 19:13:40 +05:30
Utkarsh Gupta
4fa910a2b5 Update d/ch for buster-fasttrack 2019-10-22 15:06:01 +05:30
Utkarsh Gupta
a14a56c217 gitlab Debian release 12.1.14-1
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEbJ0QSEqa5Mw4X3xxgj6WdgbDS5YFAl2uzGUACgkQgj6WdgbD
 S5ZYjw//aK8dS4kXeVgR0sD1T41hFvihmy8Qjj9dT2hx8S2QRK8RZgLUTCwvEfG9
 pfFLXxZAGRkqHdQ9a+eGexWI/5wYU1KDTknV6pKpWihpWVPbrrC67Er2ZO4Xc1kR
 xE35sW56Px04taXDLCJqUkVYUu+F2EkvUsSx0/FgXK0JqNKC9qb1cnDzJCaxZOhq
 dleTiY2CMIP7sKq3g0UnpJFDjY1703TKj0sNDbL9ik7hZJ9wf0c9r8b5Iu5kPXUD
 v5gTXZnDPPydRs5IL/vOVU+iXrkqrfAaQyQkfeQaOzOOLy8ALNXEoUhyJOvi4ivP
 d5jGkUO6hf7ordOtUT4FOYLEAkflco+oCGltM++tmefTLIpugHZPQwQziHjnNQaB
 U6wznMUynluu9qQiZSckWkHqxQPxvCVS4yr8H6IHng8y1bovx1CGqUVfOCt5MurQ
 e2286dLaZ+C1/se9EH8S1gPutiTYc1kv/ljfIx8JdlyfHnyGa5C3+oNQ+mN4Z+kO
 N1FiFoI7O/o6PROYxiS8nxNE5HnKgohOohIoflvWM0svdLx/L0BGI/4buVfWbpsA
 cOslt3mC1HGFx81AiVMQzB+Do/iZUG/WGVK52MgrvqDKhtAZKAQ2XSXwnGEx++Gj
 iRntL5Z2JKcbfvVxzuiTCaogaq6YJpDJRxgRzsUK0i8pUkSOUn0=
 =TTgt
 -----END PGP SIGNATURE-----

Merge tag 'debian/12.1.14-1' into buster-fasttrack

gitlab Debian release 12.1.14-1
2019-10-22 15:02:19 +05:30
Pirate Praveen
5357f91bab Add node-core-js (<< 3~) constraint to use embedded core-js module 2019-10-07 00:06:50 +05:30
Pirate Praveen
3005232959 Update changelog 2019-10-06 23:52:26 +05:30
Pirate Praveen
8563fafbe4 Merge tag 'debian/12.1.13-2' into buster-fasttrack
gitlab Debian release 12.1.13-2
2019-10-06 23:51:24 +05:30
Dmitry Smirnov
c735470424 CI: minor update 2019-10-04 09:06:09 +10:00
Utkarsh Gupta
7795860bef Update d/ch for buster-fasttrack rebuild 2019-09-12 14:00:43 +05:30
Utkarsh Gupta
74ab46c1ab Merge tag 'debian/12.0.9-1' into buster-fasttrack
gitlab Debian release 12.0.9-1
2019-09-12 13:56:15 +05:30
Pirate Praveen
d86b4ebc5e Update changelog 2019-09-11 12:14:15 +05:30
Pirate Praveen
d0e3e28998 Merge tag 'debian/12.0.8-3' into buster-fasttrack
gitlab Debian release 12.0.8-3
2019-09-11 12:11:44 +05:30
Pirate Praveen
a30f3cfa00 Update changelog 2019-08-14 17:40:36 +05:30
Pirate Praveen
8f9fd99498 gitlab Debian release 11.11.8+dfsg-1
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEsclPZxif+sAmSPvz1N6yL8C5bhUFAl1T9iQACgkQ1N6yL8C5
 bhVwQw//aD3Auw6HLlF/6yTVxIhf5+dx/X/UQ2V2etf433KyO4VrEUSPHnvtg3go
 gmZXm+1o19u93AhnWjC892uqWCNFEdHCVpliHcbDjGy8YI84wFuFO8hWByWYhEhf
 4QrasySvmiP/VSjLtZFNETBYhS3+qjGtH1c1DuddzNUnsjBKg/B4BO7N4yljLkDa
 rM5kMgWY9/oU4QGLqV6pWIEhcvMWPQMAt0lBjzpcTjW3PgxfkAqEwdJ7IJhEiDaE
 wJWvLMksLLmEowFxvwSK66im/WlOrKDNX7w2qW1r7kQGy4xk+oDSS6sjzf2FOx8l
 jlUqynB95nWeIOHycfe/vVAxvQwBTzZuanRq9jO1a36p/Hh7uOaPvMCEeLf2bykc
 HZJxxLrph4PUXLbIRy+u/lf/xyJ9nMQdhpvKZE3tYNA+dx/xduWaIzwlPKoxmXgC
 qNoP9F3To/TBhCez5wtM7TziPqh5VS1fPIk5CBLFr5G0zO+b0ohhiIHQnWiBa5iX
 n6Cy60BK0XMeQwlMuE6VrAjoixBZ/z1wJWnDiWHyGc4f6n6lD25qnC1WfkzQH9VH
 WJHf5g+H+jzRBPM7SdiLQ4rTQsPIztpvuwj7X6KTddY/F7usTFFYuRJNMJ6YDz8G
 u0pqhel8aUmXjQSH6lFfLMEeVr2VQ/ovngwtzdzsWMPEOtKGUdc=
 =8+DS
 -----END PGP SIGNATURE-----

Merge tag 'debian/11.11.8+dfsg-1' into buster-fasttrack

gitlab Debian release 11.11.8+dfsg-1
2019-08-14 17:39:10 +05:30
Pirate Praveen
0b56d3d594 Use packaged version of webpack 2019-08-08 12:08:50 +05:30
Pirate Praveen
509c56361d Update dependency on ruby-google-protobuf 2019-08-07 00:42:33 +05:30
Pirate Praveen
1a2640645f Ready for upload 2019-08-06 15:58:49 +00:00
Pirate Praveen
1a337d5fe1 Merge branch 'master' into buster-fasttrack 2019-08-06 15:57:03 +00:00
Pirate Praveen
52886cd1ac Update changelog 2019-07-07 14:52:52 +05:30
Pirate Praveen
3a9fffe295 Merge tag 'debian/11.10.8+dfsg-1' into buster-fasttrack
gitlab Debian release 11.10.8+dfsg-1
2019-07-07 14:50:48 +05:30
Pirate Praveen
628e6475a9 Update changelog 2019-06-05 15:41:20 +05:30
Pirate Praveen
bc16220487 gitlab Debian release 11.10.5+dfsg-1
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEKnl0ri/BUtd4Z9pKzh+cZ0USwioFAlz3eFAACgkQzh+cZ0US
 wiqp6g//TVGgcAeuzFiRwBEDXy6e4Deybmq5IMAbgogczfT5iwkHDtKVvunM2vty
 3nsuYTlsv6dHLSDXwUHrubwz4bQW6mPzsUZw6492UjHrb2NYFZXtQY4xfYq9at2R
 0dQFky8HgoT/mxlsz4BwUcCsm6Xp+/abPj4oROkeH+4hKkbKiSmqXwI6nwSfmPJj
 ILYNF7FgRlxGTmngSoYtMiGK9Of2Hvkkl8lRDTDBZu5J/OyLkd8lekeQZVoXrOKF
 JF8jKawNskm7NFs6ee8gceEQ0wvhsOgZnbcHbJpmip0mp9ODntkVAJt4XbIE3i/m
 GeAKHoVoxVwTA12fBaGtua2B/NfcT94+hOWNuoCxgZywd8YqMKrJY5L0WydBz4dN
 Y/dBW028QKuob9XoG9vEZSOvpJP2lqBLUlWg3vB/pzQOQl5/JHccGZ9RKtu5J7dQ
 TghzBrCdTttSA57VZO7PJ3rAwkf8cPhRdxJES1dvDem8qm8xI3Au4kUNpfrwV9HO
 uXXMiaPzIjHgVk79xhmVnhPcfpkHBvAdwvaZziTuiHNoZ4M4dKVhjItxHf7IK0p7
 6yYh0ylbo4+HVQngxBpL4LqUYeqyc8w7fwSWafdpTg92nx1Gw+T+SlqG6ZT8pOvd
 PvSb4pv0bKFeMFTkMto/c2RlZqeEBF9ABiiOhK+HMBH+YsM7aGI=
 =majb
 -----END PGP SIGNATURE-----

Merge tag 'debian/11.10.5+dfsg-1' into buster-fasttrack

gitlab Debian release 11.10.5+dfsg-1
2019-06-05 14:18:22 +05:30
Pirate Praveen
4e1e88068c Update changelog 2019-05-06 19:56:18 +05:30
Pirate Praveen
7ba2e71134 gitlab Debian release 11.8.9+dfsg-1
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEKnl0ri/BUtd4Z9pKzh+cZ0USwioFAlzP+foACgkQzh+cZ0US
 wir2OA//d+yo+OaGt5tBQoWkTJgSAIhT058R0KsYVrfzhhqViSiXSowmMzAV3UYQ
 izlKUrWJKws6QypmePoSHmxDMIvt4tdK2z44btteHBzYX62nXuI7mFuyFWe7608G
 ZMAmjLhiXmfQA3bmZ1Rg4CZdmi/1PUqI5upsstTW7ipvyOm7ir8TIHn5t/YHBo48
 BjzUu/gb/7llfW2jPZTTQs3QgPVPxcBSJBQIj3cZOzC3HYGpMCp3JBoZm1UyG4CO
 0Fy0toYdEtsZETuJ+YQo9F+XS9JV4ggVeSri1HOrtQgMbtZprwUrlegIOEIgZfQ0
 xJhZjph8dP5859FnAOw9JqkuLKyItZmdeid3ed5Nb6PIJ7JL02RIbcRHxT8QPM5g
 MVZZr9+CCuFA3Y3ZLtqlbK0txLnPAdLMw5QEAbC4giqwjHsDPYBuQ1hymAgGvT5P
 5KVUlqp2OOruV31IdqeOOKZK4z0TBESXv5K60uqWBV4MwW1Ll3gvpsHRB+jP1U4V
 vvvZzXqk2sSnECSqlEGB6ltStIqLJgYwAKFRldSNFvePFkFYELVBjlTN+4zt2Efc
 Oz1nPzIHA6BKapztpjQLNgEN3UvsSu5WRidpT9lRInfFG4/iCw2hr071HNREia9u
 +aAw2zHT8XkPuPjZ5ehXdzPnPNDdnSvaSXNfU/apx0Scqm38H64=
 =S/J1
 -----END PGP SIGNATURE-----

Merge tag 'debian/11.8.9+dfsg-1' into buster-fasttrack

gitlab Debian release 11.8.9+dfsg-1
2019-05-06 19:50:45 +05:30
Utkarsh Gupta
21ee9334f8 Fix typo in d/patches/0050-* 2019-05-05 23:28:57 +05:30
Utkarsh Gupta
e9cd9414e4 Merge tag 'upstream/11.8.9+dfsg' into master-11.8
Upstream version 11.8.9+dfsg
2019-05-05 18:52:01 +05:30
Utkarsh Gupta
511555a717 New upstream version 11.8.9+dfsg 2019-05-05 18:51:08 +05:30
Utkarsh Gupta
2771465d7b Update d/copyright to exclude pdf.* 2019-05-05 18:15:54 +05:30
Utkarsh Gupta
7f10525869 Update d/upstream-file-list 2019-05-05 18:14:10 +05:30
Utkarsh Gupta
8738e1d7a0 Update d/patches 2019-05-05 18:12:09 +05:30
Utkarsh Gupta
b33747df94 Update upstream source from tag 'upstream/11.8.9+dfsg'
Update to upstream version '11.8.9+dfsg'
with Debian dir 8a8fa4ffef
2019-05-05 18:10:48 +05:30
Utkarsh Gupta
add68ad8a8 Update d/upstream-file-list 2019-05-05 17:33:51 +05:30
Utkarsh Gupta
b7eae1279a Add patch to install .babelrc.js 2019-05-05 17:02:35 +05:30
Utkarsh Gupta
a301ba6ec1 Update d/ch 2019-05-05 16:55:34 +05:30
Utkarsh Gupta
80fa3cd031 Update upstream source from tag 'upstream/11.8.9+dfsg'
Update to upstream version '11.8.9+dfsg'
with Debian dir e6ecb7368e
2019-05-05 16:48:12 +05:30
Utkarsh Gupta
eb631e4ebf Remove ee 2019-05-05 01:36:48 +05:30
Utkarsh Gupta
d8d9537d7b Update d/changelog 2019-05-04 17:41:57 +05:30
Utkarsh Gupta
8fa5059027 Update d/patches/0680-* 2019-05-04 17:41:34 +05:30
Utkarsh Gupta
cc33652d35 Update d/patches/0470-* 2019-05-04 17:41:19 +05:30
Utkarsh Gupta
3eb051e84b Update d/patches/0290-* 2019-05-04 17:41:08 +05:30
Utkarsh Gupta
b12e0ea435 Update d/patches/0050-* 2019-05-04 17:40:41 +05:30
Utkarsh Gupta
ef79d2c859 Update upstream source from tag 'upstream/11.8.9'
Update to upstream version '11.8.9'
with Debian dir a2db60371b
2019-05-04 17:39:32 +05:30
Utkarsh Gupta
f8513db4ea New upstream version 11.8.9 2019-05-04 17:39:32 +05:30
Pirate Praveen
2b7a38adef Update changelog 2019-04-30 15:33:27 +05:30
Pirate Praveen
587ea54adc Merge tag 'debian/11.8.6+dfsg-1' into buster-fasttrack 2019-04-30 15:20:33 +05:30
Abhijith PA
871f06b578 update changelog 2019-04-19 08:33:19 +05:30
Abhijith PA
8eb1be7eee update changelog 2019-04-19 08:31:38 +05:30
Abhijith PA
cf9f890a3f add gitlab.examples 2019-04-19 08:24:31 +05:30
Abhijith PA
d9db5ce207 remove embedded pdf.js and use libjs-pdf 2019-04-19 08:19:27 +05:30
Abhijith PA
52dc8ac99a Update upstream source from tag 'upstream/11.8.6+dfsg'
Update to upstream version '11.8.6+dfsg'
with Debian dir 8d6e790779
2019-04-19 08:12:50 +05:30
Abhijith PA
fe4b85100d update changelog 2019-04-06 11:59:16 +05:30
Abhijith PA
dd2b680649 update upstream-file-list with ee 2019-04-06 11:57:00 +05:30
Abhijith PA
288e956cde update changelog 2019-04-03 18:24:31 +05:30
Abhijith PA
e160278ed2 Update upstream source from tag 'upstream/11.8.6'
Update to upstream version '11.8.6'
with Debian dir 7e57ab6d69
2019-04-03 18:18:56 +05:30
Abhijith PA
540c00d914 New upstream version 11.8.6 2019-04-03 18:18:56 +05:30
Pirate Praveen
c901217746 Update changelog 2019-03-22 21:22:28 +05:30
Pirate Praveen
70c9743e18 gitlab Debian release 11.8.3-1
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEsclPZxif+sAmSPvz1N6yL8C5bhUFAlyT3qIACgkQ1N6yL8C5
 bhXi5Q//bdF+ms+S3H2dPNzKb4fbA0UP3nfdHkjhUIcFP2UV4yVibm3uKpEH82W4
 r+LjqtFhQmtbqwNzPZk/xf7+AP6PEcDCck82zK1YEBOJKUuBY+4G4Y7vciOjApwf
 zBPC0E7cXoV9e1j4up01YXeZfugxA3gZUn+rIuwdXct3BFUTYlPaVz/SqGe0KoGu
 DLOjM2Xh7P/HXZiYZeAgBh+TOXlCoKJ8BtKgmiRfjMC+f6OpfCSvtYLkrvZBgYIG
 Tm5DBWMp3J4FPM8apGUrMhAfuBrqBOgV5ah19J5Rs1i01GTDf/pyDOG8awvi20AE
 V3fHKRlLa6Jmlc2iAc++cMSCHeIoMklYjEeCAWnRelnLwOtuA7iwY3smsgBLQMJf
 1c7wTwl6t9WAIWzvLaKNn3b76eDYOJ9eyVJXNseSkuscHFUzmentT/WmxM9+ETxK
 DsjmHHsfGOjAadH1NULGAsIeujYxPIMJBJDv91/iKdZoWjMJRDorNjvhNb7ERXey
 5Da4Gyvi3pzOjJq1pqhBaTnYyC2CWqcIcy9+2uvVFlbnhrHGudPQfF1aAQ9vwK6l
 wPbJ6Tx9j1ZWrCZf5QL/1CcNxACb44IrmYNxqPwSBhrj7pLSnl/BB/hGocKUpNz9
 loCX4kRa1WNNyz6+Psm4N9I3P2gK65+kQtSGufVuxUxF6E890xE=
 =T/42
 -----END PGP SIGNATURE-----

Merge tag 'debian/11.8.3-1' into buster-fasttrack

gitlab Debian release 11.8.3-1
2019-03-22 15:24:05 +05:30
Pirate Praveen
ccaddb6da4 Update changelog 2019-03-17 18:55:31 +05:30
Pirate Praveen
bd86ee062b Merge tag 'debian/11.8.2-3' into buster-fasttrack 2019-03-17 18:51:44 +05:30
Pirate Praveen
d3cdd71e05 Update changelog 2019-03-16 15:06:04 +05:30
52196 changed files with 1489685 additions and 4593895 deletions

View file

@ -1,15 +1,16 @@
#
# This list of browsers is a conservative definition, based on
# This list of browsers is a conservative first definition, based on
# https://docs.gitlab.com/ee/install/requirements.html#supported-web-browsers
# with the following reasoning:
#
# - We should support the latest ESR of Firefox: 91, because it used quite a lot.
# - We use Edge/Chrome >= 92 because they are about as old as the Firefox ESR
# - Safari 14.1 because it is the current minor version of the previous major version
# - Edge: Pick the last two major version before the Chrome switch
# - Rest: We should support the latest ESR of Firefox: 68, because it used quite a lot.
# For the rest, pick browser versions that have a similar age to Firefox 68.
#
# See also this epic: https://gitlab.com/groups/gitlab-org/-/epics/3957
# See also this follow-up epic:
# https://gitlab.com/groups/gitlab-org/-/epics/3957
#
chrome >= 92
edge >= 92
firefox >= 91
safari >= 14.1
chrome >= 73
edge >= 17
firefox >= 68
safari >= 12

View file

@ -1,6 +1,5 @@
---
version: "2"
plugins:
engines:
bundler-audit:
enabled: true
duplication:
@ -9,22 +8,34 @@ plugins:
languages:
- ruby
- javascript
rubocop:
enabled: false
exclude_patterns:
- "{ee/,jh/,}config/"
- "{ee/,jh/,}db/"
- "**/log/"
- "**/node_modules/"
- "**/spec/"
- "**/tmp/"
- "**/vendor/"
ratings:
paths:
- Gemfile.lock
- "**.erb"
- "**.haml"
- "**.rb"
- "**.rhtml"
- "**.slim"
- "**.inc"
- "**.js"
- "**.jsx"
- "**.module"
exclude_paths:
- config/
- db/
- features/
- node_modules/
- spec/
- vendor/
- .yarn-cache/
- backups/
- tmp/
- builds/
- coverage/
- file_hooks/
- plugins/
- public/
- shared/
- webpack-report/
- log/
- backups/
- coverage-javascript/
- plugins/
- file_hooks/

20
.csscomb.json Normal file
View file

@ -0,0 +1,20 @@
{
"exclude": [
"app/assets/stylesheets/framework/tw_bootstrap_variables.scss",
"app/assets/stylesheets/framework/fonts.scss"
],
"always-semicolon": true,
"color-case": "lower",
"block-indent": " ",
"color-shorthand": false,
"element-case": "lower",
"space-before-colon": "",
"space-after-colon": " ",
"space-before-combinator": " ",
"space-after-combinator": " ",
"space-between-declarations": "\n",
"space-before-opening-brace": " ",
"space-after-opening-brace": "\n",
"space-before-closing-brace": "\n",
"unitless-zero": true
}

View file

@ -1,11 +1,11 @@
# `build_from_dir` can't find Dockerfile when `.dockerignore` is "*"
# See https://github.com/swipely/docker-api/issues/484
# Ignore all folders except the following files we need to build the QA image:
# Ignore all folders except qa/, config/initializers and the root of lib/ since
# the files we need to build the QA image are in these folders.
# Following are the files we need:
# - ./config/initializers/0_inject_enterprise_edition_module.rb
# - ./config/feature_flags
# - ./ee/config/feature_flags
# - ./ee/app/models/license.rb
# - ./lib/gitlab_edition.rb
# - ./lib/gitlab.rb
# - ./lib/gitlab/utils.rb
# - ./qa/
# - ./INSTALLATION_TYPE
@ -28,9 +28,7 @@
/docker/
/ee/bin/
/ee/changelogs/
/ee/config/events/
/ee/config/metrics/
/ee/config/routes/
/ee/config/
/ee/db/
/ee/fixtures/
/ee/lib/
@ -49,6 +47,7 @@
/lib/registry/
/lib/policy/
/lib/feature/
/lib/flowdock/
/lib/generators/
/lib/gitaly/
/lib/api/

View file

@ -16,19 +16,3 @@ charset = utf-8
[*.{md,markdown,js.snap}]
trim_trailing_whitespace = false
[doc/**/*.md]
trim_trailing_whitespace = true
[*.rb]
max_line_length = 120
# Don't apply editorconfig rules to vendor/ resources
[vendor/**]
charset = unset
end_of_line = unset
indent_size = unset
indent_style = unset
insert_final_newline = unset
trim_trailing_whitespace = unset
max_line_length = unset

View file

@ -2,12 +2,8 @@
/builds/
/coverage/
/coverage-frontend/
/coverage-javascript/
/node_modules/
/public/
/tmp/
/vendor/
/sitespeed-result/
/fixtures/**/*.graphql
# Storybook build artifacts
/storybook/public
spec/fixtures/**/*.graphql

View file

@ -3,8 +3,7 @@ extends:
- plugin:@gitlab/i18n
- plugin:no-jquery/slim
- plugin:no-jquery/deprecated-3.4
- plugin:no-unsanitized/DOM
- ./tooling/eslint-config/conditionally_ignore.js
- ./tooling/eslint-config/conditionally_ignore_ee.js
globals:
__webpack_public_path__: true
gl: false
@ -27,17 +26,16 @@ rules:
- _links
import/no-unresolved:
- error
- ignore:
# In FOSS, these import paths are rewritten using
# NormalModuleReplacementPlugin, which import/no-unresolved doesn't
# consider. See
# https://gitlab.com/gitlab-org/gitlab/-/merge_requests/89831.
- '^(ee|jh)_component/'
# Disabled for now, to make the airbnb-base 12.1.0 -> 13.1.0 update smoother
no-else-return:
- error
- allowElseIf: true
lines-between-class-members: off
# Disabled for now, to make the plugin-vue 4.5 -> 5.0 update smoother
vue/no-confusing-v-for-v-if: error
vue/no-use-v-if-with-v-for: off
vue/no-v-html: error
vue/use-v-on-exact: off
# all offenses of no-jquery/no-animate-toggle are false positives ( $toast.show() )
no-jquery/no-animate-toggle: off
no-jquery/no-event-shorthand: off
@ -45,15 +43,6 @@ rules:
promise/always-return: off
promise/no-callback-in-promise: off
'@gitlab/no-global-event-off': error
'@gitlab/vue-no-new-non-primitive-in-template':
- error
- allowNames:
- 'class(es)?$'
- '^style$'
- '^to$'
- '^$'
- '^variables$'
- 'attrs?$'
no-param-reassign:
- error
- props: true
@ -75,17 +64,15 @@ rules:
- sibling
- index
pathGroups:
- pattern: '@sentry/browser'
group: external
- pattern: ~/**
group: internal
- pattern: emojis/**
group: internal
- pattern: '{ee_,jh_,}empty_states/**'
- pattern: '{ee_,}empty_states/**'
group: internal
- pattern: '{ee_,jh_,}icons/**'
- pattern: '{ee_,}icons/**'
group: internal
- pattern: '{ee_,jh_,}images/**'
- pattern: '{ee_,}images/**'
group: internal
- pattern: vendor/**
group: internal
@ -93,77 +80,31 @@ rules:
group: internal
- pattern: '{ee_,}spec/**'
group: internal
- pattern: '{ee_,jh_,}jest/**'
- pattern: '{ee_,}jest/**'
group: internal
- pattern: '{ee_,jh_,any_}else_ce/**'
- pattern: ee_else_ce/**
group: internal
- pattern: ee/**
group: internal
- pattern: '{ee_,jh_,}component/**'
group: internal
- pattern: jh_else_ee/**
group: internal
- pattern: jh/**
- pattern: ee_component/**
group: internal
- pattern: '{test_,}helpers/**'
group: internal
- pattern: test_fixtures/**
group: internal
alphabetize:
order: ignore
'no-restricted-syntax':
- error
- selector: ImportSpecifier[imported.name='GlSkeletonLoading']
message: 'Migrate to GlSkeletonLoader, or import GlDeprecatedSkeletonLoading.'
- selector: ImportSpecifier[imported.name='GlSafeHtmlDirective']
message: 'Use directive at ~/vue_shared/directives/safe_html.js instead.'
no-restricted-imports:
- error
- paths:
- name: mousetrap
message: 'Import { Mousetrap } from ~/lib/mousetrap instead.'
# See https://gitlab.com/gitlab-org/gitlab/-/issues/360551
vue/multi-word-component-names: off
unicorn/prefer-dom-node-dataset:
- error
no-unsanitized/method:
- error
- escape:
methods: 'sanitize'
no-unsanitized/property:
- error
- escape:
methods: 'sanitize'
order: asc
overrides:
- files:
- '{,ee/,jh/}spec/frontend*/**/*'
- '**/spec/**/*'
rules:
'@gitlab/require-i18n-strings': off
'@gitlab/no-runtime-template-compiler': off
'require-await': error
'import/no-dynamic-require': off
'no-import-assign': off
'no-restricted-syntax':
- error
- selector: CallExpression[callee.object.name=/(wrapper|vm)/][callee.property.name="setData"]
message: 'Avoid using "setData" on VTU wrapper'
- selector: MemberExpression[object.type!='ThisExpression'][property.type='Identifier'][property.name='$nextTick']
message: 'Using $nextTick from a component instance is discouraged. Import nextTick directly from the Vue package.'
- selector: Identifier[name='setImmediate']
message: 'Prefer explicit waitForPromises (or equivalent), or jest.runAllTimers (or equivalent) to vague setImmediate calls.'
- selector: ImportSpecifier[imported.name='GlSkeletonLoading']
message: 'Migrate to GlSkeletonLoader, or import GlDeprecatedSkeletonLoading.'
- selector: CallExpression[arguments.length=1][arguments.0.type='Literal'] CallExpression[callee.property.name='toBe'] CallExpression[callee.property.name='attributes'][arguments.length=1][arguments.0.value='disabled']
message: Avoid asserting disabled attribute exact value, because Vue.js 2 and Vue.js 3 renders it differently. Use toBeDefined / toBeUndefined instead
no-unsanitized/method: off
no-unsanitized/property: off
- files:
- 'config/**/*'
- 'scripts/**/*'
- '*.config.js'
- '*.config.*.js'
- 'jest_resolver.js'
- storybook/config/*.js
rules:
'@gitlab/require-i18n-strings': off
import/no-extraneous-dependencies: off
@ -171,35 +112,3 @@ overrides:
import/no-nodejs-modules: off
filenames/match-regex: off
no-console: off
- files:
- '*.stories.js'
rules:
filenames/match-regex: off
'@gitlab/require-i18n-strings': off
- files:
- '*.graphql'
plugins:
- '@graphql-eslint'
parserOptions:
parser: '@graphql-eslint/eslint-plugin'
operations: '{,ee/,jh/}app/**/*.graphql'
schema: './tmp/tests/graphql/gitlab_schema_apollo.graphql'
rules:
filenames/match-regex: off
spaced-comment: off
# TODO: We need a way to include this rule + support ee_else_ce fragments
#'@graphql-eslint/unique-fragment-name': error
# TODO: Uncomment these rules when then `schema` is available
#'@graphql-eslint/fragments-on-composite-type': error
#'@graphql-eslint/known-argument-names': error
#'@graphql-eslint/known-type-names': error
'@graphql-eslint/no-anonymous-operations': error
'@graphql-eslint/unique-operation-name': error
'@graphql-eslint/require-id-when-available': error
'@graphql-eslint/no-unused-variables': error
'@graphql-eslint/no-unused-fragments': error
'@graphql-eslint/no-duplicate-fields': error
- files:
- '{,ee/}spec/contracts/consumer/**/*'
rules:
'@gitlab/require-i18n-strings': off

View file

@ -1,130 +0,0 @@
# This file contains revisions to be ignored by git blame.
# These revisions are expected to be formatting-only changes.
#
# Calling `git blame --ignore-revs-file .git-blame-ignore-revs` will
# tell git blame to ignore changes made by these revisions when assigning
# assigning blame, as if the change never happened.
#
# You can enable this as a default for your local repository by running
# `git config blame.ignoreRevsFile .git-blame-ignore-revs`
# This will probably be automatically picked by your IDE
# (VSCode+GitLens and JetBrains products are confirmed to this)
#
# Important: if you are switching to the branch without this file,
# `git blame` will fail with an error
#
# Guidelines:
# - Only large automated refactorings are expected to be included in this file.
# Do not add new revision just because it feels unimportant
# - When adding sinle revision use inline comment to link relevant issue/MR
# Example:
## d4a8b7307acc2dc8a8833ccfa65426ad28b3ffc9 # https://gitlab.com/gitlab-org/frontend/rfcs/-/issues/60
# - When adding multiple revisions precede each addition (this could be multiple revisions) with a link to
# line with word START and link to relevant issue/MR/epic and conclude with line END and link to the
# same issue/MR/epic
# Example:
# # START https://gitlab.com/gitlab-org/issues/12345
# 6f0bd2d8a1e6cd2e794cd39976e9756e0c85ac66
# d53974df11dbc22cbea9dc7dcbc9896c25979a27
# ... <rest of the list>
# # END https://gitlab.com/gitlab-org/issues/12345
# - Please append new lines to the end of the file, no matter of real chronological
# order of revisions
# - Since this is using hashes for reformatting it might be a good idea to update
# this file in separate MR when relevant changes already landed in master. By
# utilizing this manner you will be safe from random rebase/squash issues
# - Only put full 40-character hashes on this list
# START https://gitlab.com/gitlab-org/frontend/rfcs/-/issues/60
f2d28d7ab8525944fda634241a780006594fbe1a
94cfbb0ce38e893edda33ebc069bfa616a08a961
b69f448f0685ad96edc474f75a17e0278a6d6011
52907ac20c3af337544bdf18023730b9ada4b157
d4a8b7307acc2dc8a8833ccfa65426ad28b3ffc9
468cb9f0a4b88bf686f3e78250834f7c9d31ff76
a536349d1d219f0b79a7a711d37dd1c705e49128
6f0bd2d8a1e6cd2e794cd39976e9756e0c85ac66
d53974df11dbc22cbea9dc7dcbc9896c25979a27
818537524d13469cbc7ac5cb89263378b4cddca4
bcbbcb2e708868099301ad5039badfba2128d47b
a4c662da544b38b7e593eb79f24b24c5cb2f205e
9aa1f6207a91a76940b34c921ce89894fcd74a06
66da09846a17435f332296f73af44919ff2cfb52
216f795bab0e8fbf6023f22f6e54cc07514a04ec
e820c22892d207e138bdff717100e5240f8ffd94
2f8dbd483242575f9ceca0a2947c9b21e5ab59a0
e7d50054818ada29751539f548ef72f46deca8bb
00827a74cf3bfef985ed6046fb2d42f29cbb19ac
333bad893e98068053c888f6b020632f1c6f472e
85af3689eea96b4d9131d80d8c5c8936de520074
325fb305ea395a7f44ae1eea0a3e77e46e10c2b6
e37a6d7aa61039734025474ce901f2907283e239
dff561fa8c50e9b96aec9800b6b88ad6c7a2777b
19b0ba7265cfb154505f74b6856e73662829af2e
7c1fa749efcd59e81b565d6803285f6bd4bcefaf
5c23cb94c5d1aed2a4b02b7c1f3e5a53a0aa4760
c35cc92c80969e7c87bbcda7db6cbd04f6719589
280a79c0ec4c1383e49480f3028f5b2025a2a76b
e94556e9f9a145374bf26feb5e1823dae8a4004d
b6a8d9baf700dbb3f780b27d9a9820c9cb7a346c
9180eaee4d58a9e91c5f960148290b5271ba870c
0fdc1fc0380056836dff7aba9be3b1e4b531daea
157e117fcb530436561e3fb8faba6f751dc19e91
7cfe360c9e5460a595dfe729e81cf404c1106638
e3aca8c8f8488c55a199fc28595709b393f5040b
3b1593f2d53b735299381ad0878959cbc2fc9923
39bb37cc0d18f620006d85dfdff7b9a54077708e
cdc1a4a8eec43e6a3df05403af8d05ab6ea7a213
87ad67fef574cd102887f3dde98917f3b2bbcab8
99bff4450248457ba877dec0388241625fb0144b
d1b6d05c08e0730463084acd1a387cd9d6acea8b
557c22a8242d1d7ccf2228b9b3156e2aa0dd05aa
7f4e951ce8073b50a245ebe216a8961c88846cfa
7dae714f23f423ff362d73e0d16da7b3a6cd721f
cfb368284545a4bd1e759cfe9e3e3bde54a1ec6f
aa653d5a380d88493050b22d84df36ae6df2cddd
96ed4677c602e8f9c83b28fbc0d802aa26527ab8
72c11eb5a15735dc52dcd893e9112a10444d46e4
b48e14b89b94a1a87affabd09bc603a67fc6bb01
d46581c1fbcef34cfdd85c6c542fb4ac1b974861
ef02363c9cd41a9ce41443661efed1c0399c5551
075a78b319466aff9e94149c41c286544af91782
9f4b4de2df17268732ae198d5f48c9b99d071a35
0a8f575e365804239d29b45562ca6594b9da59e9
c04bd24738b1775b963bba3f78b48007fccce37e
1173c801ea53c9d814fdf27d878f73a1702eb4e9
a2f5e7395004c255ecaadef30d7a6b5bf453d372
80f1ea7e3f11063a4f15bdd4a2e4a1ca7f770d87
0e6e345f3b4dcb7b51403bcd096e6d3d294743f4
06ee932e0844fa4cc91c15d5ca581de262d7bedd
b3ece842f7c05230f77055ad11e3c4a07c34e1e9
5ebea3a48831351169e0a312e9d6985b31c9975b
02fad0bc640f5f91c748d692c01d6221c9b03b6e
16d4df3c7130b5a0995fdc685b272bef65ff84d5
281cc7306ab92d2e053d0bb2d79e4f3646b980f6
1877bf550016eac9ecac53ec498ec83bdd24339c
7e9741c59d1e3612017925a7b7cf0946bbdd6eca
b282f7dda6d7e93fcb0f000db8aa6634ac8d1b88
81e82875704ffb35842534433216e797c41f89c3
4914a729d17efbe250ac2cab2153f72caef3a7b7
792e349390327fa11721e2f744cafec3b05f51f4
8869ce0866823b229a863e435aa108c5d4fcf448
a223014afe14686a4e18a826fd0bac9bdaaf969b
482d756cf69e3f0dd5997ea0e58d35c0eb694e35
4700ac1d1da533cfefd50bd640db77a12c458fda
21fa9ca4832cfb57f791ff057e7c5987349aa964
8b24c8d64d9328e0884725a2075a4a21faa76842
86ce5406c3b60757f40d4c434b5ce7dfc602a643
da4eea76b3cc1d68d4bfd2705bb86e904d1b54bc
8ac8a1f21a21840b53175e9f4a423b9ffa083f71
ed189d0e9925eb08f3eb444176fad2614a3a4f83
6e183d5016afc50e60892c7f1cf79035619c2deb
9b1d8b4c2897792be067e33442ebf3ce0961a5d0
57da632154bfc193224d5a290b9c2b6cbd7fa0ad
1e3190b0049ba1b502918dc018681808b9203803
0e334037bf0f93ff6f7bc922c48fa97556f39808
07f5bc94bd983e77361c9a5020f8f229da3a465a
888002a62696ba66d8eb49f1dfe83a5a49bdf421
c152d51445d9d9dd7c2c328ca8c407fa5438d16b
26b68c70df73289210aa600fa3c1fe45f05afee4
# END https://gitlab.com/gitlab-org/frontend/rfcs/-/issues/60

1
.gitattributes vendored
View file

@ -1,4 +1,5 @@
VERSION merge=ours
Dangerfile gitlab-language=ruby
*.pdf filter=lfs diff=lfs merge=lfs -text
*.rb diff=ruby
workhorse/testdata/*.pdf -filter -diff -merge

20
.gitignore vendored
View file

@ -3,7 +3,6 @@
*.mo
*.edit.po
*.rej
.dir-locals.el
.DS_Store
.bundle
.chef
@ -13,7 +12,6 @@
eslint-report.html
/.gitlab_shell_secret
.idea
.nova
/.vscode/*
/.rbenv-version
.rbx/
@ -41,14 +39,17 @@ eslint-report.html
/config/initializers/smtp_settings.rb
/config/initializers/relative_url.rb
/config/resque.yml
/config/redis.*.yml
/config/redis.yml
/config/redis.cache.yml
/config/redis.queues.yml
/config/redis.shared_state.yml
/config/unicorn.rb
/config/puma.rb
/config/puma_actioncable.rb
/config/secrets.yml
/config/sidekiq.yml
/config/registry.key
/coverage/*
/coverage-javascript/
/db/*.sqlite3
/db/*.sqlite3-journal
/db/data.yml
@ -69,20 +70,16 @@ eslint-report.html
/rails_best_practices_output.html
/tags
/vendor/bundle/*
/vendor/package_metadata_db/
/vendor/gitaly-ruby
/builds*
/.gitlab_workhorse_secret
/.gitlab_pages_secret
/.gitlab_kas_secret
/.gitlab_suggested_reviewers_secret
/webpack-report/
/crystalball/
/test_results/
/deprecations/
/knapsack/
/query_recorder/
/rspec_flaky/
/rspec/
/locale/**/LC_MESSAGES
/locale/**/*.time_stamp
/.rspec
@ -93,11 +90,16 @@ package-lock.json
/coverage-frontend/
jsdoc/
**/tmp/rubocop_cache/**
.overcommit.yml
.overcommit.yml.backup
.projections.json
/qa/.rakeTasks
webpack-dev-server.json
/.nvimrc
.solargraph.yml
apollo.config.js
/tmp/matching_foss_tests.txt
/tmp/matching_tests.txt
ee/changelogs/unreleased-ee
/sitespeed-result
tags.lock

View file

@ -1,24 +1,23 @@
stages:
- sync
- preflight
- prepare
- build-images
- fixtures
- lint
- test
- post-test
- review-prepare
- review
- dast
- qa
- post-qa
- pages
- notify
- release-environments
# always use `gitlab-org` runners, however
# in cases where jobs require Docker-in-Docker, the job
# definition must be extended with `.use-docker-in-docker`
default:
image: $DEFAULT_CI_IMAGE
image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.7.2.patched-golang-1.14-git-2.31-lfs-2.9-chrome-89-node-14.15-yarn-1.22-postgresql-11-graphicsmagick-1.3.36"
tags:
- gitlab-org
# All jobs are interruptible by default
@ -26,27 +25,10 @@ default:
# Default job timeout set to 90m https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/10520
timeout: 90m
.default-ruby-variables: &default-ruby-variables
RUBY_VERSION: "3.0"
OMNIBUS_GITLAB_RUBY3_BUILD: "true"
OMNIBUS_GITLAB_CACHE_EDITION: "GITLAB_RUBY3"
.backcompat-ruby-variables: &backcompat-ruby-variables
RUBY_VERSION: "2.7"
OMNIBUS_GITLAB_RUBY2_BUILD: "true"
OMNIBUS_GITLAB_CACHE_EDITION: "GITLAB_RUBY2"
.default-branch-pipeline-failure-variables: &default-branch-pipeline-failure-variables
CREATE_ISSUES_FOR_FAILING_TESTS: "true"
workflow:
name: '$PIPELINE_NAME'
rules:
# If `$FORCE_GITLAB_CI` is set, create a pipeline.
- if: '$FORCE_GITLAB_CI'
variables:
<<: *default-ruby-variables
PIPELINE_NAME: 'Ruby $RUBY_VERSION forced pipeline'
# As part of the process of creating RCs automatically, we update stable
# branches with the changes of the most recent production deployment. The
# merge requests used for this merge a branch release-tools/X into a stable
@ -54,153 +36,74 @@ workflow:
# they serve no purpose and will run anyway when the changes are merged.
- if: '$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^release-tools\/\d+\.\d+\.\d+-rc\d+$/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME =~ /^[\d-]+-stable(-ee)?$/ && $CI_PROJECT_PATH == "gitlab-org/gitlab"'
when: never
# For merge requests running exclusively in Ruby 2.7
- if: '$CI_MERGE_REQUEST_LABELS =~ /pipeline:run-in-ruby2/'
variables:
<<: *backcompat-ruby-variables
PIPELINE_NAME: 'Ruby $RUBY_VERSION $CI_MERGE_REQUEST_EVENT_TYPE MR pipeline'
NO_SOURCEMAPS: 'true'
- if: '$CI_MERGE_REQUEST_LABELS =~ /Community contribution/'
variables:
<<: *default-ruby-variables
GITLAB_DEPENDENCY_PROXY_ADDRESS: ""
PIPELINE_NAME: 'Ruby $RUBY_VERSION $CI_MERGE_REQUEST_EVENT_TYPE MR pipeline (community contribution)'
NO_SOURCEMAPS: 'true'
# For (detached) merge request pipelines.
# For merge requests, create a pipeline.
- if: '$CI_MERGE_REQUEST_IID'
variables:
<<: *default-ruby-variables
PIPELINE_NAME: 'Ruby $RUBY_VERSION $CI_MERGE_REQUEST_EVENT_TYPE MR pipeline'
NO_SOURCEMAPS: 'true'
# For the scheduled pipelines, we set specific variables.
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_PIPELINE_SOURCE == "schedule"'
variables:
<<: *default-ruby-variables
<<: *default-branch-pipeline-failure-variables
CRYSTALBALL: "true"
PIPELINE_NAME: 'Scheduled Ruby $RUBY_VERSION $CI_COMMIT_BRANCH branch pipeline'
# Run pipelines for ruby2 branch
- if: '$CI_COMMIT_BRANCH == "ruby2" && $CI_PIPELINE_SOURCE == "schedule"'
variables:
<<: *backcompat-ruby-variables
PIPELINE_NAME: 'Scheduled Ruby $RUBY_VERSION $CI_COMMIT_BRANCH branch pipeline'
# This work around https://gitlab.com/gitlab-org/gitlab/-/issues/332411 whichs prevents usage of dependency proxy
# when pipeline is triggered by a project access token.
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $GITLAB_USER_LOGIN =~ /project_\d+_bot\d*/'
variables:
<<: *default-ruby-variables
<<: *default-branch-pipeline-failure-variables
GITLAB_DEPENDENCY_PROXY_ADDRESS: ""
PIPELINE_NAME: 'Ruby $RUBY_VERSION $CI_COMMIT_BRANCH branch pipeline (triggered by a project token)'
# For `$CI_DEFAULT_BRANCH` branch, create a pipeline (this includes on schedules, pushes, merges, etc.).
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
variables:
<<: *default-ruby-variables
<<: *default-branch-pipeline-failure-variables
PIPELINE_NAME: 'Ruby $RUBY_VERSION $CI_COMMIT_BRANCH branch pipeline'
# For tags, create a pipeline.
- if: '$CI_COMMIT_TAG'
variables:
<<: *default-ruby-variables
PIPELINE_NAME: 'Ruby $RUBY_VERSION $CI_COMMIT_TAG tag pipeline'
# If `$GITLAB_INTERNAL` isn't set, don't create a pipeline.
- if: '$GITLAB_INTERNAL == null'
when: never
# For stable, auto-deploy, and security branches, create a pipeline.
- if: '$CI_COMMIT_BRANCH =~ /^[\d-]+-stable(-ee)?$/'
variables:
<<: *default-ruby-variables
PIPELINE_NAME: 'Ruby $RUBY_VERSION $CI_COMMIT_BRANCH branch pipeline'
- if: '$CI_COMMIT_BRANCH =~ /^\d+-\d+-auto-deploy-\d+$/'
variables:
<<: *default-ruby-variables
PIPELINE_NAME: 'Ruby $RUBY_VERSION $CI_COMMIT_BRANCH branch pipeline'
- if: '$CI_COMMIT_BRANCH =~ /^security\//'
variables:
<<: *default-ruby-variables
PIPELINE_NAME: 'Ruby $RUBY_VERSION $CI_COMMIT_BRANCH branch pipeline'
variables:
PG_VERSION: "13"
DEFAULT_CI_IMAGE: "${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images/debian-${DEBIAN_VERSION}-ruby-${RUBY_VERSION}.patched-golang-${GO_VERSION}-rust-${RUST_VERSION}-node-16.14-postgresql-${PG_VERSION}:rubygems-${RUBYGEMS_VERSION}-git-2.36-lfs-2.9-chrome-${CHROME_VERSION}-yarn-1.22-graphicsmagick-1.3.36"
# We set $GITLAB_DEPENDENCY_PROXY to another variable (since it's set at the group level and has higher precedence than .gitlab-ci.yml)
# so that we can override $GITLAB_DEPENDENCY_PROXY_ADDRESS in workflow rules.
GITLAB_DEPENDENCY_PROXY_ADDRESS: "${GITLAB_DEPENDENCY_PROXY}"
RAILS_ENV: "test"
NODE_ENV: "test"
BUNDLE_WITHOUT: "production:development"
BUNDLE_INSTALL_FLAGS: "--jobs=$(nproc) --retry=3"
BUNDLE_FROZEN: "true"
BUNDLE_INSTALL_FLAGS: "--jobs=$(nproc) --retry=3 --quiet"
# we override the max_old_space_size to prevent OOM errors
NODE_OPTIONS: --max_old_space_size=4096
NODE_OPTIONS: --max_old_space_size=3584
GIT_DEPTH: "20"
# 'GIT_STRATEGY: clone' optimizes the pack-objects cache hit ratio
GIT_STRATEGY: "clone"
GIT_SUBMODULE_STRATEGY: "none"
GET_SOURCES_ATTEMPTS: "3"
DEBIAN_VERSION: "bullseye"
UBI_VERSION: "8.6"
CHROME_VERSION: "109"
DOCKER_VERSION: "23.0.1"
RUBY_VERSION: "2.7"
RUBYGEMS_VERSION: "3.4"
GO_VERSION: "1.19"
RUST_VERSION: "1.65"
FLAKY_RSPEC_SUITE_REPORT_PATH: rspec/flaky/report-suite.json
FRONTEND_FIXTURES_MAPPING_PATH: crystalball/frontend_fixtures_mapping.json
GITLAB_WORKHORSE_FOLDER: "gitlab-workhorse"
JUNIT_RESULT_FILE: rspec/junit_rspec.xml
JUNIT_RETRY_FILE: rspec/junit_rspec-retry.xml
KNAPSACK_RSPEC_SUITE_REPORT_PATH: knapsack/report-master.json
RSPEC_CHANGED_FILES_PATH: rspec/changed_files.txt
RSPEC_FOSS_IMPACT_PIPELINE_TEMPLATE_YML: .gitlab/ci/rails/rspec-foss-impact.gitlab-ci.yml.erb
RSPEC_PREDICTIVE_PIPELINE_TEMPLATE_YML: .gitlab/ci/rails/rspec-predictive.gitlab-ci.yml.erb
RSPEC_LAST_RUN_RESULTS_FILE: rspec/rspec_last_run_results.txt
RSPEC_MATCHING_JS_FILES_PATH: rspec/js_matching_files.txt
RSPEC_VIEWS_INCLUDING_PARTIALS_PATH: rspec/views_including_partials.txt
RSPEC_MATCHING_TESTS_PATH: rspec/matching_tests.txt
RSPEC_MATCHING_TESTS_FOSS_PATH: rspec/matching_tests-foss.txt
RSPEC_MATCHING_TESTS_EE_PATH: rspec/matching_tests-ee.txt
RSPEC_PACKED_TESTS_MAPPING_PATH: crystalball/packed-mapping.json
RSPEC_PROFILING_FOLDER_PATH: rspec/profiling
FLAKY_RSPEC_SUITE_REPORT_PATH: rspec_flaky/report-suite.json
RSPEC_TESTS_MAPPING_PATH: crystalball/mapping.json
RSPEC_FAST_QUARANTINE_LOCAL_PATH: rspec/fast_quarantine-gitlab.txt
TMP_TEST_FOLDER: "${CI_PROJECT_DIR}/tmp/tests"
TMP_TEST_GITLAB_WORKHORSE_PATH: "${TMP_TEST_FOLDER}/${GITLAB_WORKHORSE_FOLDER}"
RSPEC_PACKED_TESTS_MAPPING_PATH: crystalball/packed-mapping.json
ES_JAVA_OPTS: "-Xms256m -Xmx256m"
ELASTIC_URL: "http://elastic:changeme@elasticsearch:9200"
BUNDLER_CHECKSUM_VERIFICATION_OPT_IN: "1"
DOCKER_VERSION: "20.10.1"
CACHE_CLASSES: "true"
CHECK_PRECOMPILED_ASSETS: "true"
FF_USE_FASTZIP: "true"
RETRY_FAILED_TESTS_IN_NEW_PROCESS: "true"
# Run with decomposed databases by default
DECOMPOSED_DB: "true"
DOCS_REVIEW_APPS_DOMAIN: "docs.gitlab-review.app"
DOCS_REVIEW_APPS_DOMAIN: "178.62.207.141.nip.io"
DOCS_GITLAB_REPO_SUFFIX: "ee"
REVIEW_APPS_IMAGE: "${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images/ruby-3.0:gcloud-383-kubectl-1.23-helm-3.5"
REVIEW_APPS_DOMAIN: "gitlab-review.app"
REVIEW_APPS_GCP_PROJECT: "gitlab-review-apps"
REVIEW_APPS_GCP_REGION: "us-central1"
CACHE_ASSETS_AS_PACKAGE: "true"
BUILD_ASSETS_IMAGE: "true" # Set it to "false" to disable assets image building, used in `build-assets-image`
RSPEC_FAIL_FAST_ENABLED: "true" # Set it to "false" to disable RSpec fail-fast
SIMPLECOV: "true"
REGISTRY_HOST: "registry.gitlab.com"
REGISTRY_GROUP: "gitlab-org"
# Disable useless network connections when installing some NPM packages.
# See https://gitlab.com/gitlab-com/gl-security/engineering-and-research/inventory/-/issues/827#note_1203181407
DISABLE_OPENCOLLECTIVE: "true"
# This is set at the gitlab-org level, but we set it here for forks
DANGER_DO_NOT_POST_INVALID_DANGERFILE_ERROR: "1"
# Preparing custom clone path to reduce space used by all random forks
# on GitLab.com's Shared Runners. Our main forks - especially the security
# ones - will have this variable overwritten in the project settings, so that
# a security-related code or code using our protected variables will be never
# stored on the same path as the community forks.
# Part of the solution for the `no space left on device` problem described at
# https://gitlab.com/gitlab-org/gitlab/issues/197876.
#
# For this purpose the https://gitlab.com/gitlab-org-forks group was created
# to host a placeholder for the `/builds/gitlab-org-forks` path and ensure
# that no legitimate project will ever use it and - by mistake - execute its
# job on a shared working directory. It also requires proper configuration of
# the Runner that executes the job (which was prepared for our shared runners
# by https://ops.gitlab.net/gitlab-cookbooks/chef-repo/-/merge_requests/3977).
#
# Because of all of that PLEASE DO NOT CHANGE THE PATH.
#
# For more details and reasoning that brought this change please check
# https://gitlab.com/gitlab-org/gitlab/-/merge_requests/24887
GIT_CLONE_PATH: "/builds/gitlab-org-forks/${CI_PROJECT_NAME}"
include:
- local: .gitlab/ci/*.gitlab-ci.yml
- remote: 'https://gitlab.com/gitlab-org/frontend/untamper-my-lockfile/-/raw/main/templates/merge_request_pipelines.yml'

File diff suppressed because it is too large Load diff

View file

@ -1 +0,0 @@
# This empty file is used for agent-based integration with Kubernetes

View file

@ -11,8 +11,6 @@ categories:
security: Security
performance: Performance
other: Other
include_groups:
- gitlab-org/gitlab-core-team/community-members
template: |
{% if categories %}
{% each categories %}
@ -20,7 +18,7 @@ template: |
{% each entries %}
- [{{ title }}]({{ commit.reference }})\
{% if author.credit %} by {{ author.reference }}{% end %}\
{% if author.contributor %} by {{ author.reference }}{% end %}\
{% if commit.trailers.MR %}\
([merge request]({{ commit.trailers.MR }}))\
{% else %}\
@ -38,8 +36,3 @@ template: |
{% else %}
No changes.
{% end %}
# The tag format for gitlab-org/gitlab is vX.Y.Z(-rcX)-ee. The -ee prefix would
# be treated as a pre-release identifier, which can result in the wrong tag
# being used as the starting point of a changelog commit range. The custom regex
# here is used to ensure we find the correct tag.
tag_regex: '^v(?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)-ee$'

View file

@ -1,11 +0,0 @@
# no-op pipeline template for skipping whole child pipeline execution
no-op:
image: ${GITLAB_DEPENDENCY_PROXY_ADDRESS}alpine:latest
stage: test
variables:
GIT_STRATEGY: none
script:
- echo "${SKIP_MESSAGE:-no-op run, nothing will be executed!}"
rules:
- when: always

View file

@ -1,106 +0,0 @@
.as-if-jh-sandbox-variables:
variables:
AS_IF_JH_BRANCH: "as-if-jh/${CI_COMMIT_REF_NAME}"
SANDBOX_REPOSITORY: "https://dummy:${AS_IF_JH_TOKEN}@gitlab.com/gitlab-org-sandbox/gitlab-jh-validation.git"
.shared-as-if-jh:
extends:
- .as-if-jh-sandbox-variables
variables:
GITLAB_JH_MIRROR_PROJECT: "33019816"
JH_FILES_TO_COMMIT: "jh package.json yarn.lock"
add-jh-files:
extends:
- .shared-as-if-jh
- .as-if-jh:rules:prepare-as-if-jh
image: ${GITLAB_DEPENDENCY_PROXY_ADDRESS}ruby:${RUBY_VERSION}
stage: prepare
before_script:
- source ./scripts/utils.sh
- source ./scripts/setup/as-if-jh.sh
- install_gitlab_gem
script:
- prepare_jh_branch
- download_jh_path ${JH_FILES_TO_COMMIT}
- echoinfo "Changes after downloading JiHu files:"
- git diff
- git status
artifacts:
expire_in: 2d
paths:
# This should match JH_FILES_TO_COMMIT
- jh/
- package.json
- yarn.lock
prepare-as-if-jh-branch:
extends:
- .shared-as-if-jh
- .as-if-jh:rules:prepare-as-if-jh
stage: prepare
needs:
- add-jh-files
variables:
# We can't apply --filter=tree:0 for runner to set up the repository,
# so instead we tell runner to not clone anything, and we set up the
# repository by ourselves.
GIT_STRATEGY: "none"
before_script:
- git clone --filter=tree:0 "${CI_REPOSITORY_URL}" gitlab
# We should checkout before moving/changing files
- cd gitlab
- git checkout -b "${AS_IF_JH_BRANCH}" "${CI_COMMIT_SHA}"
- cd ..
- mv ${JH_FILES_TO_COMMIT} gitlab/
script:
- cd gitlab
- git add ${JH_FILES_TO_COMMIT}
- git commit -m 'Add JH files' # TODO: Mark which SHA we add
- git push -f "${SANDBOX_REPOSITORY}" "${AS_IF_JH_BRANCH}"
sync-as-if-jh-branch:
extends:
- .as-if-jh-sandbox-variables
- .as-if-jh:rules:sync-as-if-jh
stage: prepare
needs: ["prepare-as-if-jh-branch"]
inherit:
variables:
# From .gitlab-ci.yml for the default Docker image and cache
- DEFAULT_CI_IMAGE
- REGISTRY_HOST
- REGISTRY_GROUP
- DEBIAN_VERSION
- RUBY_VERSION
- GO_VERSION
- RUST_VERSION
- PG_VERSION
- RUBYGEMS_VERSION
- CHROME_VERSION
- NODE_ENV
variables:
MERGE_FROM: "${CI_COMMIT_SHA}" # This is used in https://jihulab.com/gitlab-cn/gitlab/-/blob/e98bcb37aea4cfe1e78e1daef1b58b5f732cf289/jh/bin/build_packagejson where we run in https://gitlab.com/gitlab-org-sandbox/gitlab-jh-validation
trigger:
# What this runs can be found at:
# https://gitlab.com/gitlab-org-sandbox/gitlab-jh-validation/-/blob/as-if-jh-code-sync/jh/.gitlab-ci.yml
project: gitlab-org-sandbox/gitlab-jh-validation
branch: as-if-jh-code-sync
strategy: depend
start-as-if-jh:
extends:
- .as-if-jh:rules:start-as-if-jh
stage: prepare
needs:
- job: "prepare-as-if-jh-branch"
- job: "sync-as-if-jh-branch"
optional: true
inherit:
variables: false
variables:
FORCE_GITLAB_CI: "true" # TODO: Trigger a merge request pipeline
trigger:
project: gitlab-org-sandbox/gitlab-jh-validation
branch: as-if-jh/${CI_COMMIT_REF_NAME}
strategy: depend

View file

@ -1,80 +1,43 @@
.base-image-build:
extends: .use-kaniko
variables:
GIT_LFS_SKIP_SMUDGE: 1 # disable pulling objects from lfs
retry: 2
.base-image-build-buildx:
extends: .use-buildx
variables:
GIT_LFS_SKIP_SMUDGE: 1 # disable pulling objects from lfs
retry: 2
# This image is used by:
# - The `review-qa-*` jobs
# - The `e2e:package-and-test` child pipeline test stage jobs
# This image is used by the `review-qa-*` jobs. The image name is also passed to the downstream `omnibus-gitlab-mirror` pipeline
# triggered by `package-and-qa` so that it doesn't have to rebuild it a second time. The downstream `omnibus-gitlab-mirror` pipeline
# itself passes the image name to the `gitlab-qa-mirror` pipeline so that it can use it instead of inferring an end-to-end image
# from the GitLab image built by the downstream `omnibus-gitlab-mirror` pipeline.
# See https://docs.gitlab.com/ee/development/testing_guide/end_to_end/index.html#testing-code-in-merge-requests for more details.
build-qa-image:
extends:
- .base-image-build-buildx
- .use-kaniko
- .build-images:rules:build-qa-image
stage: build-images
needs: []
script:
- run_timed_command "scripts/build_qa_image"
build-qa-image as-if-foss:
extends:
- build-qa-image
- .as-if-foss
- .build-images:rules:build-qa-image-as-if-foss
# Prepares an image with GDK configured based on code in master. This saves some time in MRs because some installation
# and complilation will have already been performed.
build-qa-on-gdk-master-image:
extends:
- .base-image-build-buildx
- .build-images:rules:build-qa-on-gdk-master-image
tags:
- e2e
stage: build-images
needs: []
variables:
QA_GDK_IMAGE: "${CI_REGISTRY}/${CI_PROJECT_PATH}/gitlab-qa-gdk"
before_script:
- !reference [.use-buildx, before_script]
- sysctl -n -w fs.inotify.max_user_watches=524288
QA_IMAGE: "${CI_REGISTRY}/${CI_PROJECT_PATH}/gitlab-ee-qa:${CI_COMMIT_REF_SLUG}"
script:
- |
docker buildx build \
--cache-to=type=inline \
--cache-from ${QA_GDK_IMAGE}:master \
--platform=${ARCH:-amd64} \
--add-host gdk.test:127.0.0.1 \
--tag ${QA_GDK_IMAGE}:master \
--file="qa/gdk/Dockerfile" \
--push \
${CI_PROJECT_DIR}
# With .git/hooks/post-checkout in place, Git tries to pull LFS objects, but the image doesn't have Git LFS, and we actually don't care about it for this specific so we just remove the file.
# Without removing the file, the error is as follows: "This repository is configured for Git LFS but 'git-lfs' was not found on your path. If you no longer wish to use Git LFS, remove this hook by deleting .git/hooks/post-checkout."
- rm .git/hooks/post-checkout
# Use $CI_MERGE_REQUEST_SOURCE_BRANCH_SHA so that GitLab image built in omnibus-gitlab-mirror and QA image are in sync.
# This falls back to $CI_COMMIT_SHA (the default checked out commit) for the non-merged result pipelines.
# See https://docs.gitlab.com/ee/development/testing_guide/end_to_end/index.html#with-pipeline-for-merged-results.
- if [ -n "$CI_MERGE_REQUEST_SOURCE_BRANCH_SHA" ]; then
git checkout -f ${CI_MERGE_REQUEST_SOURCE_BRANCH_SHA};
fi
- /kaniko/executor --context=${CI_PROJECT_DIR} --dockerfile=${CI_PROJECT_DIR}/qa/Dockerfile --destination=${QA_IMAGE} --cache=true
retry: 2
# This image is used by:
# - The `CNG` pipelines (via the `review-build-cng` job): https://gitlab.com/gitlab-org/build/CNG/-/blob/cfc67136d711e1c8c409bf8e57427a644393da2f/.gitlab-ci.yml#L335
# - The `omnibus-gitlab` pipelines (via the `package-and-qa` job): https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/dfd1ad475868fc84e91ab7b5706aa03e46dc3a86/.gitlab-ci.yml#L130
build-assets-image:
extends:
- .base-image-build
- .use-kaniko
- .build-images:rules:build-assets-image
stage: build-images
needs: ["compile-production-assets"]
variables:
GIT_DEPTH: "1"
script:
- skopeo login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
# TODO: Change the image tag to be the MD5 of assets files and skip image building if the image exists
# We'll also need to pass GITLAB_ASSETS_TAG to the trigerred omnibus-gitlab pipeline similarly to how we do it for trigerred CNG pipelines
# https://gitlab.com/gitlab-org/gitlab/issues/208389
- run_timed_command "scripts/build_assets_image"
artifacts:
expire_in: 7 days
paths:
# The `cached-assets-hash.txt` file is used in `review-build-cng-env` (`.gitlab/ci/review-apps/main.gitlab-ci.yml`)
# to pass the assets image tag to the CNG downstream pipeline.
- cached-assets-hash.txt
build-assets-image as-if-foss:
extends:
- build-assets-image
- .as-if-foss
- .build-images:rules:build-assets-image-as-if-foss
needs: ["compile-production-assets as-if-foss"]
retry: 2

View file

@ -0,0 +1,63 @@
# Builds a cached .tar.gz of the $CI_DEFAULT_BRANCH branch with full history and
# uploads it to Google Cloud Storage. This archive is downloaded by a
# script defined by a CI/CD variable named CI_PRE_CLONE_SCRIPT. This has
# two benefits:
#
# 1. It speeds up builds. A 800 MB download only takes seconds.
# 2. It significantly reduces load on the file server. Smaller deltas
# means less time spent in git pack-objects.
#
# Since the destination directory of the archive depends on the project
# ID, this is only run on GitLab.com.
#
# CI_REPO_CACHE_CREDENTIALS contains the Google Cloud service account
# JSON for uploading to the gitlab-ci-git-repo-cache bucket. These
# credentials are stored in the Production vault.
#
# Note that this bucket should be located in the same continent as the
# runner, or network egress charges will apply:
# https://cloud.google.com/storage/pricing
cache-repo:
extends: .cache-repo:rules
image: gcr.io/google.com/cloudsdktool/cloud-sdk:alpine
stage: sync
variables:
GIT_STRATEGY: none
SHALLOW_CLONE_TAR_FILENAME: gitlab-master-shallow.tar
FULL_CLONE_TAR_FILENAME: gitlab-master.tar
before_script:
- '[ -z "$CI_REPO_CACHE_CREDENTIALS" ] || gcloud auth activate-service-account --key-file=$CI_REPO_CACHE_CREDENTIALS'
script:
# Enable shallow repo caching unless the $DISABLE_SHALLOW_REPO_CACHING variable exists (in the case the shallow clone caching isn't working well)
# The `git repack` call works around a Git bug with shallow clones: https://gitlab.com/gitlab-org/git/-/issues/86
- if [ -z "$DISABLE_SHALLOW_REPO_CACHING" ]; then
cd .. && rm -rf $CI_PROJECT_NAME;
today=$(date +%Y-%m-%d);
year=$(date +%Y);
last_year=`expr $year - 1`;
one_year_ago=$(echo $today | sed "s/$year/$last_year/");
echo "Cloning $CI_REPOSITORY_URL into $CI_PROJECT_NAME with commits from $one_year_ago.";
time git clone --progress --no-checkout --shallow-since=$one_year_ago $CI_REPOSITORY_URL $CI_PROJECT_NAME;
cd $CI_PROJECT_NAME;
time git repack -d;
echo "Archiving $CI_PROJECT_NAME into /tmp/$SHALLOW_CLONE_TAR_FILENAME.";
time git remote rm origin;
time tar cf /tmp/$SHALLOW_CLONE_TAR_FILENAME .;
echo "GZipping /tmp/$SHALLOW_CLONE_TAR_FILENAME.";
time gzip /tmp/$SHALLOW_CLONE_TAR_FILENAME;
[ -z "$CI_REPO_CACHE_CREDENTIALS" ] || (echo "Uploading /tmp/$SHALLOW_CLONE_TAR_FILENAME.gz to GCloud." && time gsutil cp /tmp/$SHALLOW_CLONE_TAR_FILENAME.gz gs://gitlab-ci-git-repo-cache/project-$CI_PROJECT_ID/$SHALLOW_CLONE_TAR_FILENAME.gz);
fi
# Disable the full repo caching unless the $DISABLE_SHALLOW_REPO_CACHING variable exists (in the case the shallow clone caching isn't working well)
- if [ -n "$DISABLE_SHALLOW_REPO_CACHING" ]; then
cd .. && rm -rf $CI_PROJECT_NAME;
echo "Cloning $CI_REPOSITORY_URL into $CI_PROJECT_NAME.";
time git clone --progress $CI_REPOSITORY_URL $CI_PROJECT_NAME;
cd $CI_PROJECT_NAME;
time git repack -d;
echo "Archiving $CI_PROJECT_NAME into /tmp/$FULL_CLONE_TAR_FILENAME.";
time git remote rm origin;
time tar cf /tmp/$FULL_CLONE_TAR_FILENAME .;
echo "GZipping /tmp/$FULL_CLONE_TAR_FILENAME.";
time gzip /tmp/$FULL_CLONE_TAR_FILENAME;
[ -z "$CI_REPO_CACHE_CREDENTIALS" ] || (echo "Uploading /tmp/$FULL_CLONE_TAR_FILENAME.gz to GCloud." && time gsutil cp /tmp/$FULL_CLONE_TAR_FILENAME.gz gs://gitlab-ci-git-repo-cache/project-$CI_PROJECT_ID/$FULL_CLONE_TAR_FILENAME.gz);
fi

View file

@ -1,64 +0,0 @@
cache-workhorse:
extends:
- .default-retry
- .default-before_script
- .rails-cache
- .setup-test-env-cache
- .caching:rules:cache-workhorse
stage: prepare
variables:
SETUP_DB: "false"
script:
- source scripts/gitlab_component_helpers.sh
- 'gitlab_workhorse_archive_doesnt_exist || { echoinfo "INFO: Exiting early as package exists."; exit 0; }'
- run_timed_command "scripts/setup-test-env"
- run_timed_command "select_gitlab_workhorse_essentials"
- run_timed_command "create_gitlab_workhorse_package"
- run_timed_command "upload_gitlab_workhorse_package"
artifacts:
expire_in: 7d
paths:
- ${TMP_TEST_GITLAB_WORKHORSE_PATH}/
.cache-assets-base:
extends:
- .compile-assets-base
- .assets-compile-cache
- .caching:rules:cache-assets
stage: prepare
variables:
WEBPACK_REPORT: "false"
script:
- yarn_install_script
- export GITLAB_ASSETS_HASH=$(bundle exec rake gitlab:assets:hash_sum)
- source scripts/gitlab_component_helpers.sh
- 'gitlab_assets_archive_doesnt_exist || { echoinfo "INFO: Exiting early as package exists."; exit 0; }'
- assets_compile_script
- echo -n "${GITLAB_ASSETS_HASH}" > "cached-assets-hash.txt"
- run_timed_command "create_gitlab_assets_package"
- run_timed_command "upload_gitlab_assets_package"
cache-assets:test:
extends: .cache-assets-base
cache-assets:test as-if-foss:
extends:
- .cache-assets-base
- .as-if-foss
cache-assets:production:
extends:
- .cache-assets-base
- .production
packages-cleanup:
extends:
- .default-retry
- .caching:rules:packages-cleanup
image: ${GITLAB_DEPENDENCY_PROXY_ADDRESS}ruby:${RUBY_VERSION}
stage: prepare
before_script:
- source scripts/utils.sh
- install_gitlab_gem
script:
- scripts/packages/automated_cleanup.rb

View file

@ -1,13 +0,0 @@
templates-shellcheck:
extends:
- .ci-templates:rules:shellcheck
- .default-before_script
- .default-retry
- .ruby-cache
- .use-pg14
stage: test
needs:
- setup-test-env
script:
- apt update && apt install -y shellcheck=0.7.1-1+deb11u1
- bundle exec scripts/lint_templates_bash.rb

View file

@ -0,0 +1,10 @@
cloud-native-image:
extends: .cng:rules
image: ${GITLAB_DEPENDENCY_PROXY}ruby:2.7-alpine
dependencies: []
stage: post-test
variables:
GIT_DEPTH: "1"
script:
- install_gitlab_gem
- CNG_PROJECT_PATH="gitlab-org/build/CNG" ./scripts/trigger-build cng

View file

@ -0,0 +1,205 @@
.dast_conf:
tags:
- prm
# For scheduling dast job
extends:
- .reports:rules:schedule-dast
image:
name: "registry.gitlab.com/gitlab-org/security-products/dast:$DAST_VERSION"
resource_group: dast_scan
variables:
DAST_USERNAME_FIELD: "user[login]"
DAST_PASSWORD_FIELD: "user[password]"
DAST_FULL_SCAN_ENABLED: "true"
DAST_SPIDER_MINS: 0
# TBD pin to a version
DAST_VERSION: 1.22.1
# -Xmx is used to set the JVM memory to 6GB to prevent DAST OutOfMemoryError.
DAST_ZAP_CLI_OPTIONS: "-Xmx6144m"
DAST_RULES: "41,42,43,10027,10032,10041,10042,10045,10047,10052,10053,10057,10061,10096,10097,10104,10106,20012,20014,20015,20016,20017,20018,40019,40020,40021,40024,40025,40027,40029,40032,90001,90019,10109,10026,10028,10029,10030,10031,10033,10034,10035,10036,10038,10039,10043,10044,10048,10050,10051,10058,10062,10095,10107,10108,30003,40013,40022,40023,40028,90021,90023,90024,90025,90027,90028,10003,50003,0,2,3,6,7,10010,10011,10015,10017,10019,10020,10021,10023,10024,10025,10037,10040,10054,10055,10056,10098,10105,10202,20019,30001,30002,40003,40008,40009,40012,40014,40016,40017,40018,50000,50001,90011,90020,90022,90033"
before_script:
- 'export DAST_WEBSITE="${DAST_WEBSITE:-$(cat environment_url.txt)}"'
- 'export DAST_AUTH_URL="${DAST_WEBSITE}/users/sign_in"'
- 'export DAST_PASSWORD="${REVIEW_APPS_ROOT_PASSWORD}"'
# Below three lines can be removed once https://gitlab.com/gitlab-org/gitlab/-/issues/230687 is fixed
- mkdir -p /zap/xml
- 'sed -i "84 s/true/false/" /zap/xml/config.xml'
- cat /zap/xml/config.xml
# Help pages are excluded from scan as they are static pages.
# profile/two_factor_auth is excluded from scan to prevent 2FA from being turned on from user profile, which will reduce coverage.
- 'export DAST_AUTH_EXCLUDE_URLS="${DAST_WEBSITE}/help/.*,${DAST_WEBSITE}/profile/two_factor_auth,${DAST_WEBSITE}/users/sign_out"'
# Exclude the automatically generated monitoring project from being tested due to https://gitlab.com/gitlab-org/gitlab/-/issues/260362
- 'DAST_AUTH_EXCLUDE_URLS="${DAST_AUTH_EXCLUDE_URLS},https://.*\.gitlab-review\.app/gitlab-instance-(administrators-)?[a-zA-Z0-9]{8}/.*"'
- enable_rule () { read all_rules; rule=$1; echo $all_rules | sed -r "s/(,)?$rule(,)?/\1-1\2/" ; }
# Sort ids in DAST_RULES ascendingly, which is required when using DAST_RULES as argument to enable_rule
- 'DAST_RULES=$(echo $DAST_RULES | tr "," "\n" | sort -n | paste -sd ",")'
needs: ["review-deploy"]
stage: dast
# Default job timeout set to 90m and dast rules needs 2h to so that it won't timeout.
timeout: 2h
# Add retry because of intermittent connection problems. See https://gitlab.com/gitlab-org/gitlab/-/issues/244313
retry: 1
artifacts:
paths:
- gl-dast-report.json # GitLab-specific
reports:
dast: gl-dast-report.json
expire_in: 1 week # GitLab-specific
# DAST scan with a subset of Release scan rules.
DAST-fullscan-ruleset1:
extends:
- .dast_conf
variables:
DAST_USERNAME: "user1"
script:
- export DAST_EXCLUDE_RULES=$(echo $DAST_RULES | enable_rule 10019 | enable_rule 10020 | enable_rule 10021 | enable_rule 10023 | enable_rule 10024 | enable_rule 10025 | enable_rule 10037 | enable_rule 10040 | enable_rule 10054 | enable_rule 10055 | enable_rule 10056)
- echo $DAST_EXCLUDE_RULES
- /analyze -t $DAST_WEBSITE -d
# DAST scan with a subset of Release scan rules.
DAST-fullscan-ruleset2:
extends:
- .dast_conf
variables:
DAST_USERNAME: "user2"
script:
- export DAST_EXCLUDE_RULES=$(echo $DAST_RULES | enable_rule 90011 | enable_rule 90020 | enable_rule 90022 | enable_rule 90033)
- echo $DAST_EXCLUDE_RULES
- /analyze -t $DAST_WEBSITE -d
# DAST scan with a subset of Release scan rules.
DAST-fullscan-ruleset3:
extends:
- .dast_conf
variables:
DAST_USERNAME: "user3"
script:
- export DAST_EXCLUDE_RULES=$(echo $DAST_RULES | enable_rule 40016 | enable_rule 40017 | enable_rule 50000 | enable_rule 50001)
- echo $DAST_EXCLUDE_RULES
- /analyze -t $DAST_WEBSITE -d
# DAST scan with a subset of Release scan rules.
DAST-fullscan-ruleset4:
extends:
- .dast_conf
variables:
DAST_USERNAME: "user4"
script:
- export DAST_EXCLUDE_RULES=$(echo $DAST_RULES | enable_rule 0 | enable_rule 2 | enable_rule 3 | enable_rule 7 )
- echo $DAST_EXCLUDE_RULES
- /analyze -t $DAST_WEBSITE -d
# DAST scan with a subset of Release scan rules.
DAST-fullscan-ruleset5:
extends:
- .dast_conf
variables:
DAST_USERNAME: "user5"
script:
- export DAST_EXCLUDE_RULES=$(echo $DAST_RULES | enable_rule 10010 | enable_rule 10011 | enable_rule 10017 | enable_rule 10019)
- echo $DAST_EXCLUDE_RULES
- /analyze -t $DAST_WEBSITE -d
# DAST scan with a subset of Release scan rules.
DAST-fullscan-ruleset6:
extends:
- .dast_conf
variables:
DAST_USERNAME: "user6"
script:
- export DAST_EXCLUDE_RULES=$(echo $DAST_RULES | enable_rule 30001 | enable_rule 40009)
- echo $DAST_EXCLUDE_RULES
- /analyze -t $DAST_WEBSITE -d
# Enable when https://gitlab.com/gitlab-org/gitlab/-/merge_requests/39749 is fixed
# DAST scan with a subset of Beta scan rules.
# DAST-fullscan-ruleset7:
# extends:
# - .dast_conf
# variables:
# DAST_USERNAME: "user7"
# script:
# - export DAST_EXCLUDE_RULES=$(echo $DAST_RULES | enable_rule 10098 | enable_rule 10105 | enable_rule 10202 | enable_rule 30002 | enable_rule 40003 | enable_rule 40008 | enable_rule 40009)
# - echo $DAST_EXCLUDE_RULES
# - /analyze -t $DAST_WEBSITE -d
# Enable when https://gitlab.com/gitlab-org/gitlab/-/merge_requests/39749 is fixed
# Below jobs runs DAST scans with one time consuming scan rule. These scan rules are disabled in above jobs so that those jobs won't timeout.
# DAST scan with rule - 20019 External Redirect
# DAST-fullscan-rule-20019:
# extends:
# - .dast_conf
# variables:
# DAST_USERNAME: "user8"
# script:
# - export DAST_EXCLUDE_RULES=$(echo $DAST_RULES | enable_rule 20019)
# - echo $DAST_EXCLUDE_RULES
# - /analyze -t $DAST_WEBSITE -d
# Enable when https://gitlab.com/gitlab-org/gitlab/-/merge_requests/39749 is fixed
# DAST scan with rule - 10107 Httpoxy - Proxy Header Misuse - Active/beta
# DAST-fullscan-rule-10107:
# extends:
# - .dast_conf
# variables:
# DAST_USERNAME: "user9"
# script:
# - export DAST_EXCLUDE_RULES=$(echo $DAST_RULES | enable_rule 10107)
# - echo $DAST_EXCLUDE_RULES
# - /analyze -t $DAST_WEBSITE -d
# DAST scan with rule - 90020 Remote OS Command Injection
DAST-fullscan-rule-90020:
extends:
- .dast_conf
variables:
DAST_USERNAME: "user10"
script:
- export DAST_EXCLUDE_RULES=$(echo $DAST_RULES | enable_rule 90020)
- echo $DAST_EXCLUDE_RULES
- /analyze -t $DAST_WEBSITE -d
# DAST scan with rule - 40018 SQL Injection - Active/release
DAST-fullscan-rule-40018:
extends:
- .dast_conf
variables:
DAST_USERNAME: "user11"
script:
- export DAST_EXCLUDE_RULES=$(echo $DAST_RULES | enable_rule 40018)
- echo $DAST_EXCLUDE_RULES
- /analyze -t $DAST_WEBSITE -d
# DAST scan with rule - 40014 Cross Site Scripting (Persistent) - Active/release
DAST-fullscan-rule-40014:
extends:
- .dast_conf
variables:
DAST_USERNAME: "user12"
script:
- export DAST_EXCLUDE_RULES=$(echo $DAST_RULES | enable_rule 40014)
- echo $DAST_EXCLUDE_RULES
- /analyze -t $DAST_WEBSITE -d
# DAST scan with rule - 6 Path travesal
DAST-fullscan-rule-6:
extends:
- .dast_conf
variables:
DAST_USERNAME: "user13"
script:
- export DAST_EXCLUDE_RULES=$(echo $DAST_RULES | enable_rule 6)
- echo $DAST_EXCLUDE_RULES
- /analyze -t $DAST_WEBSITE -d
# DAST scan with rule - 40012 Cross Site Scripting (Reflected)
DAST-fullscan-rule-40012:
extends:
- .dast_conf
variables:
DAST_USERNAME: "user14"
script:
- export DAST_EXCLUDE_RULES=$(echo $DAST_RULES | enable_rule 40012)
- echo $DAST_EXCLUDE_RULES
- /analyze -t $DAST_WEBSITE -d

View file

@ -1,147 +0,0 @@
include:
- local: .gitlab/ci/rails/shared.gitlab-ci.yml
db:rollback single-db-ci-connection:
extends:
- db:rollback
- .single-db-ci-connection
- .rails:rules:single-db-ci-connection
db:migrate:reset single-db-ci-connection:
extends:
- db:migrate:reset
- .single-db-ci-connection
- .rails:rules:single-db-ci-connection
db:check-schema-single-db-ci-connection:
extends:
- db:check-schema
- .single-db-ci-connection
- .rails:rules:single-db-ci-connection
db:post_deployment_migrations_validator-single-db-ci-connection:
extends:
- db:post_deployment_migrations_validator
- .single-db-ci-connection
- .rails:rules:db:check-migrations-single-db-ci-connection
db:backup_and_restore single-db-ci-connection:
extends:
- db:backup_and_restore
- .single-db-ci-connection
- .rails:rules:db-backup
db:rollback:
extends:
- .db-job-base
- .rails:rules:db-rollback
script:
- bundle exec rake db:migrate VERSION=20220502173045 # 14.10 (last 14.x version)
- bundle exec rake db:migrate
db:rollback single-db:
extends:
- db:rollback
- .single-db
- .rails:rules:single-db
db:migrate:reset:
extends: .db-job-base
script:
- bundle exec rake db:migrate:reset
db:migrate:reset single-db:
extends:
- db:migrate:reset
- .single-db
- .rails:rules:single-db
db:check-schema:
extends:
- .db-job-base
- .rails:rules:ee-mr-and-default-branch-only
script:
- run_timed_command "bundle exec rake db:drop db:create db:migrate"
db:check-schema-single-db:
extends:
- db:check-schema
- .single-db
- .rails:rules:single-db
db:check-migrations:
extends:
- .db-job-base
- .rails:rules:ee-and-foss-mr-with-migration
script:
- git fetch origin $CI_MERGE_REQUEST_TARGET_BRANCH_NAME:$CI_MERGE_REQUEST_TARGET_BRANCH_NAME --depth 20
- scripts/validate_migration_schema
allow_failure: true
db:check-migrations-single-db:
extends:
- db:check-migrations
- .single-db
- .rails:rules:db:check-migrations-single-db
db:post_deployment_migrations_validator:
extends:
- .db-job-base
- .rails:rules:ee-and-foss-mr-with-migration
script:
- git fetch origin $CI_MERGE_REQUEST_TARGET_BRANCH_NAME:$CI_MERGE_REQUEST_TARGET_BRANCH_NAME --depth 20
- scripts/post_deployment_migrations_validator
allow_failure: true
db:post_deployment_migrations_validator-single-db:
extends:
- db:post_deployment_migrations_validator
- .single-db
- .rails:rules:db:check-migrations-single-db
db:migrate-non-superuser:
extends:
- .db-job-base
- .rails:rules:ee-and-foss-mr-with-migration
script:
- bundle exec rake gitlab:db:reset_as_non_superuser
db:gitlabcom-database-testing:
extends: .rails:rules:db:gitlabcom-database-testing
stage: test
image: ruby:${RUBY_VERSION}-alpine
needs: []
allow_failure: true
script:
- source scripts/utils.sh
- install_gitlab_gem
- ./scripts/trigger-build.rb gitlab-com-database-testing
db:backup_and_restore:
extends:
- .db-job-base
- .rails:rules:db-backup
variables:
SETUP_DB: "false"
GITLAB_ASSUME_YES: "1"
script:
- . scripts/prepare_build.sh
- bundle exec rake db:drop db:create db:schema:load db:seed_fu
- mkdir -p tmp/tests/public/uploads tmp/tests/{artifacts,pages,lfs-objects,terraform_state,registry,packages}
- bundle exec rake gitlab:backup:create
- date
- bundle exec rake gitlab:backup:restore
db:backup_and_restore single-db:
extends:
- db:backup_and_restore
- .single-db
- .rails:rules:db-backup
db:rollback geo:
extends:
- db:rollback
- .rails:rules:ee-only-migration
script:
- bundle exec rake db:migrate:geo VERSION=20170627195211
- bundle exec rake db:migrate:geo

View file

@ -3,7 +3,7 @@
- .default-retry
- .rails-cache
- .default-before_script
- .use-pg13
- .use-pg12
stage: test
needs: ["setup-test-env"]
variables:
@ -15,8 +15,8 @@
# SEED_NESTED_GROUPS: "false" # requires network connection
.run-dev-fixtures-script: &run-dev-fixtures-script
- section_start "gitaly-test-spawn" "Spawning Gitaly"; scripts/gitaly-test-spawn; section_end "gitaly-test-spawn"; # Do not use 'bundle exec' here
- section_start "seeding-db" "Seeding DB"; bundle exec rake db:seed_fu; section_end "seeding-db";
- run_timed_command "scripts/gitaly-test-spawn"
- run_timed_command "RAILS_ENV=test bundle exec rake db:seed_fu"
run-dev-fixtures:
extends:
@ -29,7 +29,7 @@ run-dev-fixtures-ee:
extends:
- .run-dev-fixtures
- .dev-fixtures:rules:ee-only
- .use-pg13-es7-ee
- .use-pg12-ee
script:
- cp ee/db/fixtures/development/* $FIXTURE_PATH
- *run-dev-fixtures-script

View file

@ -2,7 +2,7 @@
extends:
- .default-retry
- .docs:rules:review-docs
image: ${GITLAB_DEPENDENCY_PROXY_ADDRESS}ruby:${RUBY_VERSION}-alpine
image: ${GITLAB_DEPENDENCY_PROXY}ruby:2.7-alpine
stage: review
needs: []
variables:
@ -10,8 +10,8 @@
# because some repos are private and CI_JOB_TOKEN cannot access files.
# See https://gitlab.com/gitlab-org/gitlab/issues/191273
GIT_DEPTH: 1
# By default, deploy the Review App using the `main` branch of the `gitlab-org/gitlab-docs` project
DOCS_BRANCH: main
# By default, deploy the Review App using the `master` branch of the `gitlab-org/gitlab-docs` project
DOCS_BRANCH: master
environment:
name: review-docs/mr-${CI_MERGE_REQUEST_IID}
# DOCS_REVIEW_APPS_DOMAIN and DOCS_GITLAB_REPO_SUFFIX are CI variables
@ -28,7 +28,7 @@
review-docs-deploy:
extends: .review-docs
script:
- ./scripts/trigger-build.rb docs deploy
- ./scripts/trigger-build docs deploy
# Cleanup remote environment of gitlab-docs
review-docs-cleanup:
@ -37,13 +37,26 @@ review-docs-cleanup:
name: review-docs/mr-${CI_MERGE_REQUEST_IID}
action: stop
script:
- ./scripts/trigger-build.rb docs cleanup
- ./scripts/trigger-build docs cleanup
docs-lint markdown:
extends:
- .default-retry
- .docs:rules:docs-lint
# When updating the image version here, update it in /scripts/lint-doc.sh too.
image: registry.gitlab.com/gitlab-org/gitlab-docs/lint-markdown:alpine-3.13-vale-2.10.2-markdownlint-0.26.0
stage: test
needs: []
script:
- scripts/lint-doc.sh
docs-lint links:
extends:
- .docs:rules:docs-lint
image: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-docs/lint-html:alpine-3.17-ruby-3.2.1-f53af000
stage: lint
image: registry.gitlab.com/gitlab-org/gitlab-docs/lint-html:alpine-3.13-ruby-2.7.2
# TODO: revert to .default-retry when https://gitlab.com/gitlab-org/gitlab/-/issues/331002 is fixed.
retry: 2
stage: test
needs: []
script:
# Prepare docs for build
@ -56,71 +69,11 @@ docs-lint links:
# Check the internal links and anchors (in parallel)
- "parallel time bundle exec nanoc check ::: internal_links internal_anchors"
.docs-markdown-lint-image:
# When updating the image version here, update it in /scripts/lint-doc.sh too.
image: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-docs/lint-markdown:alpine-3.17-vale-2.24.0-markdownlint-0.33.0-markdownlint2-0.6.0
docs-lint markdown:
extends:
- .default-retry
- .docs:rules:docs-lint
- .docs-markdown-lint-image
- .yarn-cache
stage: lint
needs: []
script:
- source ./scripts/utils.sh
- yarn_install_script
- scripts/lint-doc.sh
docs-lint blueprint:
extends:
- .default-retry
- .docs:rules:docs-blueprints-lint
image: ${GITLAB_DEPENDENCY_PROXY_ADDRESS}ruby:${RUBY_VERSION}-slim
stage: lint
needs: []
script:
- scripts/lint-docs-blueprints.rb
docs code_quality:
extends:
- .reports:rules:code_quality
- .docs-markdown-lint-image
stage: lint
needs: []
dependencies: []
allow_failure: true
script:
- vale --output=doc/.vale/vale-json.tmpl --minAlertLevel warning doc > gl-code-quality-report-docs.json || exit_code=$?
artifacts:
reports:
codequality: gl-code-quality-report-docs.json
paths:
- gl-code-quality-report-docs.json
expire_in: 2 weeks
when: always
ui-docs-links lint:
extends:
- .docs:rules:docs-lint
- .static-analysis-base
- .ruby-cache
stage: lint
stage: test
needs: []
script:
- bundle exec haml-lint -i DocumentationLinks
docs-lint deprecations-and-removals:
variables:
SETUP_DB: "false"
extends:
- .default-retry
- .rails-cache
- .default-before_script
- .docs:rules:deprecations-and-removals
stage: lint
needs: []
script:
- bundle exec rake gitlab:docs:check_deprecations
- bundle exec rake gitlab:docs:check_removals

View file

@ -1,40 +1,33 @@
.yarn-install: &yarn-install
- source scripts/utils.sh
- run_timed_command "retry yarn install --frozen-lockfile"
.compile-assets-base:
extends:
- .default-retry
- .default-before_script
- .assets-compile-cache
image: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images/debian-${DEBIAN_VERSION}-ruby-${RUBY_VERSION}-node-16.14:rubygems-${RUBYGEMS_VERSION}-git-2.33-lfs-2.9-yarn-1.22-graphicsmagick-1.3.36
image: registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.7.2-git-2.31-lfs-2.9-node-14.15-yarn-1.22-graphicsmagick-1.3.36
variables:
SETUP_DB: "false"
WEBPACK_VENDOR_DLL: "true"
# Disable warnings in browserslist which can break on backports
# https://github.com/browserslist/browserslist/blob/a287ec6/node.js#L367-L384
BROWSERSLIST_IGNORE_OLD_DATA: "true"
WEBPACK_COMPILE_LOG_PATH: "tmp/webpack-output.log"
stage: prepare
script:
- yarn_install_script
- export GITLAB_ASSETS_HASH=$(bin/rake gitlab:assets:hash_sum)
- 'echo "CACHE_ASSETS_AS_PACKAGE: ${CACHE_ASSETS_AS_PACKAGE}"'
# The new strategy to cache assets as generic packages is experimental and can be disabled by removing the `CACHE_ASSETS_AS_PACKAGE` variable
- |
if [[ "${CACHE_ASSETS_AS_PACKAGE}" == "true" ]]; then
source scripts/gitlab_component_helpers.sh
if ! gitlab_assets_archive_doesnt_exist; then
# We remove all assets from the native cache as they could pollute the fresh assets from the package
rm -rf public/assets/ app/assets/javascripts/locale/**/app.js
run_timed_command "download_and_extract_gitlab_assets"
fi
fi
- assets_compile_script
- echo -n "${GITLAB_ASSETS_HASH}" > "cached-assets-hash.txt"
- *yarn-install
- run_timed_command "bin/rake gitlab:assets:compile"
- run_timed_command "scripts/clean-old-cached-assets"
compile-production-assets:
extends:
- .compile-assets-base
- .production
- .frontend:rules:compile-production-assets
variables:
NODE_ENV: "production"
RAILS_ENV: "production"
WEBPACK_REPORT: "true"
artifacts:
name: webpack-report
expire_in: 31d
@ -42,19 +35,12 @@ compile-production-assets:
# These assets are used in multiple locations:
# - in `build-assets-image` job to create assets image for packaging systems
# - GitLab UI for integration tests: https://gitlab.com/gitlab-org/gitlab-ui/-/blob/e88493b3c855aea30bf60baee692a64606b0eb1e/.storybook/preview-head.pug#L1
- cached-assets-hash.txt
- public/assets/
- "${WEBPACK_COMPILE_LOG_PATH}"
- webpack-report/
when: always
after_script:
- rm -f /etc/apt/sources.list.d/google*.list # We don't need to update Chrome here
compile-production-assets as-if-foss:
extends:
- compile-production-assets
- .as-if-foss
- .frontend:rules:compile-production-assets-as-if-foss
compile-test-assets:
extends:
- .compile-assets-base
@ -64,7 +50,6 @@ compile-test-assets:
paths:
- public/assets/
- node_modules/@gitlab/svgs/dist/icons.json # app/helpers/icons_helper.rb uses this file
- "${WEBPACK_COMPILE_LOG_PATH}"
when: always
compile-test-assets as-if-foss:
@ -87,67 +72,32 @@ update-assets-compile-test-cache:
- .assets-compile-cache-push
- .shared:rules:update-cache
stage: prepare
script:
- !reference [compile-test-assets, script]
- echo -n "${GITLAB_ASSETS_HASH}" > "cached-assets-hash.txt"
artifacts: {} # This job's purpose is only to update the cache.
update-storybook-yarn-cache:
update-yarn-cache:
extends:
- .default-retry
- .default-utils-before_script
- .storybook-yarn-cache-push
- .yarn-cache-push
- .shared:rules:update-cache
stage: prepare
script:
- yarn_install_script
- *yarn-install
retrieve-frontend-fixtures:
variables:
SETUP_DB: "false"
extends:
- .default-retry
- .frontend:rules:default-frontend-jobs
stage: prepare
script:
- source scripts/utils.sh
- source scripts/gitlab_component_helpers.sh
- install_gitlab_gem
- export_fixtures_sha_for_download
- |
if check_fixtures_download; then
run_timed_command "download_and_extract_fixtures"
fi
artifacts:
paths:
- tmp/tests/frontend/
# Download fixtures only when a merge request contains changes to only JS files
# and fixtures are present in the package registry.
.frontend-fixtures-base:
extends:
- .default-retry
- .default-before_script
- .rails-cache
- .use-pg13
- .use-pg12
stage: fixtures
needs: ["setup-test-env", "retrieve-tests-metadata", "retrieve-frontend-fixtures"]
needs: ["setup-test-env", "retrieve-tests-metadata", "compile-test-assets"]
variables:
# Don't add `CRYSTALBALL: "false"` here as we're enabling Crystalball for scheduled pipelines (in `.gitlab-ci.yml`), so that we get coverage data
# for the `frontend fixture RSpec files` that will be added to the Crystalball mapping in `update-tests-metadata`.
# More information in https://gitlab.com/gitlab-org/gitlab/-/merge_requests/74003.
WEBPACK_VENDOR_DLL: "true"
script:
- source scripts/gitlab_component_helpers.sh
- |
if check_fixtures_reuse; then
echoinfo "INFO: Reusing frontend fixtures from 'retrieve-frontend-fixtures'."
exit 0
fi
- run_timed_command "gem install knapsack --no-document"
- section_start "gitaly-test-spawn" "Spawning Gitaly"; scripts/gitaly-test-spawn; section_end "gitaly-test-spawn"; # Do not use 'bundle exec' here
- run_timed_command "scripts/gitaly-test-spawn"
- source ./scripts/rspec_helpers.sh
- rspec_paralellized_job
- rspec_paralellized_job "--tag frontend_fixture"
artifacts:
name: frontend-fixtures
expire_in: 31d
@ -155,51 +105,23 @@ retrieve-frontend-fixtures:
paths:
- tmp/tests/frontend/
- knapsack/
- crystalball/
# Builds FOSS, and EE fixtures in the EE project.
# Builds FOSS fixtures in the FOSS project.
rspec-all frontend_fixture:
rspec frontend_fixture:
extends:
- .frontend-fixtures-base
- .frontend:rules:default-frontend-jobs
needs:
- !reference [.frontend-fixtures-base, needs]
- "compile-test-assets"
parallel: 7
# Builds FOSS fixtures in the EE project, with the `ee/` folder removed (due to `as-if-foss`).
rspec-all frontend_fixture as-if-foss:
rspec frontend_fixture as-if-foss:
extends:
- .frontend-fixtures-base
- .frontend:rules:frontend_fixture-as-if-foss
- .frontend:rules:default-frontend-jobs-as-if-foss
- .as-if-foss
variables:
# We explicitely disable Crystalball here so as even in scheduled pipelines we don't need it since it's already enabled for `rspec-all frontend_fixture` there.
CRYSTALBALL: "false"
WEBPACK_VENDOR_DLL: "true"
KNAPSACK_GENERATE_REPORT: ""
FLAKY_RSPEC_GENERATE_REPORT: ""
needs:
- !reference [.frontend-fixtures-base, needs]
- "compile-test-assets as-if-foss"
# Uploads EE fixtures in the EE project.
# Uploads FOSS fixtures in the FOSS project.
upload-frontend-fixtures:
rspec-ee frontend_fixture:
extends:
- .frontend-fixtures-base
- .frontend:rules:upload-frontend-fixtures
stage: fixtures
needs: ["rspec-all frontend_fixture"]
script:
- source scripts/utils.sh
- source scripts/gitlab_component_helpers.sh
- export_fixtures_sha_for_upload
- 'fixtures_archive_doesnt_exist || { echoinfo "INFO: Exiting early as package exists."; exit 0; }'
- run_timed_command "create_fixtures_package"
- run_timed_command "upload_fixtures_package"
artifacts: {}
- .frontend:rules:default-frontend-jobs-ee
parallel: 2
graphql-schema-dump:
variables:
@ -219,12 +141,6 @@ graphql-schema-dump:
- tmp/tests/graphql/gitlab_schema.graphql
- tmp/tests/graphql/gitlab_schema.json
graphql-schema-dump as-if-foss:
extends:
- graphql-schema-dump
- .frontend:rules:default-frontend-jobs-as-if-foss
- .as-if-foss
.frontend-test-base:
extends:
- .default-retry
@ -233,23 +149,67 @@ graphql-schema-dump as-if-foss:
# Disable warnings in browserslist which can break on backports
# https://github.com/browserslist/browserslist/blob/a287ec6/node.js#L367-L384
BROWSERSLIST_IGNORE_OLD_DATA: "true"
USE_BUNDLE_INSTALL: "false"
SETUP_DB: "false"
before_script:
- !reference [.default-before_script, before_script]
- yarn_install_script
stage: test
eslint-as-if-foss:
extends:
- .frontend-test-base
- .frontend:rules:eslint-as-if-foss
- .as-if-foss
needs: []
script:
- *yarn-install
- run_timed_command "yarn run lint:eslint:all"
.karma-base:
extends: .frontend-test-base
script:
- export BABEL_ENV=coverage CHROME_LOG_FILE=chrome_debug.log
- *yarn-install
- run_timed_command "yarn karma"
karma:
extends:
- .karma-base
- .frontend:rules:default-frontend-jobs
needs:
- job: "rspec frontend_fixture"
- job: "rspec-ee frontend_fixture"
optional: true
coverage: '/^Statements *: (\d+\.\d+%)/'
artifacts:
name: coverage-javascript
expire_in: 31d
when: always
paths:
- chrome_debug.log
- coverage-javascript/
- tmp/tests/frontend/
reports:
junit: junit_karma.xml
cobertura: coverage-javascript/cobertura-coverage.xml
karma-as-if-foss:
extends:
- .karma-base
- .frontend:rules:default-frontend-jobs-as-if-foss
- .as-if-foss
needs: ["rspec frontend_fixture as-if-foss"]
.jest-base:
extends: .frontend-test-base
script:
- run_timed_command "yarn jest:ci"
- *yarn-install
- run_timed_command "yarn jest --ci --coverage --testSequencer ./scripts/frontend/parallel_ci_sequencer.js"
jest:
extends:
- .jest-base
- .frontend:rules:jest
needs: ["rspec-all frontend_fixture"]
- .frontend:rules:default-frontend-jobs
needs:
- job: "rspec frontend_fixture"
- job: "rspec-ee frontend_fixture"
optional: true
artifacts:
name: coverage-frontend
expire_in: 31d
@ -260,64 +220,42 @@ jest:
- tmp/tests/frontend/
reports:
junit: junit_jest.xml
parallel: 7
jest predictive:
extends:
- jest
- .frontend:rules:jest:predictive
needs:
- !reference [jest, needs]
- "detect-tests"
script:
- if [[ -s "$RSPEC_CHANGED_FILES_PATH" ]] || [[ -s "$RSPEC_MATCHING_JS_FILES_PATH" ]]; then run_timed_command "yarn jest:ci:predictive"; fi
jest as-if-foss:
extends:
- .jest-base
- .frontend:rules:jest:as-if-foss
- .as-if-foss
needs: ["rspec-all frontend_fixture as-if-foss"]
parallel: 4
jest predictive as-if-foss:
extends:
- .jest-base
- .frontend:rules:jest:predictive:as-if-foss
- .as-if-foss
needs:
- "rspec-all frontend_fixture as-if-foss"
- "detect-tests"
script:
- if [[ -s "$RSPEC_CHANGED_FILES_PATH" ]] || [[ -s "$RSPEC_MATCHING_JS_FILES_PATH" ]]; then run_timed_command "yarn jest:ci:predictive"; fi
jest-integration:
extends:
- .frontend-test-base
- .frontend:rules:default-frontend-jobs
script:
- *yarn-install
- run_timed_command "yarn jest:integration --ci"
needs: ["rspec-all frontend_fixture", "graphql-schema-dump"]
needs:
- job: "rspec frontend_fixture"
- job: "rspec-ee frontend_fixture"
optional: true
- job: "graphql-schema-dump"
jest-as-if-foss:
extends:
- .jest-base
- .frontend:rules:default-frontend-jobs-as-if-foss
- .as-if-foss
needs: ["rspec frontend_fixture as-if-foss"]
parallel: 2
coverage-frontend:
extends:
- .default-retry
- .default-utils-before_script
- .yarn-cache
- .frontend:rules:coverage-frontend
needs:
- job: "jest"
optional: true
- job: "jest predictive"
optional: true
- .frontend:rules:ee-mr-and-default-branch-only
needs: ["jest"]
stage: post-test
before_script:
- *yarn-install
script:
- yarn_install_script
- run_timed_command "yarn node scripts/frontend/merge_coverage_frontend.js"
# Removing the individual coverage results, as we just merged them.
- if ls coverage-frontend/jest-* > /dev/null 2>&1; then
rm -r coverage-frontend/jest-*;
fi
- rm -r coverage-frontend/jest-*
coverage: '/^Statements\s*:\s*?(\d+(?:\.\d+)?)%/'
artifacts:
name: coverage-frontend
@ -325,14 +263,32 @@ coverage-frontend:
paths:
- coverage-frontend/
reports:
coverage_report:
coverage_format: cobertura
path: coverage-frontend/cobertura-coverage.xml
cobertura: coverage-frontend/cobertura-coverage.xml
.qa-frontend-node:
extends:
- .default-retry
- .yarn-cache
- .frontend:rules:qa-frontend-node
stage: test
dependencies: []
script:
- *yarn-install
- run_timed_command "retry yarn run webpack-prod"
qa-frontend-node:10:
extends: .qa-frontend-node
image: ${GITLAB_DEPENDENCY_PROXY}node:dubnium
qa-frontend-node:latest:
extends:
- .qa-frontend-node
- .frontend:rules:qa-frontend-node-latest
image: ${GITLAB_DEPENDENCY_PROXY}node:latest
webpack-dev-server:
extends:
- .default-retry
- .default-utils-before_script
- .yarn-cache
- .frontend:rules:default-frontend-jobs
stage: test
@ -341,7 +297,7 @@ webpack-dev-server:
WEBPACK_MEMORY_TEST: "true"
WEBPACK_VENDOR_DLL: "true"
script:
- yarn_install_script
- *yarn-install
- run_timed_command "retry yarn webpack-vendor"
- run_timed_command "node --expose-gc node_modules/.bin/webpack-dev-server --config config/webpack.config.js"
artifacts:
@ -353,72 +309,18 @@ webpack-dev-server:
bundle-size-review:
extends:
- .default-retry
- .default-utils-before_script
- .assets-compile-cache
- .frontend:rules:bundle-size-review
image: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:danger
image: registry.gitlab.com/gitlab-org/gitlab-build-images:danger
stage: test
needs: []
needs: ["compile-production-assets"]
script:
- yarn_install_script
- scripts/bundle_size_review
- mkdir -p bundle-size-review
- cp webpack-report/index.html bundle-size-review/bundle-report.html
- yarn global add https://gitlab.com/gitlab-org/frontend/playground/webpack-memory-metrics.git
- danger --dangerfile=danger/bundle_size/Dangerfile --fail-on-errors=true --verbose --danger_id=bundle-size-review
artifacts:
when: always
name: bundle-size-review
expire_in: 31d
paths:
- bundle-size-review/
.startup-css-check-base:
extends:
- .frontend-test-base
script:
- run_timed_command "yarn generate:startup_css"
- yarn check:startup_css
startup-css-check:
extends:
- .startup-css-check-base
- .frontend:rules:default-frontend-jobs
needs: ["compile-test-assets", "rspec-all frontend_fixture"]
startup-css-check as-if-foss:
extends:
- .startup-css-check-base
- .as-if-foss
- .frontend:rules:default-frontend-jobs-as-if-foss
needs:
- job: "compile-test-assets as-if-foss"
- job: "rspec-all frontend_fixture as-if-foss"
.compile-storybook-base:
extends:
- .frontend-test-base
- .storybook-yarn-cache
script:
- run_timed_command "retry yarn run storybook:install --frozen-lockfile"
- run_timed_command "yarn run storybook:build"
needs: ["graphql-schema-dump"]
compile-storybook:
extends:
- .compile-storybook-base
- .frontend:rules:default-frontend-jobs
needs:
- !reference [.compile-storybook-base, needs]
- job: "rspec-all frontend_fixture"
artifacts:
name: storybook
expire_in: 31d
when: always
paths:
- storybook/public
compile-storybook as-if-foss:
extends:
- .compile-storybook-base
- .as-if-foss
- .frontend:rules:default-frontend-jobs-as-if-foss
needs:
- job: "graphql-schema-dump as-if-foss"
- job: "rspec-all frontend_fixture as-if-foss"
- bundle-size-review

View file

@ -1,16 +0,0 @@
glfm-verify:
extends:
- .rails-job-base
- .glfm:rules:glfm-verify
- .use-pg13
stage: test
needs: ["setup-test-env"]
script:
- !reference [.base-script, script]
- bundle exec scripts/glfm/verify-all-generated-files-are-up-to-date.rb
artifacts:
name: changed-files
when: on_failure
expire_in: 31d
paths:
- glfm_specification/

View file

@ -2,35 +2,22 @@
retry:
max: 2 # This is confusing but this means "3 runs at max".
when:
- api_failure
- data_integrity_failure
- job_execution_timeout
- runner_system_failure
- scheduler_failure
- stuck_or_timeout_failure
- unknown_failure
- api_failure
- runner_system_failure
- job_execution_timeout
- stuck_or_timeout_failure
.default-utils-before_script:
.default-before_script:
before_script:
- echo $FOSS_ONLY
- '[ "$FOSS_ONLY" = "1" ] && rm -rf ee/ qa/spec/ee/ qa/qa/specs/features/ee/ qa/qa/ee/ qa/qa/ee.rb'
- export GOPATH=$CI_PROJECT_DIR/.go
- mkdir -p $GOPATH
- source scripts/utils.sh
.default-before_script:
before_script:
- !reference [.default-utils-before_script, before_script]
- source scripts/prepare_build.sh
.production:
variables:
RAILS_ENV: "production"
NODE_ENV: "production"
GITLAB_ALLOW_SEPARATE_CI_DATABASE: "true"
.ruby-gems-cache: &ruby-gems-cache
key: "ruby-gems-debian-${DEBIAN_VERSION}-ruby-${RUBY_VERSION}"
key: "ruby-gems-v1"
paths:
- vendor/ruby/
policy: pull
@ -39,27 +26,18 @@
<<: *ruby-gems-cache
policy: push # We want to rebuild the cache from scratch to ensure stale dependencies are cleaned up.
.gitaly-binaries-cache: &gitaly-binaries-cache
key:
files:
- GITALY_SERVER_VERSION
- lib/gitlab/setup_helper.rb
prefix: "gitaly-binaries-debian-${DEBIAN_VERSION}"
.gitaly-ruby-gems-cache: &gitaly-ruby-gems-cache
key: "gitaly-ruby-gems-v1"
paths:
- ${TMP_TEST_FOLDER}/gitaly/_build/bin/
- ${TMP_TEST_FOLDER}/gitaly/_build/deps/git/install/
- ${TMP_TEST_FOLDER}/gitaly/config.toml
- ${TMP_TEST_FOLDER}/gitaly/gitaly2.config.toml
- ${TMP_TEST_FOLDER}/gitaly/internal/
- ${TMP_TEST_FOLDER}/gitaly/run/
- ${TMP_TEST_FOLDER}/gitaly/run2/
- ${TMP_TEST_FOLDER}/gitaly/Makefile
- ${TMP_TEST_FOLDER}/gitaly/praefect.config.toml
- ${TMP_TEST_FOLDER}/gitaly/praefect-db.config.toml
- vendor/gitaly-ruby/
policy: pull
.gitaly-ruby-gems-cache-push: &gitaly-ruby-gems-cache-push
<<: *gitaly-ruby-gems-cache
policy: push # We want to rebuild the cache from scratch to ensure stale dependencies are cleaned up.
.go-pkg-cache: &go-pkg-cache
key: "go-pkg-${DEBIAN_VERSION}"
key: "go-pkg-v1"
paths:
- .go/pkg/mod/
policy: pull
@ -69,7 +47,7 @@
policy: push # We want to rebuild the cache from scratch to ensure stale dependencies are cleaned up.
.node-modules-cache: &node-modules-cache
key: "node-modules-${DEBIAN_VERSION}-${NODE_ENV}"
key: "node-modules-${NODE_ENV}-v1"
paths:
- node_modules/
- tmp/cache/webpack-dlls/
@ -79,30 +57,22 @@
<<: *node-modules-cache
policy: push # We want to rebuild the cache from scratch to ensure stale dependencies are cleaned up.
.assets-tmp-cache: &assets-tmp-cache
key: "assets-tmp-debian-${DEBIAN_VERSION}-ruby-${RUBY_VERSION}-node-${NODE_ENV}-v1"
.assets-cache: &assets-cache
key: "assets-${NODE_ENV}-v1"
paths:
- assets-hash.txt
- public/assets/webpack/
- tmp/cache/assets/sprockets/
- tmp/cache/babel-loader/
- tmp/cache/vue-loader/
policy: pull
.assets-tmp-cache-push: &assets-tmp-cache-push
<<: *assets-tmp-cache
policy: push # We want to rebuild the cache from scratch to ensure we don't pile up outdated cache files.
.storybook-node-modules-cache: &storybook-node-modules-cache
key: "storybook-node-modules-${DEBIAN_VERSION}-${NODE_ENV}"
paths:
- storybook/node_modules/
policy: pull
.storybook-node-modules-cache-push: &storybook-node-modules-cache-push
<<: *storybook-node-modules-cache
.assets-cache-push: &assets-cache-push
<<: *assets-cache
policy: push # We want to rebuild the cache from scratch to ensure stale dependencies are cleaned up.
.rubocop-cache: &rubocop-cache
key: "rubocop-debian-${DEBIAN_VERSION}-ruby-${RUBY_VERSION}"
key: "rubocop-v1"
paths:
- tmp/rubocop_cache/
policy: pull
@ -111,44 +81,34 @@
<<: *rubocop-cache
# We want to rebuild the cache from scratch to ensure stale dependencies are cleaned up but RuboCop has a mechanism
# for keeping only the N latest cache files, so we take advantage of it with `pull-push`.
policy: push
policy: pull-push
.qa-ruby-gems-cache: &qa-ruby-gems-cache
key:
prefix: "qa-ruby-gems-debian-${DEBIAN_VERSION}-ruby-${RUBY_VERSION}"
files:
- qa/Gemfile.lock
key: "qa-ruby-gems-v1"
paths:
- qa/vendor/ruby
- qa/vendor/ruby/
policy: pull
.qa-ruby-gems-cache-push: &qa-ruby-gems-cache-push
<<: *qa-ruby-gems-cache
policy: pull-push
policy: push # We want to rebuild the cache from scratch to ensure stale dependencies are cleaned up.
.setup-test-env-cache:
cache:
- *ruby-gems-cache
- *gitaly-binaries-cache
- *gitaly-ruby-gems-cache
- *go-pkg-cache
.setup-test-env-cache-push:
cache:
- *ruby-gems-cache-push
- *gitaly-ruby-gems-cache-push
- *go-pkg-cache-push
.gitaly-binaries-cache-push:
cache:
- <<: *gitaly-binaries-cache
policy: push # We want to rebuild the cache from scratch to ensure stale dependencies are cleaned up.
.ruby-cache:
cache:
- *ruby-gems-cache
.rails-cache:
cache:
- *ruby-gems-cache
- *gitaly-ruby-gems-cache
.static-analysis-cache:
cache:
@ -156,12 +116,7 @@
- *node-modules-cache
- *rubocop-cache
.rubocop-job-cache:
cache:
- *ruby-gems-cache
- *rubocop-cache
.rubocop-job-cache-push:
.static-analysis-cache-push:
cache:
- *ruby-gems-cache # We don't push this cache as it's already rebuilt by `update-setup-test-env-cache`
- *rubocop-cache-push
@ -170,24 +125,16 @@
cache:
- *ruby-gems-cache
.ruby-node-cache:
.danger-review-cache:
cache:
- *ruby-gems-cache
- *node-modules-cache
.qa-bundler-variables: &qa-bundler-variables
variables:
BUNDLE_SUPPRESS_INSTALL_USING_MESSAGES: "true"
BUNDLE_SILENCE_ROOT_WARNING: "true"
BUNDLE_PATH: vendor
.qa-cache:
<<: *qa-bundler-variables
cache:
- *qa-ruby-gems-cache
.qa-cache-push:
<<: *qa-bundler-variables
cache:
- *qa-ruby-gems-cache-push
@ -195,211 +142,65 @@
cache:
- *node-modules-cache
.yarn-cache-push:
cache:
- *node-modules-cache-push
.assets-compile-cache:
cache:
- *ruby-gems-cache
- *node-modules-cache
- *assets-tmp-cache
- *assets-cache
.assets-compile-cache-push:
cache:
- *ruby-gems-cache # We don't push this cache as it's already rebuilt by `update-setup-test-env-cache`
- *node-modules-cache-push
- *assets-tmp-cache-push
- *assets-cache-push
.storybook-yarn-cache:
cache:
- *node-modules-cache
- *storybook-node-modules-cache
.storybook-yarn-cache-push:
cache:
- *node-modules-cache # We don't push this cache as it's already rebuilt by `update-assets-compile-*-cache`
- *storybook-node-modules-cache-push
.use-pg11:
image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.7.2.patched-golang-1.14-git-2.31-lfs-2.9-chrome-89-node-14.15-yarn-1.22-postgresql-11-graphicsmagick-1.3.36"
services:
- name: postgres:11.6
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
- name: redis:5.0-alpine
variables:
POSTGRES_HOST_AUTH_METHOD: trust
.use-pg12:
image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.7.2.patched-golang-1.14-git-2.31-lfs-2.9-chrome-89-node-14.15-yarn-1.22-postgresql-12-graphicsmagick-1.3.36"
services:
- name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-12-pgvector-0.4.1
- name: postgres:12
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
alias: postgres
- name: redis:6.0-alpine
- name: redis:5.0-alpine
variables:
POSTGRES_HOST_AUTH_METHOD: trust
PG_VERSION: "12"
.use-pg13:
.use-pg11-ee:
image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.7.2.patched-golang-1.14-git-2.31-lfs-2.9-chrome-89-node-14.15-yarn-1.22-postgresql-11-graphicsmagick-1.3.36"
services:
- name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-13-pgvector-0.4.1
- name: postgres:11.6
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
alias: postgres
- name: redis:6.2-alpine
- name: redis:5.0-alpine
- name: elasticsearch:7.11.1
command: ["elasticsearch", "-E", "discovery.type=single-node"]
variables:
POSTGRES_HOST_AUTH_METHOD: trust
PG_VERSION: "13"
.use-pg14:
.use-pg12-ee:
image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.7.2.patched-golang-1.14-git-2.31-lfs-2.9-chrome-89-node-14.15-yarn-1.22-postgresql-12-graphicsmagick-1.3.36"
services:
- name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-14-pgvector-0.4.1
- name: postgres:12
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
alias: postgres
- name: redis:6.2-alpine
- name: redis:5.0-alpine
- name: elasticsearch:7.11.1
command: ["elasticsearch", "-E", "discovery.type=single-node"]
variables:
POSTGRES_HOST_AUTH_METHOD: trust
PG_VERSION: "14"
.use-pg12-es7-ee:
services:
- name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-12-pgvector-0.4.1
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
alias: postgres
- name: redis:6.0-alpine
- name: elasticsearch:7.17.6
command: ["elasticsearch", "-E", "discovery.type=single-node", "-E", "xpack.security.enabled=false"]
- name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:zoekt-ci-image-1.0
alias: zoekt-ci-image
variables:
POSTGRES_HOST_AUTH_METHOD: trust
PG_VERSION: "12"
ZOEKT_INDEX_BASE_URL: http://zoekt-ci-image:6060
ZOEKT_SEARCH_BASE_URL: http://zoekt-ci-image:6070
.use-pg13-es7-ee:
services:
- name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-13-pgvector-0.4.1
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
alias: postgres
- name: redis:6.2-alpine
- name: elasticsearch:7.17.6
command: ["elasticsearch", "-E", "discovery.type=single-node", "-E", "xpack.security.enabled=false"]
- name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:zoekt-ci-image-1.0
alias: zoekt-ci-image
variables:
POSTGRES_HOST_AUTH_METHOD: trust
PG_VERSION: "13"
ZOEKT_INDEX_BASE_URL: http://zoekt-ci-image:6060
ZOEKT_SEARCH_BASE_URL: http://zoekt-ci-image:6070
.use-pg14-es7-ee:
services:
- name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-14-pgvector-0.4.1
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
alias: postgres
- name: redis:6.2-alpine
- name: elasticsearch:7.17.6
command: ["elasticsearch", "-E", "discovery.type=single-node", "-E", "xpack.security.enabled=false"]
- name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:zoekt-ci-image-1.0
alias: zoekt-ci-image
variables:
POSTGRES_HOST_AUTH_METHOD: trust
PG_VERSION: "14"
ZOEKT_INDEX_BASE_URL: http://zoekt-ci-image:6060
ZOEKT_SEARCH_BASE_URL: http://zoekt-ci-image:6070
.use-pg13-es8-ee:
services:
- name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-13-pgvector-0.4.1
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
alias: postgres
- name: redis:6.0-alpine
- name: elasticsearch:8.6.2
- name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:zoekt-ci-image-1.0
alias: zoekt-ci-image
variables:
POSTGRES_HOST_AUTH_METHOD: trust
PG_VERSION: "13"
ES_SETTING_DISCOVERY_TYPE: "single-node"
ES_SETTING_XPACK_SECURITY_ENABLED: "false"
ZOEKT_INDEX_BASE_URL: http://zoekt-ci-image:6060
ZOEKT_SEARCH_BASE_URL: http://zoekt-ci-image:6070
.use-pg14-es8-ee:
services:
- name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-14-pgvector-0.4.1
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
alias: postgres
- name: redis:6.0-alpine
- name: elasticsearch:8.6.2
- name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:zoekt-ci-image-1.0
alias: zoekt-ci-image
variables:
POSTGRES_HOST_AUTH_METHOD: trust
PG_VERSION: "14"
ES_SETTING_DISCOVERY_TYPE: "single-node"
ES_SETTING_XPACK_SECURITY_ENABLED: "false"
ZOEKT_INDEX_BASE_URL: http://zoekt-ci-image:6060
ZOEKT_SEARCH_BASE_URL: http://zoekt-ci-image:6070
.use-pg13-opensearch1-ee:
services:
- name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-13-pgvector-0.4.1
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
alias: postgres
- name: redis:6.0-alpine
- name: opensearchproject/opensearch:1.3.5
alias: elasticsearch
command: ["bin/opensearch", "-E", "discovery.type=single-node", "-E", "plugins.security.disabled=true"]
- name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:zoekt-ci-image-1.0
alias: zoekt-ci-image
variables:
POSTGRES_HOST_AUTH_METHOD: trust
PG_VERSION: "13"
ZOEKT_INDEX_BASE_URL: http://zoekt-ci-image:6060
ZOEKT_SEARCH_BASE_URL: http://zoekt-ci-image:6070
.use-pg13-opensearch2-ee:
services:
- name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-13-pgvector-0.4.1
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
alias: postgres
- name: redis:6.0-alpine
- name: opensearchproject/opensearch:2.2.1
alias: elasticsearch
command: ["bin/opensearch", "-E", "discovery.type=single-node", "-E", "plugins.security.disabled=true"]
- name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:zoekt-ci-image-1.0
alias: zoekt-ci-image
variables:
POSTGRES_HOST_AUTH_METHOD: trust
PG_VERSION: "13"
ZOEKT_INDEX_BASE_URL: http://zoekt-ci-image:6060
ZOEKT_SEARCH_BASE_URL: http://zoekt-ci-image:6070
.use-pg14-opensearch1-ee:
services:
- name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-14-pgvector-0.4.1
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
alias: postgres
- name: redis:6.0-alpine
- name: opensearchproject/opensearch:1.3.5
alias: elasticsearch
command: ["bin/opensearch", "-E", "discovery.type=single-node", "-E", "plugins.security.disabled=true"]
- name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:zoekt-ci-image-1.0
alias: zoekt-ci-image
variables:
POSTGRES_HOST_AUTH_METHOD: trust
PG_VERSION: "14"
ZOEKT_INDEX_BASE_URL: http://zoekt-ci-image:6060
ZOEKT_SEARCH_BASE_URL: http://zoekt-ci-image:6070
.use-pg14-opensearch2-ee:
services:
- name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-14-pgvector-0.4.1
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
alias: postgres
- name: redis:6.0-alpine
- name: opensearchproject/opensearch:2.2.1
alias: elasticsearch
command: ["bin/opensearch", "-E", "discovery.type=single-node", "-E", "plugins.security.disabled=true"]
- name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:zoekt-ci-image-1.0
alias: zoekt-ci-image
variables:
POSTGRES_HOST_AUTH_METHOD: trust
PG_VERSION: "14"
ZOEKT_INDEX_BASE_URL: http://zoekt-ci-image:6060
ZOEKT_SEARCH_BASE_URL: http://zoekt-ci-image:6070
.use-kaniko:
image:
name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:kaniko
name: registry.gitlab.com/gitlab-org/gitlab-build-images:kaniko
entrypoint: [""]
before_script:
- source scripts/utils.sh
@ -411,7 +212,7 @@
FOSS_ONLY: '1'
.use-docker-in-docker:
image: ${GITLAB_DEPENDENCY_PROXY_ADDRESS}docker:${DOCKER_VERSION}
image: ${GITLAB_DEPENDENCY_PROXY}docker:${DOCKER_VERSION}
services:
- docker:${DOCKER_VERSION}-dind
variables:
@ -421,25 +222,3 @@
tags:
# See https://gitlab.com/gitlab-com/www-gitlab-com/-/issues/7019 for tag descriptions
- gitlab-org-docker
.use-buildx:
extends: .use-docker-in-docker
image: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images/debian-bullseye-slim:docker-${DOCKER_VERSION}
variables:
QEMU_IMAGE: tonistiigi/binfmt:qemu-v7.0.0
before_script:
- !reference [.default-utils-before_script, before_script]
- echo "$CI_REGISTRY_PASSWORD" | docker login "$CI_REGISTRY" -u "$CI_REGISTRY_USER" --password-stdin
- |
if [[ "${ARCH}" =~ arm64 ]]; then
echo -e "\033[1;33mInstalling latest qemu emulators\033[0m"
docker pull -q ${QEMU_IMAGE};
docker run --rm --privileged ${QEMU_IMAGE} --uninstall qemu-*;
docker run --rm --privileged ${QEMU_IMAGE} --install all;
fi
- docker buildx create --use # creates and set's to active buildkit builder
.use-kube-context:
before_script:
- export KUBE_CONTEXT="gitlab-org/gitlab:review-apps"
- kubectl config use-context ${KUBE_CONTEXT}

View file

@ -11,5 +11,3 @@ graphql-verify:
script:
- bundle exec rake gitlab:graphql:validate
- bundle exec rake gitlab:graphql:check_docs
- bundle exec rake gitlab:graphql:schema:dump
- node scripts/frontend/graphql_possible_types_extraction.js --check

View file

@ -4,38 +4,52 @@
- .rails-cache
- .default-before_script
- .memory:rules
variables:
METRICS_FILE: "metrics.txt"
artifacts:
reports:
metrics: "${METRICS_FILE}"
expire_in: 62d
memory-static:
extends: .only-code-memory-job-base
stage: test
needs: ["setup-test-env"]
variables:
SETUP_DB: "false"
script:
# Uses two different reports from the 'derailed_benchmars' gem.
# Loads each of gems in the Gemfile and checks how much memory they consume when they are required.
# 'derailed_benchmarks' internally uses 'get_process_mem'
- bundle exec derailed bundle:mem > tmp/memory_bundle_mem.txt
- scripts/generate-gems-size-metrics-static tmp/memory_bundle_mem.txt >> 'tmp/memory_metrics.txt'
# Outputs detailed information about objects created while gems are loaded.
# 'derailed_benchmarks' internally uses 'memory_profiler'
- bundle exec derailed bundle:objects > tmp/memory_bundle_objects.txt
- scripts/generate-gems-memory-metrics-static tmp/memory_bundle_objects.txt >> 'tmp/memory_metrics.txt'
artifacts:
paths:
- tmp/memory_*.txt
reports:
metrics: tmp/memory_metrics.txt
expire_in: 31d
# Show memory usage caused by invoking require per gem.
# Hits the app with one request to ensure that any last minute require-s have been called.
# Unlike `memory-static`, it hits the app with one request to ensure that any last minute require-s have been called.
# The application is booted in `production` environment.
# All tests are run without a webserver (directly using Rack::Mock by default).
memory-on-boot:
extends:
- .only-code-memory-job-base
- .production
- .use-pg13
- .use-pg12
stage: test
needs: ["setup-test-env", "compile-test-assets"]
variables:
NODE_ENV: "production"
RAILS_ENV: "production"
SETUP_DB: "true"
MEMORY_ON_BOOT_FILE_PREFIX: "tmp/memory_on_boot_"
TEST_COUNT: 5
script:
- |
for i in $(seq 1 $TEST_COUNT)
do
echo "Starting run $i out of $TEST_COUNT"
PATH_TO_HIT="/users/sign_in" CUT_OFF=0.3 bundle exec derailed exec perf:mem >> "${MEMORY_ON_BOOT_FILE_PREFIX}$i.txt"
done
- scripts/generate-memory-metrics-on-boot "${MEMORY_ON_BOOT_FILE_PREFIX}" "$TEST_COUNT" >> "${METRICS_FILE}"
- PATH_TO_HIT="/users/sign_in" CUT_OFF=0.3 bundle exec derailed exec perf:mem >> 'tmp/memory_on_boot.txt'
- scripts/generate-memory-metrics-on-boot tmp/memory_on_boot.txt >> 'tmp/memory_on_boot_metrics.txt'
artifacts:
paths:
- "${METRICS_FILE}"
- "${MEMORY_ON_BOOT_FILE_PREFIX}*.txt"
- tmp/memory_*.txt
reports:
metrics: tmp/memory_on_boot_metrics.txt
expire_in: 31d

View file

@ -1,41 +1,36 @@
.notify-defaults:
.notify-slack:
image: ${GITLAB_DEPENDENCY_PROXY}alpine
stage: notify
dependencies: []
cache: {}
create-issues-for-failing-tests:
extends:
- .notify-defaults
- .notify:rules:create-issues-for-failing-tests
image: ${GITLAB_DEPENDENCY_PROXY_ADDRESS}ruby:${RUBY_VERSION}
variables:
FAILED_TESTS_DIR: "${CI_PROJECT_DIR}/tmp/failed_tests"
FAILING_ISSUES_PROJECT: "gitlab-org/quality/engineering-productivity/flaky-tests-playground"
FAILING_ISSUE_JSON_DIR: "${CI_PROJECT_DIR}/tmp/issues"
MERGE_REQUEST_URL: ${CI_MERGE_REQUEST_PROJECT_URL}/-/merge_requests/${CI_MERGE_REQUEST_IID}
before_script:
- source ./scripts/utils.sh
- source ./scripts/rspec_helpers.sh
- install_gitlab_gem
script:
- mkdir -p "${FAILING_ISSUE_JSON_DIR}"
- retrieve_failed_tests "${FAILED_TESTS_DIR}" "json" "latest"
- scripts/pipeline/create_test_failure_issues.rb --project "${FAILING_ISSUES_PROJECT}" --tests-report-file "${FAILED_TESTS_DIR}/rspec_failed_tests.json" --issues-json-folder "${FAILING_ISSUE_JSON_DIR}" --api-token "${FAILING_ISSUES_PROJECT_TOKEN}"
- scripts/pipeline/create_test_failure_issues.rb --project "${FAILING_ISSUES_PROJECT}" --tests-report-file "${FAILED_TESTS_DIR}/rspec_ee_failed_tests.json" --issues-json-folder "${FAILING_ISSUE_JSON_DIR}" --api-token "${FAILING_ISSUES_PROJECT_TOKEN}"
artifacts:
paths:
- ${FAILED_TESTS_DIR}/
- ${FAILING_ISSUE_JSON_DIR}/
when: always
expire_in: 2 days
- apk update && apk add git curl bash
notify-package-and-test-failure:
notify-update-gitaly:
extends:
- .notify-defaults
- .notify:rules:notify-package-and-test-failure
image: ${GITLAB_DEPENDENCY_PROXY_ADDRESS}ruby:${RUBY_VERSION}
before_script:
- source scripts/utils.sh
- apt-get update
- install_gitlab_gem
- .notify-slack
rules:
- if: '$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME == $GITALY_UPDATE_BRANCH'
when: on_failure
allow_failure: true
variables:
NOTIFY_CHANNEL: g_create_gitaly
GITALY_UPDATE_BRANCH: release-tools/update-gitaly
script:
- scripts/generate-failed-package-and-test-mr-message.rb
- echo "NOTIFY_CHANNEL is ${NOTIFY_CHANNEL}"
- echo "CI_PIPELINE_URL is ${CI_PIPELINE_URL}"
- scripts/slack ${NOTIFY_CHANNEL} "☠️ \`${GITALY_UPDATE_BRANCH}\` failed! ☠️ See ${CI_PIPELINE_URL} (triggered from ${MERGE_REQUEST_URL})" ci_failing "GitLab QA Bot"
notify-security-pipeline:
extends:
- .notify-slack
- .delivery:rules:security-pipeline-merge-result-failure
variables:
NOTIFY_CHANNEL: f_upcoming_release
script:
- echo "NOTIFY_CHANNEL is ${NOTIFY_CHANNEL}"
- echo "CI_PIPELINE_URL is ${CI_PIPELINE_URL}"
# <!subteam^S0127FU8PDE> mentions the `@release-managers` group
- scripts/slack ${NOTIFY_CHANNEL} "<!subteam^S0127FU8PDE> ☠️ Pipeline for merged result failed! ☠️ See ${CI_PIPELINE_URL} (triggered from ${MERGE_REQUEST_URL})" ci_failing "GitLab Release Tools Bot"

View file

@ -1,87 +0,0 @@
include:
- local: .gitlab/ci/qa-common/main.gitlab-ci.yml
- local: .gitlab/ci/qa-common/rules.gitlab-ci.yml
- local: .gitlab/ci/qa-common/variables.gitlab-ci.yml
workflow:
rules:
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_PIPELINE_SOURCE == "schedule" && $SCHEDULE_TYPE == "nightly"'
.ce:
variables:
RELEASE: ${REGISTRY_HOST}/${REGISTRY_GROUP}/build/omnibus-gitlab-mirror/gitlab-ce:${CI_COMMIT_SHA}
.ee:
variables:
RELEASE: ${REGISTRY_HOST}/${REGISTRY_GROUP}/build/omnibus-gitlab-mirror/gitlab-ee:${CI_COMMIT_SHA}
# ==========================================
# Prepare stage
# ==========================================
# TODO: enable once ee jobs are added
# trigger-omnibus-env:
# extends:
# - .trigger-omnibus-env
trigger-omnibus-env-ce:
extends:
- .trigger-omnibus-env-ce
variables:
FOSS_ONLY: "1" # set FOSS_ONLY because we don't pass it via trigger job
# TODO: enable once ee jobs are added
# trigger-omnibus:
# extends:
# - .trigger-omnibus
# needs:
# - trigger-omnibus-env
trigger-omnibus-ce:
extends:
- .trigger-omnibus-ce
needs:
- trigger-omnibus-env-ce
# TODO: enable when first parallel job is added
# download-knapsack-report:
# extends:
# - .download-knapsack-report
# - .rules:download-knapsack
# ==========================================
# Test stage
# ==========================================
update-ee-to-ce:
extends:
- .qa
- .update-script
- .ce
variables:
UPDATE_TYPE: minor
UPDATE_FROM_EDITION: ee
QA_RSPEC_TAGS: --tag smoke
# ==========================================
# Post test stage
# ==========================================
e2e-test-report:
extends:
- .e2e-test-report
# TODO: enable when first parallel job is added
# upload-knapsack-report:
# extends:
# - .upload-knapsack-report
# - .rules:report:process-results
export-test-metrics:
extends:
- .export-test-metrics
relate-test-failures:
extends:
- .relate-test-failures
notify-slack:
extends:
- .notify-slack

View file

@ -1,525 +0,0 @@
# E2E tests pipeline loaded dynamically by script: scripts/generate-e2e-pipeline
# For adding new tests, refer to: doc/development/testing_guide/end_to_end/package_and_test_pipeline.md
include:
- local: .gitlab/ci/qa-common/main.gitlab-ci.yml
- local: .gitlab/ci/qa-common/rules.gitlab-ci.yml
- local: .gitlab/ci/qa-common/variables.gitlab-ci.yml
# ==========================================
# Prepare stage
# ==========================================
check-release-set:
extends: .rules:prepare
stage: .pre
script:
- |
if [ -z "$RELEASE" ]; then
echo "E2E test pipeline requires omnibus installation docker image to be set via $RELEASE environment variable"
exit 1
else
echo "Omnibus installation image is set to '$RELEASE'"
fi
trigger-omnibus-env:
extends:
- .trigger-omnibus-env
- .rules:omnibus-build
trigger-omnibus-env-ce:
extends:
- .trigger-omnibus-env-ce
- .rules:omnibus-build-ce
trigger-omnibus:
extends:
- .trigger-omnibus
- .rules:omnibus-build
needs:
- trigger-omnibus-env
trigger-omnibus-ce:
extends:
- .trigger-omnibus-ce
- .rules:omnibus-build-ce
needs:
- trigger-omnibus-env-ce
download-knapsack-report:
extends:
- .download-knapsack-report
- .rules:download-knapsack
cache-gems:
extends:
- .qa-install
- .ruby-image
- .rules:update-cache
stage: .pre
tags:
- e2e
script:
- echo "Populated qa cache"
cache:
policy: pull-push
# ==========================================
# Test stage
# ==========================================
# ------------------------------------------
# Manual jobs
# ------------------------------------------
# Run manual quarantine job
# this job requires passing QA_SCENARIO variable
# and optionally QA_TESTS to run specific quarantined tests
_quarantine:
extends:
- .qa
- .rules:test:manual
needs:
- job: trigger-omnibus
optional: true
stage: test
variables:
QA_RSPEC_TAGS: --tag quarantine
# ------------------------------------------
# FF changes
# ------------------------------------------
# Run specs with feature flags set to the opposite of the default state
instance-ff-inverse:
extends:
- .qa
- .parallel
variables:
QA_SCENARIO: Test::Instance::Image
QA_KNAPSACK_REPORT_NAME: ee-instance
GITLAB_QA_OPTS: --set-feature-flags $QA_FEATURE_FLAGS
rules:
- !reference [.rules:test:feature-flags-set, rules]
# ------------------------------------------
# Jobs with parallel variant
# ------------------------------------------
instance-selective:
extends: .qa
variables:
QA_SCENARIO: Test::Instance::Image
rules:
- !reference [.rules:test:qa-selective, rules]
- if: $QA_SUITES =~ /Test::Instance::All/
instance:
extends:
- .parallel
- instance-selective
rules:
- !reference [.rules:test:feature-flags-set, rules] # always run instance to validate ff change
- !reference [.rules:test:qa-parallel, rules]
- if: $QA_SUITES =~ /Test::Instance::All/
praefect-selective:
extends: .qa
variables:
QA_SCENARIO: Test::Integration::Praefect
QA_CAN_TEST_PRAEFECT: "true"
rules:
- !reference [.rules:test:qa-selective, rules]
- if: $QA_SUITES =~ /Test::Instance::All/
praefect:
extends:
- .parallel
- praefect-selective
rules:
- !reference [.rules:test:qa-parallel, rules]
- if: $QA_SUITES =~ /Test::Instance::All/
relative-url-selective:
extends: .qa
variables:
QA_SCENARIO: Test::Instance::RelativeUrl
rules:
- !reference [.rules:test:qa-selective, rules]
- if: $QA_SUITES =~ /Test::Instance::All/
relative-url:
extends:
- .parallel
- relative-url-selective
rules:
- !reference [.rules:test:qa-parallel, rules]
- if: $QA_SUITES =~ /Test::Instance::All/
decomposition-single-db-selective:
extends: .qa
variables:
QA_SCENARIO: Test::Instance::Image
GITLAB_QA_OPTS: --omnibus-config decomposition_single_db $EXTRA_GITLAB_QA_OPTS
rules:
- !reference [.rules:test:qa-selective, rules]
- if: $QA_SUITES =~ /Test::Instance::All/
decomposition-single-db:
extends:
- .parallel
- decomposition-single-db-selective
rules:
- !reference [.rules:test:qa-parallel, rules]
- if: $QA_SUITES =~ /Test::Instance::All/
decomposition-multiple-db-selective:
extends: .qa
variables:
QA_SCENARIO: Test::Instance::Image
GITLAB_ALLOW_SEPARATE_CI_DATABASE: "true"
GITLAB_QA_OPTS: --omnibus-config decomposition_multiple_db $EXTRA_GITLAB_QA_OPTS
rules:
- !reference [.rules:test:qa-selective, rules]
- if: $QA_SUITES =~ /Test::Instance::All/
decomposition-multiple-db:
extends:
- .parallel
- decomposition-multiple-db-selective
rules:
- !reference [.rules:test:qa-parallel, rules]
- if: $QA_SUITES =~ /Test::Instance::All/
object-storage-selective:
extends: .qa
variables:
QA_SCENARIO: Test::Instance::Image
QA_RSPEC_TAGS: --tag object_storage
GITLAB_QA_OPTS: --omnibus-config object_storage $EXTRA_GITLAB_QA_OPTS
rules:
- !reference [.rules:test:qa-selective, rules]
- if: $QA_SUITES =~ /Test::Instance::ObjectStorage/
object-storage:
extends: object-storage-selective
parallel: 2
rules:
- !reference [.rules:test:qa-parallel, rules]
- if: $QA_SUITES =~ /Test::Instance::ObjectStorage/
object-storage-aws-selective:
extends: object-storage-selective
variables:
AWS_S3_ACCESS_KEY: $QA_AWS_S3_ACCESS_KEY
AWS_S3_BUCKET_NAME: $QA_AWS_S3_BUCKET_NAME
AWS_S3_KEY_ID: $QA_AWS_S3_KEY_ID
AWS_S3_REGION: $QA_AWS_S3_REGION
GITLAB_QA_OPTS: --omnibus-config object_storage_aws $EXTRA_GITLAB_QA_OPTS
object-storage-aws:
extends: object-storage-aws-selective
parallel: 2
rules:
- !reference [object-storage, rules]
object-storage-gcs-selective:
extends: object-storage-selective
variables:
GCS_BUCKET_NAME: $QA_GCS_BUCKET_NAME
GOOGLE_PROJECT: $QA_GOOGLE_PROJECT
GOOGLE_JSON_KEY: $QA_GOOGLE_JSON_KEY
GOOGLE_CLIENT_EMAIL: $QA_GOOGLE_CLIENT_EMAIL
GITLAB_QA_OPTS: --omnibus-config object_storage_gcs $EXTRA_GITLAB_QA_OPTS
object-storage-gcs:
extends: object-storage-gcs-selective
parallel: 2
rules:
- !reference [object-storage, rules]
packages-selective:
extends: .qa
variables:
QA_SCENARIO: Test::Instance::Image
QA_RSPEC_TAGS: --tag packages
GITLAB_QA_OPTS: --omnibus-config packages $EXTRA_GITLAB_QA_OPTS
rules:
- !reference [.rules:test:qa-selective, rules]
- if: $QA_SUITES =~ /Test::Instance::Packages/
packages:
extends: packages-selective
parallel: 2
rules:
- !reference [.rules:test:qa-parallel, rules]
- if: $QA_SUITES =~ /Test::Instance::Packages/
# ------------------------------------------
# Non parallel jobs
# ------------------------------------------
update-minor:
extends:
- .qa
- .update-script
variables:
UPDATE_TYPE: minor
QA_RSPEC_TAGS: --tag smoke
rules:
- !reference [.rules:test:update, rules]
- if: $QA_SUITES =~ /Test::Instance::Smoke/
- !reference [.rules:test:manual, rules]
update-major:
extends:
- .qa
- .update-script
variables:
UPDATE_TYPE: major
QA_RSPEC_TAGS: --tag smoke
rules:
- !reference [.rules:test:update, rules]
- if: $QA_SUITES =~ /Test::Instance::Smoke/
- !reference [.rules:test:manual, rules]
gitlab-pages:
extends: .qa
variables:
QA_SCENARIO: Test::Integration::GitlabPages
rules:
- !reference [.rules:test:qa, rules]
- if: $QA_SUITES =~ /Test::Instance::GitlabPages/
- !reference [.rules:test:manual, rules]
gitaly-cluster:
extends: .qa
variables:
QA_SCENARIO: Test::Integration::GitalyCluster
rules:
- !reference [.rules:test:qa, rules]
- if: $QA_SUITES =~ /Test::Integration::GitalyCluster/
- !reference [.rules:test:manual, rules]
group-saml:
extends: .qa
variables:
QA_SCENARIO: Test::Integration::GroupSAML
rules:
- !reference [.rules:test:ee-only, rules]
- !reference [.rules:test:qa, rules]
- if: $QA_SUITES =~ /Test::Integration::GroupSAML/
- !reference [.rules:test:manual, rules]
oauth:
extends: .qa
variables:
QA_SCENARIO: Test::Integration::OAuth
rules:
- !reference [.rules:test:qa-default-branch, rules]
- if: $QA_SUITES =~ /Test::Integration::OAuth/
- !reference [.rules:test:manual, rules]
instance-saml:
extends: .qa
variables:
QA_SCENARIO: Test::Integration::InstanceSAML
rules:
- !reference [.rules:test:qa, rules]
- if: $QA_SUITES =~ /Test::Integration::InstanceSAML/
- !reference [.rules:test:manual, rules]
jira:
extends: .qa
variables:
QA_SCENARIO: Test::Integration::Jira
rules:
- !reference [.rules:test:qa, rules]
- if: $QA_SUITES =~ /Test::Integration::Jira/
- !reference [.rules:test:manual, rules]
integrations:
extends: .qa
variables:
QA_SCENARIO: Test::Integration::Integrations
rules:
- !reference [.rules:test:qa, rules]
- if: $QA_SUITES =~ /Test::Integration::Integrations/
- !reference [.rules:test:manual, rules]
ldap-no-server:
extends: .qa
variables:
QA_SCENARIO: Test::Integration::LDAPNoServer
rules:
- !reference [.rules:test:qa, rules]
- if: $QA_SUITES =~ /Test::Integration::LDAPNoServer/
- !reference [.rules:test:manual, rules]
ldap-tls:
extends: .qa
variables:
QA_SCENARIO: Test::Integration::LDAPTLS
rules:
- !reference [.rules:test:qa, rules]
- if: $QA_SUITES =~ /Test::Integration::LDAPTLS/
- !reference [.rules:test:manual, rules]
ldap-no-tls:
extends: .qa
variables:
QA_SCENARIO: Test::Integration::LDAPNoTLS
rules:
- !reference [.rules:test:qa, rules]
- if: $QA_SUITES =~ /Test::Integration::LDAPNoTLS/
- !reference [.rules:test:manual, rules]
mtls:
extends: .qa
variables:
QA_SCENARIO: Test::Integration::MTLS
rules:
- !reference [.rules:test:qa, rules]
- if: $QA_SUITES =~ /Test::Integration::Mtls/
- !reference [.rules:test:manual, rules]
mattermost:
extends: .qa
variables:
QA_SCENARIO: Test::Integration::Mattermost
rules:
- !reference [.rules:test:qa, rules]
- if: $QA_SUITES =~ /Test::Integration::Mattermost/
- !reference [.rules:test:manual, rules]
registry:
extends: .qa
variables:
QA_SCENARIO: Test::Integration::Registry
rules:
- !reference [.rules:test:qa, rules]
- if: $QA_SUITES =~ /Test::Integration::Registry/
- !reference [.rules:test:manual, rules]
registry-with-cdn:
extends: .qa
variables:
QA_SCENARIO: Test::Integration::RegistryWithCDN
GCS_CDN_BUCKET_NAME: $QA_GCS_CDN_BUCKET_NAME
GOOGLE_CDN_LB: $QA_GOOGLE_CDN_LB
GOOGLE_CDN_JSON_KEY: $QA_GOOGLE_CDN_JSON_KEY
GOOGLE_CDN_SIGNURL_KEY: $QA_GOOGLE_CDN_SIGNURL_KEY
GOOGLE_CDN_SIGNURL_KEY_NAME: $QA_GOOGLE_CDN_SIGNURL_KEY_NAME
before_script:
- unset GITLAB_QA_ADMIN_ACCESS_TOKEN
- !reference [.qa, before_script]
rules:
- !reference [.rules:test:qa, rules]
- if: $QA_SUITES =~ /Test::Integration::RegistryWithCDN/
- !reference [.rules:test:manual, rules]
repository-storage:
extends: .qa
variables:
QA_SCENARIO: Test::Instance::RepositoryStorage
rules:
- !reference [.rules:test:qa, rules]
- if: $QA_SUITES =~ /Test::Instance::RepositoryStorage/
- !reference [.rules:test:manual, rules]
service-ping-disabled:
extends: .qa
variables:
QA_SCENARIO: Test::Integration::ServicePingDisabled
rules:
- !reference [.rules:test:qa, rules]
- if: $QA_SUITES =~ /Test::Integration::ServicePingDisabled/
- !reference [.rules:test:manual, rules]
smtp:
extends: .qa
variables:
QA_SCENARIO: Test::Integration::SMTP
rules:
- !reference [.rules:test:qa, rules]
- if: $QA_SUITES =~ /Test::Integration::SMTP/
- !reference [.rules:test:manual, rules]
cloud-activation:
extends: .qa
variables:
QA_SCENARIO: Test::Instance::Image
QA_RSPEC_TAGS: --tag cloud_activation
rules:
- !reference [.rules:test:qa, rules]
- if: $QA_SUITES =~ /Test::Instance::CloudActivation/
- !reference [.rules:test:manual, rules]
large-setup:
extends: .qa
variables:
QA_SCENARIO: Test::Instance::Image
QA_RSPEC_TAGS: --tag can_use_large_setup
rules:
- !reference [.rules:test:qa, rules]
- if: $QA_SUITES =~ /Test::Instance::LargeSetup/
- !reference [.rules:test:manual, rules]
metrics:
extends: .qa
variables:
QA_SCENARIO: Test::Integration::Metrics
rules:
- !reference [.rules:test:qa, rules]
- if: $QA_SUITES =~ /Test::Instance::Metrics/
- !reference [.rules:test:manual, rules]
elasticsearch:
extends: .qa
variables:
QA_SCENARIO: "Test::Integration::Elasticsearch"
before_script:
- !reference [.qa, before_script]
rules:
- !reference [.rules:test:ee-only, rules]
- !reference [.rules:test:qa, rules]
- if: $QA_SUITES =~ /Test::Integration::Elasticsearch/
- !reference [.rules:test:manual, rules]
registry-object-storage-tls:
extends: object-storage-aws-selective
variables:
QA_SCENARIO: Test::Integration::RegistryTLS
QA_RSPEC_TAGS: ""
GITLAB_TLS_CERTIFICATE: $QA_GITLAB_TLS_CERTIFICATE
GITLAB_QA_OPTS: --omnibus-config registry_object_storage $EXTRA_GITLAB_QA_OPTS
importers:
extends: .qa
variables:
QA_SCENARIO: Test::Integration::Import
QA_MOCK_GITHUB: "true"
rules:
- !reference [.rules:test:qa, rules]
- if: $QA_SUITES =~ /Test::Integration::Import/
- !reference [.rules:test:manual, rules]
# ==========================================
# Post test stage
# ==========================================
e2e-test-report:
extends:
- .e2e-test-report
- .rules:report:allure-report
upload-knapsack-report:
extends:
- .upload-knapsack-report
- .rules:report:process-results
export-test-metrics:
extends:
- .export-test-metrics
- .rules:report:process-results
relate-test-failures:
extends:
- .relate-test-failures
- .rules:report:process-results
generate-test-session:
extends:
- .generate-test-session
- .rules:report:process-results
notify-slack:
extends:
- .notify-slack
- .rules:report:process-results

View file

@ -1,36 +1,21 @@
.compress-public: &compress-public
- find public -type f -regex '.*\.\(htm\|html\|txt\|text\|js\|json\|css\|svg\|xml\)$' -exec gzip -f -k {} \;
- find public -type f -regex '.*\.\(htm\|html\|txt\|text\|js\|json\|css\|svg\|xml\)$' -exec brotli -f -k {} \;
pages:
extends:
- .default-retry
- .pages:rules
stage: pages
environment: pages
resource_group: pages
needs:
- "rspec:coverage"
- "coverage-frontend"
- "compile-production-assets"
- "compile-storybook"
- "update-tests-metadata"
- "generate-frontend-fixtures-mapping"
before_script:
- apt-get update && apt-get -y install brotli gzip
- rspec:coverage
- coverage-frontend
- karma
- compile-production-assets
script:
- mv public/ .public/
- mkdir public/
- mkdir -p public/$(dirname "$KNAPSACK_RSPEC_SUITE_REPORT_PATH") public/$(dirname "$FLAKY_RSPEC_SUITE_REPORT_PATH") public/$(dirname "$RSPEC_PACKED_TESTS_MAPPING_PATH") public/$(dirname "$FRONTEND_FIXTURES_MAPPING_PATH")
- mv coverage/ public/coverage-ruby/ || true
- mv coverage-frontend/ public/coverage-frontend/ || true
- mv storybook/public public/storybook || true
- mv coverage-javascript/ public/coverage-javascript/ || true
- cp .public/assets/application-*.css public/application.css || true
- mv $KNAPSACK_RSPEC_SUITE_REPORT_PATH public/$KNAPSACK_RSPEC_SUITE_REPORT_PATH || true
- mv $FLAKY_RSPEC_SUITE_REPORT_PATH public/$FLAKY_RSPEC_SUITE_REPORT_PATH || true
- mv $RSPEC_PACKED_TESTS_MAPPING_PATH.gz public/$RSPEC_PACKED_TESTS_MAPPING_PATH.gz || true
- mv $FRONTEND_FIXTURES_MAPPING_PATH public/$FRONTEND_FIXTURES_MAPPING_PATH || true
- *compress-public
- cp .public/assets/application-*.css.gz public/application.css.gz || true
artifacts:
paths:
- public

View file

@ -1,66 +0,0 @@
.preflight-job-base:
stage: preflight
extends:
- .default-retry
needs: []
.qa-preflight-job:
image: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images/debian-bullseye-ruby-${RUBY_VERSION}:bundler-2.3-chrome-${CHROME_VERSION}-docker-${DOCKER_VERSION}
extends:
- .preflight-job-base
- .qa-cache
variables:
USE_BUNDLE_INSTALL: "false"
SETUP_DB: "false"
before_script:
- !reference [.default-before_script, before_script]
- cd qa && bundle install
rails-production-server-boot:
extends:
- .preflight-job-base
- .default-before_script
- .production
- .ruby-cache
- .setup:rules:rails-production-server-boot
- .use-pg13
variables:
BUNDLE_WITHOUT: "development:test"
BUNDLE_WITH: "production"
needs: []
script:
- source scripts/utils.sh
- cp config/puma.rb.example config/puma.rb
- sed --in-place "s:/home/git/gitlab:${PWD}:" config/puma.rb
- echo 'bind "tcp://127.0.0.1:3000"' >> config/puma.rb
- bundle exec puma --environment production --config config/puma.rb &
- sleep 40 # See https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114124#note_1309506358
- retry_times_sleep 10 5 "curl http://127.0.0.1:3000"
- kill $(jobs -p)
no-ee-check:
extends:
- .preflight-job-base
- .setup:rules:no-ee-check
script:
- scripts/no-dir-check ee
no-jh-check:
extends:
- .preflight-job-base
- .setup:rules:no-jh-check
script:
- scripts/no-dir-check jh
qa:selectors:
extends:
- .qa-preflight-job
- .qa:rules:ee-and-foss
script:
- bundle exec bin/qa Test::Sanity::Selectors
qa:selectors-as-if-foss:
extends:
- qa:selectors
- .qa:rules:as-if-foss
- .as-if-foss

View file

@ -1,280 +0,0 @@
default:
interruptible: true
workflow:
name: $PIPELINE_NAME
include:
- project: gitlab-org/quality/pipeline-common
ref: 5.1.1
file:
- /ci/base.gitlab-ci.yml
- /ci/allure-report.yml
- /ci/knapsack-report.yml
stages:
- test
- report
- notify
# ==========================================
# Templates
# ==========================================
.parallel:
parallel: 5
variables:
QA_KNAPSACK_REPORT_PATH: $CI_PROJECT_DIR/qa/knapsack
.ruby-image:
# Because this pipeline template can be included directly in other projects,
# image path and registry needs to be defined explicitly
image: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images/debian-bullseye-ruby-${RUBY_VERSION}:bundler-2.3
.qa-install:
variables:
BUNDLE_SUPPRESS_INSTALL_USING_MESSAGES: "true"
BUNDLE_SILENCE_ROOT_WARNING: "true"
extends:
- .gitlab-qa-install
.update-script:
script:
- !reference [.bundle-prefix]
- export QA_COMMAND="$BUNDLE_PREFIX gitlab-qa Test::Omnibus::UpdateFromPrevious $RELEASE $GITLAB_SEMVER_VERSION $UPDATE_TYPE $UPDATE_FROM_EDITION -- $QA_RSPEC_TAGS $RSPEC_REPORT_OPTS"
- echo "Running - '$QA_COMMAND'"
- eval "$QA_COMMAND"
.qa:
extends:
- .qa-base
- .qa-install
- .gitlab-qa-report
stage: test
tags:
- e2e
variables:
QA_GENERATE_ALLURE_REPORT: "true"
QA_CAN_TEST_PRAEFECT: "false"
QA_INTERCEPT_REQUESTS: "true"
GITLAB_LICENSE_MODE: test
GITLAB_QA_ADMIN_ACCESS_TOKEN: $QA_ADMIN_ACCESS_TOKEN
GITLAB_QA_OPTS: $EXTRA_GITLAB_QA_OPTS
# todo: remove in 16.1 milestone when not needed for backwards compatibility anymore
EE_LICENSE: $QA_EE_LICENSE
GITHUB_ACCESS_TOKEN: $QA_GITHUB_ACCESS_TOKEN
# Allow QA jobs to fail as they are flaky. The top level `package-and-e2e:ee`
# pipeline is not allowed to fail, so without allowing QA to fail, we will be
# blocking merges due to flaky tests.
allow_failure: true
.trigger-omnibus-env:
stage: .pre
needs:
# We need this job because we need its `cached-assets-hash.txt` artifact, so that we can pass the assets image tag to the downstream omnibus-gitlab pipeline.
- pipeline: $PARENT_PIPELINE_ID
job: build-assets-image
variables:
BUILD_ENV: build.env
before_script:
- |
# This is duplicating the function from `scripts/utils.sh` since `.gitlab/ci/package-and-test/main.gitlab-ci.yml` can be included in other projects.
function assets_image_tag() {
local cache_assets_hash_file="cached-assets-hash.txt"
if [[ -n "${CI_COMMIT_TAG}" ]]; then
echo -n "${CI_COMMIT_REF_NAME}"
elif [[ -f "${cache_assets_hash_file}" ]]; then
echo -n "assets-hash-$(cat ${cache_assets_hash_file} | cut -c1-10)"
else
echo -n "${CI_COMMIT_SHA}"
fi
}
script:
- |
SECURITY_SOURCES=$([[ ! "$CI_PROJECT_NAMESPACE" =~ ^gitlab-org\/security ]] || echo "true")
echo "SECURITY_SOURCES=${SECURITY_SOURCES:-false}" > $BUILD_ENV
echo "OMNIBUS_GITLAB_CACHE_UPDATE=${OMNIBUS_GITLAB_CACHE_UPDATE:-false}" >> $BUILD_ENV
for version_file in *_VERSION; do echo "$version_file=$(cat $version_file)" >> $BUILD_ENV; done
echo "OMNIBUS_GITLAB_RUBY3_BUILD=${OMNIBUS_GITLAB_RUBY3_BUILD:-false}" >> $BUILD_ENV
echo "OMNIBUS_GITLAB_RUBY2_BUILD=${OMNIBUS_GITLAB_RUBY2_BUILD:-false}" >> $BUILD_ENV
echo "OMNIBUS_GITLAB_CACHE_EDITION=${OMNIBUS_GITLAB_CACHE_EDITION:-GITLAB}" >> $BUILD_ENV
echo "OMNIBUS_GITLAB_BUILD_ON_ALL_OS=${OMNIBUS_GITLAB_BUILD_ON_ALL_OS:-false}" >> $BUILD_ENV
echo "GITLAB_ASSETS_TAG=$(assets_image_tag)" >> $BUILD_ENV
echo "EE=$([[ $FOSS_ONLY == '1' ]] && echo 'false' || echo 'true')" >> $BUILD_ENV
target_branch_name="${CI_MERGE_REQUEST_TARGET_BRANCH_NAME:-${CI_COMMIT_REF_NAME}}"
echo "TRIGGER_BRANCH=$([[ "${target_branch_name}" =~ ^[0-9-]+-stable(-ee)?$ ]] && echo ${target_branch_name%-ee} || echo 'master')" >> $BUILD_ENV
- |
echo "Built environment file for omnibus build:"
cat $BUILD_ENV
artifacts:
expire_in: 3 days
reports:
dotenv: $BUILD_ENV
paths:
- $BUILD_ENV
.trigger-omnibus-env-ce:
extends: .trigger-omnibus-env
needs:
- pipeline: $PARENT_PIPELINE_ID
job: build-assets-image as-if-foss
.trigger-omnibus:
stage: .pre
inherit:
variables: false
variables:
GITALY_SERVER_VERSION: $GITALY_SERVER_VERSION
GITLAB_ELASTICSEARCH_INDEXER_VERSION: $GITLAB_ELASTICSEARCH_INDEXER_VERSION
GITLAB_KAS_VERSION: $GITLAB_KAS_VERSION
GITLAB_METRICS_EXPORTER_VERSION: $GITLAB_METRICS_EXPORTER_VERSION
GITLAB_PAGES_VERSION: $GITLAB_PAGES_VERSION
GITLAB_SHELL_VERSION: $GITLAB_SHELL_VERSION
GITLAB_WORKHORSE_VERSION: $GITLAB_WORKHORSE_VERSION
GITLAB_VERSION: $CI_COMMIT_SHA
GITLAB_ASSETS_TAG: $GITLAB_ASSETS_TAG
IMAGE_TAG: $CI_COMMIT_SHA
TOP_UPSTREAM_SOURCE_PROJECT: $CI_PROJECT_PATH
SECURITY_SOURCES: $SECURITY_SOURCES
CACHE_UPDATE: $OMNIBUS_GITLAB_CACHE_UPDATE
RUBY3_BUILD: $OMNIBUS_GITLAB_RUBY3_BUILD
RUBY2_BUILD: $OMNIBUS_GITLAB_RUBY2_BUILD
CACHE_EDITION: $OMNIBUS_GITLAB_CACHE_EDITION
BUILD_ON_ALL_OS: $OMNIBUS_GITLAB_BUILD_ON_ALL_OS
SKIP_QA_TEST: "true"
ee: $EE
trigger:
project: gitlab-org/build/omnibus-gitlab-mirror
branch: $TRIGGER_BRANCH
strategy: depend
.trigger-omnibus-ce:
extends:
- .trigger-omnibus
variables:
# Override gitlab repository so that omnibus doesn't use foss repository for CE build
GITLAB_ALTERNATIVE_REPO: $CI_PROJECT_URL
.download-knapsack-report:
extends:
- .gitlab-qa-image
stage: .pre
variables:
KNAPSACK_DIR: ${CI_PROJECT_DIR}/qa/knapsack
GIT_STRATEGY: none
script:
# when using qa-image, code runs in /home/gitlab/qa folder
- bundle exec rake "knapsack:download[test]"
- mkdir -p "$KNAPSACK_DIR" && cp knapsack/*.json "${KNAPSACK_DIR}/"
allow_failure: true
artifacts:
paths:
- qa/knapsack/*.json
expire_in: 1 day
.e2e-test-report:
extends:
- .generate-allure-report-base
stage: report
variables:
GITLAB_AUTH_TOKEN: $PROJECT_TOKEN_FOR_CI_SCRIPTS_API_USAGE
ALLURE_PROJECT_PATH: $CI_PROJECT_PATH
ALLURE_MERGE_REQUEST_IID: $CI_MERGE_REQUEST_IID
.upload-knapsack-report:
extends:
- .generate-knapsack-report-base
- .qa-install
- .ruby-image
stage: report
when: always
.export-test-metrics:
extends:
- .qa-install
- .ruby-image
stage: report
when: always
variables:
QA_METRICS_REPORT_FILE_PATTERN: $CI_PROJECT_DIR/gitlab-qa-run-*/**/test-metrics-*.json
script:
- bundle exec rake "ci:export_test_metrics[$QA_METRICS_REPORT_FILE_PATTERN]"
.relate-test-failures:
extends:
- .qa-install
- .ruby-image
stage: report
when: always
variables:
QA_RSPEC_JSON_FILE_PATTERN: "${CI_PROJECT_DIR}/gitlab-qa-run-*/**/rspec-*.json"
script:
- |
if [ "$SUITE_FAILED" != "true" ] && [ "$SUITE_RAN" == "true" ]; then
echo "Test suite passed. Exiting..."
exit 0
fi
- |
bundle exec relate-failure-issue \
--input-files "${QA_RSPEC_JSON_FILE_PATTERN}" \
--project "gitlab-org/gitlab" \
--token "${QA_RELATE_FAILURE_ISSUE_TOKEN}"
.generate-test-session:
extends:
- .qa-install
- .ruby-image
stage: report
when: always
variables:
QA_RSPEC_JSON_FILE_PATTERN: "${CI_PROJECT_DIR}/gitlab-qa-run-*/**/rspec-*.json"
script:
- |
bundle exec generate-test-session \
--input-files "${QA_RSPEC_JSON_FILE_PATTERN}" \
--project "gitlab-org/quality/testcase-sessions" \
--token "${QA_TEST_SESSION_TOKEN}" \
--ci-project-token "${GENERATE_TEST_SESSION_READ_API_REPORTER_TOKEN}" \
--issue-url-file report_issue_url.txt
artifacts:
when: always
expire_in: 1d
paths:
- qa/report_issue_url.txt
.notify-slack:
extends:
- .notify-slack-qa
- .qa-install
- .ruby-image
stage: notify
variables:
QA_RSPEC_XML_FILE_PATTERN: "${CI_PROJECT_DIR}/gitlab-qa-run-*/**/rspec-*.xml"
SLACK_ICON_EMOJI: ci_failing
STATUS_SYM: ☠️
STATUS: failed
TYPE: "($QA_RUN_TYPE) "
when: always
script:
- |
if [ "$SUITE_FAILED" != "true" ] && [ "$SUITE_RAN" == "true" ]; then
echo "Test suite passed. Exiting..."
exit 0
fi
- bundle exec prepare-stage-reports --input-files "${QA_RSPEC_XML_FILE_PATTERN}"
- !reference [.notify-slack-qa, script]
# ==========================================
# Pre stage
# ==========================================
dont-interrupt-me:
stage: .pre
interruptible: false
script:
- echo "This jobs makes sure this pipeline won't be interrupted! See https://docs.gitlab.com/ee/ci/yaml/#interruptible."
rules:
- if: '$CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH && $CI_MERGE_REQUEST_IID == null'
allow_failure: true
- if: '$CI_MERGE_REQUEST_EVENT_TYPE == "merged_result" || $CI_MERGE_REQUEST_EVENT_TYPE == "detached"'
when: manual
allow_failure: true

View file

@ -1,154 +0,0 @@
# Specific specs passed
.specific-specs: &specific-specs
if: $QA_TESTS != ""
# No specific specs passed
.all-specs: &all-specs
if: $QA_TESTS == ""
# FF changes
.feature-flags-set: &feature-flags-set
if: $QA_FEATURE_FLAGS =~ /enabled|disabled/
# Manually trigger job on ff changes but with default ff state instead of inverted
.feature-flags-set-manual: &feature-flags-set-manual
<<: *feature-flags-set
when: manual
allow_failure: true
# Run the job on master pipeline
.default-branch: &default-branch
if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
# Run all tests when QA framework changes present, full suite execution is explicitly enabled or a feature flag file is removed
.qa-run-all-tests: &qa-run-all-tests
if: $QA_FRAMEWORK_CHANGES == "true" || $QA_RUN_ALL_TESTS == "true" || $QA_RUN_ALL_E2E_LABEL == "true" || $QA_FEATURE_FLAGS =~ /deleted/
# Run job when MR has pipeline:run-all-e2e label
.qa-run-all-e2e-label: &qa-run-all-e2e-label
if: $QA_RUN_ALL_E2E_LABEL == "true"
# Process test results (notify failure to slack, create test session report, relate test failures)
.process-test-results: &process-test-results
if: $PROCESS_TEST_RESULTS == "true"
.not-canonical-project: &not-canonical-project
if: '$CI_PROJECT_PATH != "gitlab-org/gitlab" && $CI_PROJECT_PATH != "gitlab-cn/gitlab"'
# Selective test execution against omnibus instance have following execution scenarios:
# * only e2e spec files changed - runs only changed specs
# * qa framework changes - runs full test suite
# * feature flag changed - runs full test suite with base gitlab instance configuration with both ff states
# * quarantined e2e spec - skips execution of e2e tests by creating a no-op pipeline
# ------------------------------------------
# Prepare
# ------------------------------------------
.rules:prepare:
rules:
- when: always
.rules:omnibus-build:
rules:
- if: $SKIP_OMNIBUS_TRIGGER == "true"
when: never
- if: $FOSS_ONLY != "1"
.rules:omnibus-build-ce:
rules:
- if: $SKIP_OMNIBUS_TRIGGER == "true"
when: never
- if: $FOSS_ONLY == "1"
.rules:update-cache:
rules:
- if: '$UPDATE_QA_CACHE == "true"'
.rules:download-knapsack:
rules:
- when: always
# ------------------------------------------
# Test
# ------------------------------------------
.rules:test:manual:
rules:
- when: manual
allow_failure: true
variables:
QA_TESTS: ""
.rules:test:feature-flags-set:
rules:
# unset specific specs if pipeline has feature flag changes and run full suite
- <<: *feature-flags-set
variables:
QA_TESTS: ""
# parallel and non parallel rules are used for jobs that require parallel execution and thus need to switch
# between parallel and non parallel when only certain specs are executed
.rules:test:qa-selective:
rules:
# always run parallel with full suite when framework changes present or ff state changed
- <<: *qa-run-all-tests
when: never
- <<: *all-specs
when: never
- <<: *feature-flags-set
when: never
.rules:test:qa-parallel:
rules:
- *qa-run-all-tests
- <<: *specific-specs
when: manual
allow_failure: true
variables:
QA_TESTS: ""
- *feature-flags-set-manual
# general qa job rule for jobs without the need to run in parallel
.rules:test:qa:
rules:
- *qa-run-all-tests
- *feature-flags-set-manual
.rules:test:ee-only:
rules:
- if: $FOSS_ONLY == "1"
when: never
.rules:test:update:
rules:
# skip upgrade jobs if gitlab version is not in semver compatible format
# these jobs need gitlab version because we can't reliably detect it from just the image
- if: $GITLAB_SEMVER_VERSION !~ /^\d+\.\d+\.\d+/
when: never
# update type tests are used to check if gitlab upgrade can be performed correctly (mainly migrations)
# there isn't much benefit in running tests after update with new sidebar enabled and there
# is also an issue to properly pass feature toggle to this job due to how gitlab-qa parses cli args
- if: $QA_SUPER_SIDEBAR_ENABLED == "true"
when: never
- !reference [.rules:test:ee-only, rules]
- !reference [.rules:test:qa, rules]
.rules:test:qa-default-branch:
rules:
- *qa-run-all-e2e-label
- *default-branch
- *feature-flags-set-manual
# ------------------------------------------
# Report
# ------------------------------------------
.rules:report:allure-report:
rules:
- if: $SKIP_ALLURE_REPORT == "true"
when: never
- when: always
.rules:report:process-results:
rules:
- <<: *not-canonical-project
when: never
- *process-test-results

View file

@ -1,20 +0,0 @@
# Default variables for package-and-test
variables:
REGISTRY_HOST: "registry.gitlab.com"
REGISTRY_GROUP: "gitlab-org"
SKIP_REPORT_IN_ISSUES: "true"
SKIP_OMNIBUS_TRIGGER: "true"
OMNIBUS_GITLAB_CACHE_UPDATE: "false"
OMNIBUS_GITLAB_RUBY3_BUILD: "false"
OMNIBUS_GITLAB_RUBY2_BUILD: "false"
OMNIBUS_GITLAB_CACHE_EDITION: "GITLAB"
OMNIBUS_GITLAB_BUILD_ON_ALL_OS: "false"
ALLURE_JOB_NAME: $CI_PROJECT_NAME
COLORIZED_LOGS: "true"
QA_LOG_LEVEL: "info"
QA_TESTS: ""
QA_FEATURE_FLAGS: ""
# run all tests by default when package-and-test is included natively in other projects
# this will be overridden when selective test execution is used in gitlab canonical project
QA_RUN_ALL_TESTS: "true"

View file

@ -1,5 +1,4 @@
.qa-job-base:
image: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images/debian-bullseye-ruby-${RUBY_VERSION}:bundler-2.3-chrome-${CHROME_VERSION}-docker-${DOCKER_VERSION}
extends:
- .default-retry
- .qa-cache
@ -10,74 +9,36 @@
SETUP_DB: "false"
before_script:
- !reference [.default-before_script, before_script]
- cd qa && bundle install
.e2e-trigger-base:
extends: .production # this makes sure GITLAB_ALLOW_SEPARATE_CI_DATABASE is passed to the child pipeline
stage: qa
needs:
- build-assets-image
- build-qa-image
- e2e-test-pipeline-generate
variables:
# This is needed by `trigger-omnibus-env` (`.gitlab/ci/package-and-test/main.gitlab-ci.yml`).
PARENT_PIPELINE_ID: $CI_PIPELINE_ID
SKIP_MESSAGE: Skipping package-and-test due to mr containing only quarantine changes!
GITLAB_QA_IMAGE: "${CI_REGISTRY_IMAGE}/gitlab-ee-qa:${CI_COMMIT_SHA}"
RUN_WITH_BUNDLE: "true" # instructs pipeline to install and run gitlab-qa gem via bundler
QA_PATH: qa # sets the optional path for bundler to run from
DYNAMIC_PIPELINE_YML: package-and-test-pipeline.yml # yml files are generated by scripts/generate-e2e-pipeline script
inherit:
variables:
- CHROME_VERSION
- RUBY_VERSION
- DOCKER_VERSION
- REGISTRY_GROUP
- REGISTRY_HOST
- OMNIBUS_GITLAB_CACHE_EDITION
- OMNIBUS_GITLAB_RUBY3_BUILD
- OMNIBUS_GITLAB_RUBY2_BUILD
trigger:
strategy: depend
forward:
yaml_variables: true
pipeline_variables: true
include:
- artifact: $DYNAMIC_PIPELINE_YML
job: e2e-test-pipeline-generate
- cd qa/
- bundle_install_script
qa:internal:
extends:
- .qa-job-base
- .qa:rules:internal
- .qa:rules:ee-and-foss
script:
- bundle exec rspec -O .rspec_internal
- bundle exec rspec
qa:internal-as-if-foss:
extends:
- qa:internal
- .qa:rules:internal-as-if-foss
- .qa:rules:as-if-foss
- .as-if-foss
qa:master-auto-quarantine-dequarantine:
qa:selectors:
extends:
- .qa-job-base
rules:
- if: '$QA_TRIGGER_AUTO_QUARANTINE =~ /true|yes|1/i'
- .qa:rules:ee-and-foss
script:
- bundle exec confiner -r .confiner/master.yml
allow_failure: true
- bundle exec bin/qa Test::Sanity::Selectors
qa:nightly-auto-quarantine-dequarantine:
qa:selectors-as-if-foss:
extends:
- .qa-job-base
rules:
- if: '$QA_TRIGGER_AUTO_QUARANTINE =~ /true|yes|1/i'
script:
- bundle exec confiner -r .confiner/nightly.yml
allow_failure: true
- qa:selectors
- .qa:rules:as-if-foss
- .as-if-foss
qa:update-qa-cache:
update-qa-cache:
extends:
- .qa-job-base
- .qa-cache-push
@ -86,81 +47,25 @@ qa:update-qa-cache:
script:
- echo "Cache has been updated and ready to be uploaded."
e2e:package-and-test-ee:
extends:
- .e2e-trigger-base
- .qa:rules:package-and-test-ee
needs:
- build-assets-image
- build-qa-image
- e2e-test-pipeline-generate
variables:
RELEASE: "${REGISTRY_HOST}/${REGISTRY_GROUP}/build/omnibus-gitlab-mirror/gitlab-ee:${CI_COMMIT_SHA}"
QA_RUN_TYPE: e2e-package-and-test
ALLURE_JOB_NAME: e2e-package-and-test
PIPELINE_NAME: E2E Omnibus GitLab EE
e2e:package-and-test-ce:
extends:
- e2e:package-and-test-ee
- .qa:rules:package-and-test-ce
needs:
- build-assets-image as-if-foss
- build-qa-image as-if-foss
- e2e-test-pipeline-generate
variables:
FOSS_ONLY: "1"
RELEASE: ${REGISTRY_HOST}/${REGISTRY_GROUP}/build/omnibus-gitlab-mirror/gitlab-ce:${CI_COMMIT_SHA}
GITLAB_QA_IMAGE: ${CI_REGISTRY_IMAGE}/gitlab-ce-qa:${CI_COMMIT_SHA}
QA_RUN_TYPE: e2e-package-and-test-ce
ALLURE_JOB_NAME: e2e-package-and-test-ce
PIPELINE_NAME: E2E Omnibus GitLab CE
e2e:package-and-test-super-sidebar:
extends:
- e2e:package-and-test-ee
- .qa:rules:package-and-test-sidebar
when: manual
variables:
QA_SUPER_SIDEBAR_ENABLED: "true"
EXTRA_GITLAB_QA_OPTS: --set-feature-flags super_sidebar_nav=enabled
QA_RUN_TYPE: e2e-package-and-test-super-sidebar
ALLURE_JOB_NAME: e2e-package-and-test-super-sidebar
PIPELINE_NAME: E2E Omnibus Super Sidebar
e2e:package-and-test-nightly:
extends:
- .e2e-trigger-base
- .qa:rules:package-and-test-nightly
needs:
- build-assets-image
- build-assets-image as-if-foss
- build-qa-image
- build-qa-image as-if-foss
- e2e-test-pipeline-generate
variables:
GITLAB_SEMVER_VERSION: $GITLAB_SEMVER_VERSION
QA_RUN_TYPE: nightly
ALLURE_JOB_NAME: nightly
PIPELINE_NAME: E2E Omnibus GitLab Nightly
DYNAMIC_PIPELINE_YML: package-and-test-nightly-pipeline.yml
e2e:test-on-gdk:
extends:
- .e2e-trigger-base
- .qa:rules:e2e:test-on-gdk
.package-and-qa-base:
image: ${GITLAB_DEPENDENCY_PROXY}ruby:2.7-alpine
stage: qa
retry: 0
script:
- source scripts/utils.sh
- install_gitlab_gem
- ./scripts/trigger-build omnibus
package-and-qa:
extends:
- .package-and-qa-base
- .qa:rules:package-and-qa
# This job often times out, so temporarily use private runners and a long timeout: https://gitlab.com/gitlab-org/gitlab/-/issues/238563
tags:
- prm
timeout: 4h
needs:
# In scheduled master pipelines we wait for the image to be built.
# In MRs we assume the last scheduled master pipeline built the image already.
- job: build-qa-on-gdk-master-image
optional: true
- job: e2e-test-pipeline-generate
artifacts: true
variables:
ALLURE_JOB_NAME: e2e-test-on-gdk
QA_RUN_TYPE: e2e-test-on-gdk
PIPELINE_NAME: E2E GDK
DYNAMIC_PIPELINE_YML: test-on-gdk-pipeline.yml
SKIP_MESSAGE: Skipping test-on-gdk due to mr containing only quarantine changes!
allow_failure: true
- job: build-qa-image
artifacts: false
- job: build-assets-image
artifacts: false

File diff suppressed because it is too large Load diff

View file

@ -1,90 +0,0 @@
# RSpec FOSS impact pipeline loaded dynamically by script: scripts/generate_rspec_pipeline.rb
include:
- local: .gitlab/ci/rails/shared.gitlab-ci.yml
default:
image: $DEFAULT_CI_IMAGE
tags:
- gitlab-org
# Default job timeout set to 90m https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/10520
timeout: 90m
interruptible: true
stages:
- test
dont-interrupt-me:
extends: .rules:dont-interrupt
stage: .pre
interruptible: false
script:
- echo "This jobs makes sure this pipeline won't be interrupted! See https://docs.gitlab.com/ee/ci/yaml/#interruptible."
.base-rspec-foss-impact:
extends: .rspec-base-pg13-as-if-foss
needs:
- pipeline: $PARENT_PIPELINE_ID
job: detect-tests
- pipeline: $PARENT_PIPELINE_ID
job: setup-test-env
- pipeline: $PARENT_PIPELINE_ID
job: retrieve-tests-metadata
- pipeline: $PARENT_PIPELINE_ID
job: compile-test-assets as-if-foss
rules:
- when: always
variables:
RSPEC_TESTS_FILTER_FILE: "${RSPEC_MATCHING_TESTS_FOSS_PATH}"
RSPEC_TESTS_MAPPING_ENABLED: "true"
script:
- !reference [.base-script, script]
- rspec_paralellized_job "--tag ~quarantine --tag ~level:background_migration --tag ~zoekt"
artifacts:
expire_in: 7d
paths:
- "${RSPEC_MATCHING_TESTS_FOSS_PATH}"
- tmp/capybara/
<% if rspec_files_per_test_level[:migration][:files].size > 0 %>
rspec migration foss-impact:
extends: .base-rspec-foss-impact
<% if rspec_files_per_test_level[:migration][:parallelization] > 1 %>
parallel: <%= rspec_files_per_test_level[:migration][:parallelization] %>
<% end %>
script:
- !reference [.base-script, script]
- rspec_paralellized_job "--tag ~quarantine --tag ~zoekt"
<% end %>
<% if rspec_files_per_test_level[:background_migration][:files].size > 0 %>
rspec background_migration foss-impact:
extends: .base-rspec-foss-impact
<% if rspec_files_per_test_level[:background_migration][:parallelization] > 1 %>
parallel: <%= rspec_files_per_test_level[:background_migration][:parallelization] %>
<% end %>
<% end %>
<% if rspec_files_per_test_level[:unit][:files].size > 0 %>
rspec unit foss-impact:
extends: .base-rspec-foss-impact
<% if rspec_files_per_test_level[:unit][:parallelization] > 1 %>
parallel: <%= rspec_files_per_test_level[:unit][:parallelization] %>
<% end %>
<% end %>
<% if rspec_files_per_test_level[:integration][:files].size > 0 %>
rspec integration foss-impact:
extends: .base-rspec-foss-impact
<% if rspec_files_per_test_level[:integration][:parallelization] > 1 %>
parallel: <%= rspec_files_per_test_level[:integration][:parallelization] %>
<% end %>
<% end %>
<% if rspec_files_per_test_level[:system][:files].size > 0 %>
rspec system foss-impact:
extends: .base-rspec-foss-impact
<% if rspec_files_per_test_level[:system][:parallelization] > 1 %>
parallel: <%= rspec_files_per_test_level[:system][:parallelization] %>
<% end %>
<% end %>

View file

@ -1,153 +0,0 @@
# RSpec preditive pipeline loaded dynamically by script: scripts/generate_rspec_pipeline.rb
include:
- local: .gitlab/ci/rails/shared.gitlab-ci.yml
default:
image: $DEFAULT_CI_IMAGE
tags:
- gitlab-org
# Default job timeout set to 90m https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/10520
timeout: 90m
interruptible: true
stages:
- test
dont-interrupt-me:
extends: .rules:dont-interrupt
stage: .pre
interruptible: false
script:
- echo "This jobs makes sure this pipeline won't be interrupted! See https://docs.gitlab.com/ee/ci/yaml/#interruptible."
.base-predictive:
needs:
- pipeline: $PARENT_PIPELINE_ID
job: detect-tests
- pipeline: $PARENT_PIPELINE_ID
job: setup-test-env
- pipeline: $PARENT_PIPELINE_ID
job: retrieve-tests-metadata
- pipeline: $PARENT_PIPELINE_ID
job: compile-test-assets
rules:
- when: always
variables:
RSPEC_TESTS_MAPPING_ENABLED: "true"
<% if test_suite_prefix.nil? %>
.base-rspec-predictive:
extends:
- .rspec-base-pg12
- .base-predictive
variables:
# We're using the FOSS one here because we want to exclude EE-only ones
# For EE-only ones, we have EE-only jobs.
RSPEC_TESTS_FILTER_FILE: "${RSPEC_MATCHING_TESTS_FOSS_PATH}"
<% if rspec_files_per_test_level.dig(:migration, :files).size > 0 %>
rspec migration predictive:
extends:
- .base-rspec-predictive
- .rspec-base-migration
<% if rspec_files_per_test_level.dig(:migration, :parallelization) > 1 %>
parallel: <%= rspec_files_per_test_level.dig(:migration, :parallelization) %>
<% end %>
<% end %>
<% if rspec_files_per_test_level.dig(:background_migration, :files).size > 0 %>
rspec background_migration predictive:
extends:
- .base-rspec-predictive
- .rspec-base-migration
<% if rspec_files_per_test_level.dig(:background_migration, :parallelization) > 1 %>
parallel: <%= rspec_files_per_test_level.dig(:background_migration, :parallelization) %>
<% end %>
<% end %>
<% if rspec_files_per_test_level.dig(:unit, :files).size > 0 %>
rspec unit predictive:
extends:
- .base-rspec-predictive
<% if rspec_files_per_test_level.dig(:unit, :parallelization) > 1 %>
parallel: <%= rspec_files_per_test_level.dig(:unit, :parallelization) %>
<% end %>
<% end %>
<% if rspec_files_per_test_level.dig(:integration, :files).size > 0 %>
rspec integration predictive:
extends:
- .base-rspec-predictive
<% if rspec_files_per_test_level.dig(:integration, :parallelization) > 1 %>
parallel: <%= rspec_files_per_test_level.dig(:integration, :parallelization) %>
<% end %>
<% end %>
<% if rspec_files_per_test_level.dig(:system, :files).size > 0 %>
rspec system predictive:
extends:
- .base-rspec-predictive
<% if rspec_files_per_test_level.dig(:system, :parallelization) > 1 %>
parallel: <%= rspec_files_per_test_level.dig(:system, :parallelization) %>
<% end %>
<% end %>
<% end %>
<% if test_suite_prefix == 'ee/' %>
.base-rspec-ee-predictive:
extends:
- .rspec-ee-base-pg12
- .base-predictive
variables:
RSPEC_TESTS_FILTER_FILE: "${RSPEC_MATCHING_TESTS_EE_PATH}"
<% if rspec_files_per_test_level.dig(:migration, :files).size > 0 %>
rspec-ee migration predictive:
extends:
- .base-rspec-ee-predictive
- .rspec-base-migration
<% if rspec_files_per_test_level.dig(:migration, :parallelization) > 1 %>
parallel: <%= rspec_files_per_test_level.dig(:migration, :parallelization) %>
<% end %>
<% end %>
<% if rspec_files_per_test_level.dig(:background_migration, :files).size > 0 %>
rspec-ee background_migration predictive:
extends:
- .base-rspec-ee-predictive
- .rspec-base-migration
<% if rspec_files_per_test_level.dig(:background_migration, :parallelization) > 1 %>
parallel: <%= rspec_files_per_test_level.dig(:background_migration, :parallelization) %>
<% end %>
<% end %>
<% if rspec_files_per_test_level.dig(:unit, :files).size > 0 %>
rspec-ee unit predictive:
extends:
- .base-rspec-ee-predictive
<% if rspec_files_per_test_level.dig(:unit, :parallelization) > 1 %>
parallel: <%= rspec_files_per_test_level.dig(:unit, :parallelization) %>
<% end %>
<% end %>
<% if rspec_files_per_test_level.dig(:integration, :files).size > 0 %>
rspec-ee integration predictive:
extends:
- .base-rspec-ee-predictive
<% if rspec_files_per_test_level.dig(:integration, :parallelization) > 1 %>
parallel: <%= rspec_files_per_test_level.dig(:integration, :parallelization) %>
<% end %>
<% end %>
<% if rspec_files_per_test_level.dig(:system, :files).size > 0 %>
rspec-ee system predictive:
extends:
- .base-rspec-ee-predictive
<% if rspec_files_per_test_level.dig(:system, :parallelization) > 1 %>
parallel: <%= rspec_files_per_test_level.dig(:system, :parallelization) %>
<% end %>
<% end %>
<% end %>

View file

@ -1,218 +0,0 @@
include:
- local: .gitlab/ci/global.gitlab-ci.yml
- local: .gitlab/ci/rules.gitlab-ci.yml
.rules:dont-interrupt:
rules:
- if: $CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH
allow_failure: true
- if: $CI_MERGE_REQUEST_IID
when: manual
allow_failure: true
#######################
# rspec job base specs
.rails-job-base:
extends:
- .default-retry
- .default-before_script
- .rails-cache
.base-script:
script:
- source ./scripts/rspec_helpers.sh
# Only install knapsack after bundle install! Otherwise oddly some native
# gems could not be found under some circumstance. No idea why, hours wasted.
- run_timed_command "gem install knapsack --no-document"
- echo -e "\e[0Ksection_start:`date +%s`:gitaly-test-spawn[collapsed=true]\r\e[0KStarting Gitaly"
- section_start "gitaly-test-spawn" "Spawning Gitaly"; scripts/gitaly-test-spawn; section_end "gitaly-test-spawn" # Do not use 'bundle exec' here
- echo -e "\e[0Ksection_end:`date +%s`:gitaly-test-spawn\r\e[0K"
.single-db:
variables:
DECOMPOSED_DB: "false"
.single-db-ci-connection:
extends: .single-db
variables:
CI_CONNECTION_DB: "true"
.single-db-rspec:
extends: .single-db
.single-db-ci-connection-rspec:
extends: .single-db-ci-connection
.praefect-with-db:
variables:
GITALY_PRAEFECT_WITH_DB: '1'
.rspec-base:
extends:
- .rails-job-base
- .base-artifacts
stage: test
variables:
RUBY_GC_MALLOC_LIMIT: 67108864
RUBY_GC_MALLOC_LIMIT_MAX: 134217728
RECORD_DEPRECATIONS: "true"
GEO_SECONDARY_PROXY: 0
SUCCESSFULLY_RETRIED_TEST_EXIT_CODE: 137
needs:
- job: "setup-test-env"
- job: "retrieve-tests-metadata"
- job: "compile-test-assets"
- job: "detect-tests"
optional: true
script:
- !reference [.base-script, script]
# We need to exclude background migration because unit tests run with
# spec/lib, yet background migration tests are also sitting there,
# and they should run on their own jobs so we don't need to run them
# in unit tests again.
- rspec_paralellized_job "--tag ~quarantine --tag ~level:background_migration"
allow_failure:
exit_codes: !reference [.rspec-base, variables, SUCCESSFULLY_RETRIED_TEST_EXIT_CODE]
.base-artifacts:
artifacts:
expire_in: 31d
when: always
paths:
- coverage/
- crystalball/
- deprecations/
- knapsack/
- query_recorder/
- rspec/
- tmp/capybara/
- log/*.log
reports:
junit: ${JUNIT_RESULT_FILE}
.rspec-base-migration:
script:
- !reference [.base-script, script]
- rspec_paralellized_job "--tag ~quarantine --tag ~zoekt"
.rspec-base-pg12:
extends:
- .rspec-base
- .use-pg12
.rspec-base-pg13:
extends:
- .rspec-base
- .use-pg13
.rspec-base-pg13-as-if-foss:
extends:
- .rspec-base
- .as-if-foss
- .use-pg13
needs:
- job: "setup-test-env"
- job: "retrieve-tests-metadata"
- job: "compile-test-assets as-if-foss"
- job: "detect-tests"
optional: true
.rspec-base-pg14:
extends:
- .rspec-base
- .use-pg14
.rspec-ee-base-pg12:
extends:
- .rspec-base
- .use-pg12-es7-ee
.rspec-ee-base-pg13:
extends:
- .rspec-base
- .use-pg13-es7-ee
.rspec-ee-base-pg13-es8:
extends:
- .rspec-base
- .use-pg13-es8-ee
- .rails:rules:run-search-tests
.rspec-ee-base-pg13-opensearch1:
extends:
- .rspec-base
- .use-pg13-opensearch1-ee
- .rails:rules:run-search-tests
.rspec-ee-base-pg13-opensearch2:
extends:
- .rspec-base
- .use-pg13-opensearch2-ee
- .rails:rules:run-search-tests
.rspec-ee-base-pg14:
extends:
- .rspec-base
- .use-pg14-es7-ee
.rspec-ee-base-pg14-es8:
extends:
- .rspec-base
- .use-pg14-es8-ee
- .rails:rules:run-search-tests
.rspec-ee-base-pg14-opensearch1:
extends:
- .rspec-base
- .use-pg14-opensearch1-ee
- .rails:rules:run-search-tests
.rspec-ee-base-pg14-opensearch2:
extends:
- .rspec-base
- .use-pg14-opensearch2-ee
- .rails:rules:run-search-tests
.db-job-base:
extends:
- .rails-job-base
- .rails:rules:ee-and-foss-migration
- .use-pg13
stage: test
needs: ["setup-test-env"]
# rspec job base specs
######################
############################
# rspec job parallel configs
.rspec-migration-parallel:
parallel: 8
.rspec-background-migration-parallel:
parallel: 4
.rspec-ee-migration-parallel:
parallel: 2
.rspec-ee-background-migration-parallel:
parallel: 2
.rspec-unit-parallel:
parallel: 28
.rspec-ee-unit-parallel:
parallel: 18
.rspec-integration-parallel:
parallel: 12
.rspec-ee-integration-parallel:
parallel: 6
.rspec-system-parallel:
parallel: 28
.rspec-ee-system-parallel:
parallel: 10
# rspec job parallel configs
############################

View file

@ -1,23 +0,0 @@
---
start-release-environments-pipeline:
allow_failure: true
extends:
- .release-environments:rules:start-release-environments-pipeline
stage: release-environments
# We do not want to have ALL global variables passed as trigger variables,
# as they cannot be overridden. See this issue for more context:
#
# https://gitlab.com/gitlab-org/gitlab/-/issues/387183
inherit:
variables:
- RUBY_VERSION
# These variables are set in the pipeline schedules.
# They need to be explicitly passed on to the child pipeline.
# https://docs.gitlab.com/ee/ci/pipelines/multi_project_pipelines.html#pass-cicd-variables-to-a-downstream-pipeline-by-using-the-variables-keyword
variables:
# This is needed by `release-environments-build-cng-env` (`.gitlab/ci/release-environments/main.gitlab-ci.yml`).
PARENT_PIPELINE_ID: $CI_PIPELINE_ID
trigger:
strategy: depend
include: .gitlab/ci/release-environments/main.gitlab-ci.yml

View file

@ -1,94 +0,0 @@
---
default:
interruptible: true
stages:
- prepare
- deploy
include:
- local: .gitlab/ci/global.gitlab-ci.yml
release-environments-build-cng-env:
allow_failure: true
image: ${GITLAB_DEPENDENCY_PROXY_ADDRESS}ruby:${RUBY_VERSION}-alpine3.16
stage: prepare
needs:
# We need this job because we need its `cached-assets-hash.txt` artifact, so that we can pass the assets image tag to the downstream CNG pipeline.
- pipeline: $PARENT_PIPELINE_ID
job: build-assets-image
variables:
BUILD_ENV: build.env
before_script:
- source ./scripts/utils.sh
- install_gitlab_gem
script:
- 'ruby -r./scripts/trigger-build.rb -e "puts Trigger.variables_for_env_file(Trigger::CNG.new.variables)" > $BUILD_ENV'
- echo "GITLAB_ASSETS_TAG=$(assets_image_tag)" >> $BUILD_ENV
- ruby -e 'puts "FULL_RUBY_VERSION=#{RUBY_VERSION}"' >> build.env
- cat $BUILD_ENV
artifacts:
reports:
dotenv: $BUILD_ENV
paths:
- $BUILD_ENV
expire_in: 7 days
when: always
release-environments-build-cng:
allow_failure: true
stage: prepare
needs: ["release-environments-build-cng-env"]
inherit:
variables: false
variables:
GITLAB_REF_SLUG: "${GITLAB_REF_SLUG}"
# CNG pipeline specific variables
GITLAB_VERSION: "${GITLAB_VERSION}"
GITLAB_TAG: "${GITLAB_TAG}"
GITLAB_ASSETS_TAG: "${GITLAB_ASSETS_TAG}"
FORCE_RAILS_IMAGE_BUILDS: "${FORCE_RAILS_IMAGE_BUILDS}"
CE_PIPELINE: "${CE_PIPELINE}" # Based on https://docs.gitlab.com/ee/ci/jobs/job_control.html#check-if-a-variable-exists, `if: '$CE_PIPELINE'` will evaluate to `false` when this variable is empty
EE_PIPELINE: "${EE_PIPELINE}" # Based on https://docs.gitlab.com/ee/ci/jobs/job_control.html#check-if-a-variable-exists, `if: '$EE_PIPELINE'` will evaluate to `false` when this variable is empty
GITLAB_ELASTICSEARCH_INDEXER_VERSION: "${GITLAB_ELASTICSEARCH_INDEXER_VERSION}"
GITLAB_KAS_VERSION: "${GITLAB_KAS_VERSION}"
GITLAB_METRICS_EXPORTER_VERSION: "${GITLAB_METRICS_EXPORTER_VERSION}"
GITLAB_PAGES_VERSION: "${GITLAB_PAGES_VERSION}"
GITLAB_SHELL_VERSION: "${GITLAB_SHELL_VERSION}"
GITALY_SERVER_VERSION: "${GITALY_SERVER_VERSION}"
RUBY_VERSION: "${FULL_RUBY_VERSION}"
IMAGE_TAG_EXT: "-${CI_COMMIT_SHORT_SHA}"
trigger:
project: gitlab-org/build/CNG-mirror
branch: $TRIGGER_BRANCH
strategy: depend
release-environments-deploy-env:
allow_failure: true
stage: deploy
needs: ["release-environments-build-cng"]
variables:
DEPLOY_ENV: deploy.env
script:
- ./scripts/construct-release-environments-versions.rb > $DEPLOY_ENV
artifacts:
reports:
dotenv: $DEPLOY_ENV
paths:
- $DEPLOY_ENV
expire_in: 7 days
when: always
release-environments-deploy:
allow_failure: true
stage: deploy
needs: ["release-environments-deploy-env"]
inherit:
variables: false
variables:
VERSIONS: "${VERSIONS}"
ENVIRONMENT: "${ENVIRONMENT}"
trigger:
project: gitlab-com/gl-infra/release-environments
branch: main
strategy: depend

View file

@ -4,7 +4,7 @@
.merge-train-sync:
# We don't need/want any global before/after commands, so we overwrite these
# settings.
image: ${GITLAB_DEPENDENCY_PROXY_ADDRESS}alpine:edge
image: ${GITLAB_DEPENDENCY_PROXY}alpine:edge
stage: sync
before_script:
- apk add --no-cache --update curl bash jq

View file

@ -1,28 +1,24 @@
include:
- template: Jobs/Code-Quality.gitlab-ci.yml
- template: Jobs/SAST.gitlab-ci.yml
- template: Jobs/Secret-Detection.gitlab-ci.yml
- template: Jobs/Dependency-Scanning.gitlab-ci.yml
- template: Security/SAST.gitlab-ci.yml
- template: Security/Secret-Detection.gitlab-ci.yml
- template: Security/Dependency-Scanning.gitlab-ci.yml
- template: Security/License-Scanning.gitlab-ci.yml
code_quality:
extends:
- .default-retry
- .use-docker-in-docker
stage: lint
artifacts:
paths:
- gl-code-quality-report.json # GitLab-specific
# extends generated values cannot overwrite values from included files
# Use !reference as a workaround here
rules: !reference [".reports:rules:code_quality", rules]
allow_failure: true
.sast-analyzer:
# We need to re-`extends` from `sast` as the `extends` here overrides the one from the template.
extends:
- .default-retry
- sast
stage: lint
needs: []
artifacts:
paths:
@ -31,17 +27,22 @@ code_quality:
variables:
SAST_BRAKEMAN_LEVEL: 2 # GitLab-specific
SAST_EXCLUDED_PATHS: "qa, spec, doc, ee/spec, config/gitlab.yml.example, tmp" # GitLab-specific
SAST_EXCLUDED_ANALYZERS: bandit, flawfinder, phpcs-security-audit, pmd-apex, security-code-scan, spotbugs, eslint, nodejs-scan, sobelow
SAST_DISABLE_BABEL: "true"
brakeman-sast:
rules: !reference [".reports:rules:brakeman-sast", rules]
rules: !reference [".reports:rules:sast", rules]
eslint-sast:
rules: !reference [".reports:rules:sast", rules]
nodejs-scan-sast:
rules: !reference [".reports:rules:sast", rules]
semgrep-sast:
rules: !reference [".reports:rules:semgrep-sast", rules]
rules: !reference [".reports:rules:sast", rules]
.secret-analyzer:
extends: .default-retry
stage: lint
needs: []
artifacts:
paths:
@ -56,55 +57,50 @@ secret_detection:
extends:
- .default-retry
- dependency_scanning
stage: lint
needs: []
variables:
DS_EXCLUDED_PATHS: "qa/qa/ee/fixtures/secure_premade_reports, spec, ee/spec, tmp" # GitLab-specific
DS_EXCLUDED_ANALYZERS: "gemnasium-maven"
artifacts:
paths:
- gl-dependency-scanning-report.json # GitLab-specific
expire_in: 1 week # GitLab-specific
gemnasium-dependency_scanning:
variables:
DS_REMEDIATE: "false"
rules: !reference [".reports:rules:gemnasium-dependency_scanning", rules]
before_script:
# git-lfs is needed for auto-remediation
- apk add git-lfs
after_script:
# Post-processing
- apk add jq
# Lower execa severity based on https://gitlab.com/gitlab-org/gitlab/-/issues/223859#note_452922390
- jq '(.vulnerabilities[] | select (.cve == "yarn.lock:execa:gemnasium:05cfa2e8-2d0c-42c1-8894-638e2f12ff3d")).severity = "Medium"' gl-dependency-scanning-report.json > temp.json && mv temp.json gl-dependency-scanning-report.json
rules: !reference [".reports:rules:dependency_scanning", rules]
bundler-audit-dependency_scanning:
rules: !reference [".reports:rules:dependency_scanning", rules]
retire-js-dependency_scanning:
rules: !reference [".reports:rules:dependency_scanning", rules]
gemnasium-python-dependency_scanning:
rules: !reference [".reports:rules:gemnasium-python-dependency_scanning", rules]
yarn-audit-dependency_scanning:
extends: .ds-analyzer
image: "${REGISTRY_HOST}/${REGISTRY_GROUP}/security-products/analyzers/npm-audit:1"
variables:
TOOL: yarn
rules: !reference [".reports:rules:yarn-audit-dependency_scanning", rules]
rules: !reference [".reports:rules:dependency_scanning", rules]
# Analyze dependencies for malicious behavior
# See https://gitlab.com/gitlab-com/gl-security/security-research/package-hunter
.package_hunter-base:
extends: .default-retry
package_hunter:
extends:
- .default-retry
- .reports:rules:package_hunter
stage: test
image:
name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/security-products/package-hunter-cli:v1.3.3@sha256:1d3af9a61aa01549a62be17fa655fcf06271ac9e1b1e822c2a7930fa1d4a8a6b
name: registry.gitlab.com/gitlab-com/gl-security/security-research/package-hunter-cli:latest
entrypoint: [""]
variables:
HTR_user: '$PACKAGE_HUNTER_USER'
HTR_pass: '$PACKAGE_HUNTER_PASS'
needs: []
allow_failure: true
before_script:
script:
- rm -r spec locale .git app/assets/images doc/
- cd .. && tar -I "gzip --best" -cf gitlab.tgz gitlab/
script:
- DEBUG=* node /usr/src/app/cli.js analyze --format gitlab --manager ${PACKAGE_MANAGER} gitlab.tgz | tee ${CI_PROJECT_DIR}/gl-dependency-scanning-report.json
after_script:
- mkdir ~/.aws
- '[[ -z "${AWS_SIEM_REPORT_INGESTION_CREDENTIALS_FILE}" ]] || mv "${AWS_SIEM_REPORT_INGESTION_CREDENTIALS_FILE}" ~/.aws/credentials'
- npm install --no-save --ignore-scripts @aws-sdk/client-s3@3.49.0
- scripts/ingest-reports-to-siem || true # Allow legacy report to fail as we'll remove it in the future anyway
- scripts/ingest-reports-to-siem-devo
- DEBUG=* HTR_user=$PACKAGE_HUNTER_USER HTR_pass=$PACKAGE_HUNTER_PASS node /usr/src/app/cli.js analyze --format gitlab gitlab.tgz | tee $CI_PROJECT_DIR/gl-dependency-scanning-report.json
artifacts:
paths:
- gl-dependency-scanning-report.json
@ -112,16 +108,9 @@ yarn-audit-dependency_scanning:
dependency_scanning: gl-dependency-scanning-report.json
expire_in: 1 week
package_hunter-yarn:
extends:
- .package_hunter-base
- .reports:rules:package_hunter-yarn
variables:
PACKAGE_MANAGER: yarn
package_hunter-bundler:
extends:
- .package_hunter-base
- .reports:rules:package_hunter-bundler
variables:
PACKAGE_MANAGER: bundler
license_scanning:
extends: .default-retry
needs: []
artifacts:
expire_in: 1 week # GitLab-specific
rules: !reference [".reports:rules:license_scanning", rules]

View file

@ -1,35 +0,0 @@
include:
- template: DAST-API.gitlab-ci.yml
dast_api:
needs: ["review-deploy"]
# Uncomment resource_group if DAST_API_PROFILE is changed to an active scan
# resource_group: dast_api_scan
rules:
- when: never
dast_api_graphql:
extends: dast_api
variables:
DAST_API_GRAPHQL: /api/graphql
DAST_API_PROFILE: Passive
DAST_API_TARGET_URL: ${CI_ENVIRONMENT_URL}
DAST_API_OVERRIDES_ENV: "{\"headers\":{\"Authorization\":\"Bearer $REVIEW_APPS_ROOT_TOKEN\"}}"
rules:
- !reference [".reports:rules:schedule-dast", rules]
#
# To run this job in an MR pipeline, use this rule:
# - !reference [".reports:rules:test-dast", rules]
dast_api_rest:
extends: dast_api
variables:
DAST_API_OPENAPI: doc/api/openapi/openapi_v2.yaml
DAST_API_PROFILE: Passive
DAST_API_TARGET_URL: ${CI_ENVIRONMENT_URL}
DAST_API_OVERRIDES_ENV: "{\"headers\":{\"Authorization\":\"Bearer $REVIEW_APPS_ROOT_TOKEN\"}}"
rules:
- !reference [".reports:rules:schedule-dast", rules]
#
# To run this job in an MR pipeline, use this rule:
# - !reference [".reports:rules:test-dast", rules]

View file

@ -1,106 +0,0 @@
.dast_conf:
tags:
- prm
# For scheduling dast job
extends:
- .reports:rules:schedule-dast
image:
name: "${CI_TEMPLATE_REGISTRY_HOST}/security-products/dast:$DAST_VERSION"
resource_group: dast_scan
variables:
DAST_USERNAME_FIELD: "name:user[login]"
DAST_PASSWORD_FIELD: "name:user[password]"
DAST_SUBMIT_FIELD: "css:.js-sign-in-button"
DAST_FULL_SCAN_ENABLED: "true"
DAST_VERSION: 3
GIT_STRATEGY: none
# -Xmx is used to set the JVM memory to 6GB to prevent DAST OutOfMemoryError.
DAST_ZAP_CLI_OPTIONS: "-Xmx6144m"
before_script:
- 'export DAST_WEBSITE="${DAST_WEBSITE:-$(cat environment_url.txt)}"'
- 'export DAST_AUTH_URL="${DAST_WEBSITE}/users/sign_in"'
- 'export DAST_PASSWORD="${REVIEW_APPS_ROOT_PASSWORD}"'
# Help pages are excluded from scan as they are static pages.
# profile/two_factor_auth is excluded from scan to prevent 2FA from being turned on from user profile, which will reduce coverage.
- 'DAST_EXCLUDE_URLS="${DAST_WEBSITE}/help/.*,${DAST_WEBSITE}/-/profile/two_factor_auth,${DAST_WEBSITE}/users/sign_out"'
# Exclude the automatically generated monitoring project from being tested due to https://gitlab.com/gitlab-org/gitlab/-/issues/260362
- 'export DAST_EXCLUDE_URLS="${DAST_EXCLUDE_URLS},${DAST_WEBSITE}/gitlab-instance-.*"'
needs: ["review-deploy"]
stage: dast
# Default job timeout set to 90m and dast rules needs 2h to so that it won't timeout.
timeout: 3h
# Add retry because of intermittent connection problems. See https://gitlab.com/gitlab-org/gitlab/-/issues/244313
retry: 1
artifacts:
paths:
- gl-dast-report.json # GitLab-specific
reports:
dast: gl-dast-report.json
expire_in: 1 week # GitLab-specific
allow_failure: true
# DAST scan with a subset of Release scan rules.
# ZAP rule details can be found at https://www.zaproxy.org/docs/alerts/
dast:anti-clickjacking-header:
extends:
- .dast_conf
variables:
DAST_USERNAME: "user1"
DAST_ONLY_INCLUDE_RULES: "10020"
script:
- /analyze
dast:xss-persistant:
extends:
- .dast_conf
variables:
DAST_USERNAME: "user2"
DAST_ONLY_INCLUDE_RULES: "40014"
script:
- /analyze
dast:insecure-http-method:
extends:
- .dast_conf
variables:
DAST_USERNAME: "user3"
DAST_ONLY_INCLUDE_RULES: "90028"
script:
- /analyze
dast:server-side-template-inj:
extends:
- .dast_conf
variables:
DAST_USERNAME: "user4"
DAST_ONLY_INCLUDE_RULES: "90035"
script:
- /analyze
dast:server-side-template-inj-blind:
extends:
- .dast_conf
variables:
DAST_USERNAME: "user5"
DAST_ONLY_INCLUDE_RULES: "90035"
script:
- /analyze
dast:session-fixation:
extends:
- .dast_conf
variables:
DAST_USERNAME: "user6"
DAST_ONLY_INCLUDE_RULES: "40013"
script:
- /analyze
dast:xss-dombased:
extends:
- .dast_conf
variables:
DAST_USERNAME: "user10"
DAST_ONLY_INCLUDE_RULES: "40026"
script:
- /analyze

View file

@ -1,203 +0,0 @@
default:
interruptible: true
stages:
- prepare
- deploy
- post-deploy
- qa
- post-qa
- dast
include:
- local: .gitlab/ci/global.gitlab-ci.yml
- local: .gitlab/ci/review-apps/rules.gitlab-ci.yml
- local: .gitlab/ci/review-apps/qa.gitlab-ci.yml
- local: .gitlab/ci/review-apps/dast.gitlab-ci.yml
- local: .gitlab/ci/review-apps/dast-api.gitlab-ci.yml
.base-before_script: &base-before_script
- source ./scripts/utils.sh
- source ./scripts/review_apps/review-apps.sh
dont-interrupt-me:
extends: .rules:dont-interrupt
stage: prepare
interruptible: false
script:
- echo "This jobs makes sure this pipeline won't be interrupted! See https://docs.gitlab.com/ee/ci/yaml/#interruptible."
review-build-cng-env:
extends:
- .default-retry
- .review:rules:review-build-cng
image: ${GITLAB_DEPENDENCY_PROXY_ADDRESS}ruby:${RUBY_VERSION}-alpine3.16
stage: prepare
needs:
# We need this job because we need its `cached-assets-hash.txt` artifact, so that we can pass the assets image tag to the downstream CNG pipeline.
- pipeline: $PARENT_PIPELINE_ID
job: build-assets-image
variables:
BUILD_ENV: build.env
before_script:
- source ./scripts/utils.sh
- install_gitlab_gem
script:
- 'ruby -r./scripts/trigger-build.rb -e "puts Trigger.variables_for_env_file(Trigger::CNG.new.variables)" > $BUILD_ENV'
- echo "GITLAB_ASSETS_TAG=$(assets_image_tag)" >> $BUILD_ENV
- ruby -e 'puts "FULL_RUBY_VERSION=#{RUBY_VERSION}"' >> build.env
- cat $BUILD_ENV
artifacts:
reports:
dotenv: $BUILD_ENV
paths:
- $BUILD_ENV
expire_in: 7 days
when: always
review-build-cng:
extends: .review:rules:review-build-cng
stage: prepare
needs: ["review-build-cng-env"]
inherit:
variables: false
variables:
TOP_UPSTREAM_SOURCE_PROJECT: "${TOP_UPSTREAM_SOURCE_PROJECT}"
TOP_UPSTREAM_SOURCE_REF: "${TOP_UPSTREAM_SOURCE_REF}"
TOP_UPSTREAM_SOURCE_JOB: "${TOP_UPSTREAM_SOURCE_JOB}"
TOP_UPSTREAM_SOURCE_SHA: "${TOP_UPSTREAM_SOURCE_SHA}"
TOP_UPSTREAM_MERGE_REQUEST_PROJECT_ID: "${TOP_UPSTREAM_MERGE_REQUEST_PROJECT_ID}"
TOP_UPSTREAM_MERGE_REQUEST_IID: "${TOP_UPSTREAM_MERGE_REQUEST_IID}"
GITLAB_REF_SLUG: "${GITLAB_REF_SLUG}"
# CNG pipeline specific variables
GITLAB_VERSION: "${GITLAB_VERSION}"
GITLAB_TAG: "${GITLAB_TAG}"
GITLAB_ASSETS_TAG: "${GITLAB_ASSETS_TAG}"
FORCE_RAILS_IMAGE_BUILDS: "${FORCE_RAILS_IMAGE_BUILDS}"
CE_PIPELINE: "${CE_PIPELINE}" # Based on https://docs.gitlab.com/ee/ci/jobs/job_control.html#check-if-a-variable-exists, `if: '$CE_PIPELINE'` will evaluate to `false` when this variable is empty
EE_PIPELINE: "${EE_PIPELINE}" # Based on https://docs.gitlab.com/ee/ci/jobs/job_control.html#check-if-a-variable-exists, `if: '$EE_PIPELINE'` will evaluate to `false` when this variable is empty
GITLAB_ELASTICSEARCH_INDEXER_VERSION: "${GITLAB_ELASTICSEARCH_INDEXER_VERSION}"
GITLAB_KAS_VERSION: "${GITLAB_KAS_VERSION}"
GITLAB_METRICS_EXPORTER_VERSION: "${GITLAB_METRICS_EXPORTER_VERSION}"
GITLAB_PAGES_VERSION: "${GITLAB_PAGES_VERSION}"
GITLAB_SHELL_VERSION: "${GITLAB_SHELL_VERSION}"
GITLAB_WORKHORSE_VERSION: "${GITLAB_WORKHORSE_VERSION}"
GITALY_SERVER_VERSION: "${GITALY_SERVER_VERSION}"
RUBY_VERSION: "${FULL_RUBY_VERSION}"
trigger:
project: gitlab-org/build/CNG-mirror
branch: $TRIGGER_BRANCH
strategy: depend
.review-workflow-base:
image: ${REVIEW_APPS_IMAGE}
retry:
max: 2 # This is confusing but this means "3 runs at max"
variables:
HOST_SUFFIX: "${CI_ENVIRONMENT_SLUG}"
DOMAIN: "-${CI_ENVIRONMENT_SLUG}.${REVIEW_APPS_DOMAIN}"
GITLAB_HELM_CHART_REF: "febc4ad69acb7bba0eeb4a62daa577d0b7c3ee71" # 6.9.1: https://gitlab.com/gitlab-org/charts/gitlab/-/commit/febc4ad69acb7bba0eeb4a62daa577d0b7c3ee71
environment:
name: review/${CI_COMMIT_REF_SLUG}${SCHEDULE_TYPE} # No separator for SCHEDULE_TYPE so it's compatible as before and looks nice without it
url: https://gitlab-${CI_ENVIRONMENT_SLUG}.${REVIEW_APPS_DOMAIN}
on_stop: trigger-review-stop
review-deploy:
extends:
- .review-workflow-base
- .review:rules:review-deploy
stage: deploy
image: ${GITLAB_DEPENDENCY_PROXY_ADDRESS}dtzar/helm-kubectl:3.9.3
needs:
- review-build-cng
- review-delete-deployment # We always want to start from a clean slate (i.e. no helm release, no k8s namespace)
cache:
key: "review-deploy-dependencies-charts-${GITLAB_HELM_CHART_REF}-v1"
paths:
- "gitlab-${GITLAB_HELM_CHART_REF}"
environment:
action: start
before_script:
- export GITLAB_SHELL_VERSION=$(<GITLAB_SHELL_VERSION)
- export GITALY_VERSION=$(<GITALY_SERVER_VERSION)
- export GITLAB_WORKHORSE_VERSION=$(<GITLAB_WORKHORSE_VERSION)
- echo "${CI_ENVIRONMENT_URL}" > environment_url.txt
- echo "QA_GITLAB_URL=${CI_ENVIRONMENT_URL}" > environment.env
- *base-before_script
- !reference [".use-kube-context", before_script]
script:
- run_timed_command "retry delete_helm_release"
- run_timed_command "check_kube_domain"
- run_timed_command "download_chart"
- run_timed_command "deploy" || (display_deployment_debug && exit 1)
- run_timed_command "verify_deploy" || (display_deployment_debug && exit 1)
- run_timed_command "disable_sign_ups" || (display_deployment_debug && exit 1)
- run_timed_command "verify_commit_sha" || (display_deployment_debug && exit 1)
after_script:
# Run seed-dast-test-data.sh only when DAST_RUN is set to true. This is to pupulate review app with data for DAST scan.
# Set DAST_RUN to true when jobs are manually scheduled.
- if [ "$DAST_RUN" == "true" ]; then source scripts/review_apps/seed-dast-test-data.sh; TRACE=1 trigger_proj_user_creation; fi
artifacts:
paths:
- environment_url.txt
- curl-logs/
reports:
dotenv: environment.env
expire_in: 7 days
when: always
review-deploy-sample-projects:
extends:
- .review-workflow-base
- .review:rules:review-deploy
stage: deploy
needs: ["review-deploy"]
environment:
action: prepare
before_script:
- export GITLAB_SHELL_VERSION=$(<GITLAB_SHELL_VERSION)
- export GITALY_VERSION=$(<GITALY_SERVER_VERSION)
- export GITLAB_WORKHORSE_VERSION=$(<GITLAB_WORKHORSE_VERSION)
- echo "${CI_ENVIRONMENT_URL}" > environment_url.txt
- *base-before_script
- !reference [".use-kube-context", before_script]
script:
- date
- create_sample_projects
.review-stop-base:
extends: .review-workflow-base
environment:
action: stop
variables:
# We're cloning the repo instead of downloading the script for now
# because some repos are private and CI_JOB_TOKEN cannot access files.
# See https://gitlab.com/gitlab-org/gitlab/issues/191273
GIT_DEPTH: 1
review-delete-deployment:
extends:
- .review-stop-base
- .review:rules:review-delete-deployment
dependencies: []
stage: prepare
before_script:
- source ./scripts/utils.sh
- source ./scripts/review_apps/review-apps.sh
- !reference [".use-kube-context", before_script]
script:
- retry delete_helm_release
trigger-review-stop:
extends:
- .review-stop-base
- .review:rules:trigger-review-stop
stage: deploy
needs: []
before_script:
- source ./scripts/utils.sh
- install_gitlab_gem
script:
- review_stop_job_id="$(scripts/api/get_job_id.rb --pipeline-id "${PARENT_PIPELINE_ID}" --job-name "review-stop")"
- |
curl --request POST --header "Private-Token: ${PROJECT_TOKEN_FOR_CI_SCRIPTS_API_USAGE}" "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/jobs/${review_stop_job_id}/play"

View file

@ -1,182 +0,0 @@
include:
- project: gitlab-org/quality/pipeline-common
ref: 5.1.1
file:
- /ci/base.gitlab-ci.yml
- /ci/allure-report.yml
- /ci/knapsack-report.yml
- template: Verify/Browser-Performance.gitlab-ci.yml
.test-variables:
variables:
QA_GENERATE_ALLURE_REPORT: "true"
QA_CAN_TEST_PRAEFECT: "false"
GITLAB_USERNAME: "root"
GITLAB_PASSWORD: "${REVIEW_APPS_ROOT_PASSWORD}"
GITLAB_ADMIN_USERNAME: "root"
GITLAB_ADMIN_PASSWORD: "${REVIEW_APPS_ROOT_PASSWORD}"
GITLAB_QA_ADMIN_ACCESS_TOKEN: "${REVIEW_APPS_ROOT_TOKEN}"
GITHUB_ACCESS_TOKEN: "${QA_GITHUB_ACCESS_TOKEN}"
.bundle-base:
extends:
- .qa-cache
image: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images/debian-bullseye-ruby-${RUBY_VERSION}:bundler-2.3
before_script:
- cd qa && bundle install
.review-qa-base:
image: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images/debian-bullseye-ruby-${RUBY_VERSION}:bundler-2.3-git-2.36-lfs-2.9-chrome-${CHROME_VERSION}-docker-${DOCKER_VERSION}-gcloud-383-kubectl-1.23
extends:
- .use-docker-in-docker
- .bundle-base
- .test-variables
stage: qa
needs:
- review-deploy
- download-knapsack-report
variables:
GIT_LFS_SKIP_SMUDGE: 1
WD_INSTALL_DIR: /usr/local/bin
RSPEC_REPORT_OPTS: --force-color --order random --format documentation --format RspecJunitFormatter --out tmp/rspec-${CI_JOB_ID}.xml
script:
- QA_COMMAND="bundle exec bin/qa ${QA_SCENARIO} ${QA_GITLAB_URL} -- ${QA_TESTS} ${RSPEC_REPORT_OPTS}"
- echo "Running - '${QA_COMMAND}'"
- eval "$QA_COMMAND"
after_script:
- |
echo "Sentry errors for the current review-app test run can be found via following url:"
echo "https://sentry.gitlab.net/gitlab/gitlab-review-apps/releases/$(echo "${CI_COMMIT_SHA}" | cut -c1-11)/all-events/."
artifacts:
paths:
- qa/tmp
reports:
junit: qa/tmp/rspec-*.xml
expire_in: 7 days
when: always
# Store knapsack report as artifact so the same report is reused across all jobs
download-knapsack-report:
extends:
- .bundle-base
- .rules:prepare-report
stage: prepare
script:
- bundle exec rake "knapsack:download[qa]"
allow_failure: true
artifacts:
paths:
- qa/knapsack/review-qa-*.json
expire_in: 1 day
review-qa-smoke:
extends:
- .review-qa-base
- .rules:qa-smoke
variables:
QA_SCENARIO: Test::Instance::Smoke
QA_RUN_TYPE: review-qa-smoke
retry: 1
review-qa-blocking:
extends:
- .review-qa-base
- .rules:qa-blocking
variables:
QA_SCENARIO: Test::Instance::ReviewBlocking
QA_RUN_TYPE: review-qa-blocking
retry: 1
review-qa-blocking-parallel:
extends:
- review-qa-blocking
- .rules:qa-blocking-parallel
parallel: 10
review-qa-non-blocking:
extends:
- .review-qa-base
- .rules:qa-non-blocking
variables:
QA_SCENARIO: Test::Instance::ReviewNonBlocking
QA_RUN_TYPE: review-qa-non-blocking
when: manual
allow_failure: true
review-qa-non-blocking-parallel:
extends:
- review-qa-non-blocking
- .rules:qa-non-blocking-parallel
parallel: 5
browser_performance:
extends:
- .default-retry
- .review:rules:review-performance
stage: qa
needs: ["review-deploy"]
variables:
URL: environment_url.txt
e2e-test-report:
extends:
- .generate-allure-report-base
- .rules:prepare-report
stage: post-qa
variables:
ALLURE_JOB_NAME: e2e-review-qa
ALLURE_PROJECT_PATH: $CI_PROJECT_PATH
ALLURE_RESULTS_GLOB: qa/tmp/allure-results
ALLURE_MERGE_REQUEST_IID: $CI_MERGE_REQUEST_IID
GITLAB_AUTH_TOKEN: $PROJECT_TOKEN_FOR_CI_SCRIPTS_API_USAGE
GIT_STRATEGY: none
allow_failure: true
when: always
upload-knapsack-report:
extends:
- .generate-knapsack-report-base
- .bundle-base
stage: post-qa
variables:
QA_KNAPSACK_REPORT_FILE_PATTERN: $CI_PROJECT_DIR/qa/tmp/knapsack/*/*.json
delete-test-resources:
extends:
- .bundle-base
- .rules:prepare-report
stage: post-qa
variables:
QA_TEST_RESOURCES_FILE_PATTERN: $CI_PROJECT_DIR/qa/tmp/test-resources-*.json
GITLAB_QA_ACCESS_TOKEN: $REVIEW_APPS_ROOT_TOKEN
script:
- export GITLAB_ADDRESS="$QA_GITLAB_URL"
- bundle exec rake "test_resources:delete[$QA_TEST_RESOURCES_FILE_PATTERN]"
allow_failure: true
when: always
notify-slack:
extends:
- .notify-slack-qa
- .qa-cache
- .rules:main-run
stage: post-qa
variables:
RUN_WITH_BUNDLE: "true"
QA_PATH: qa
ALLURE_JOB_NAME: e2e-review-qa
SLACK_ICON_EMOJI: ci_failing
STATUS_SYM: ☠️
STATUS: failed
TYPE: "(review-app) "
when: on_failure
script:
- bundle exec prepare-stage-reports --input-files "${CI_PROJECT_DIR}/qa/tmp/rspec-*.xml"
- !reference [.notify-slack-qa, script]
export-test-metrics:
extends:
- .bundle-base
- .rules:main-run
stage: post-qa
when: always
script:
- bundle exec rake "ci:export_test_metrics[tmp/test-metrics-*.json]"

View file

@ -1,170 +0,0 @@
# ------------------------------------------
# Conditions
# ------------------------------------------
# Specific specs passed
.specific-specs: &specific-specs
if: $QA_TESTS != ""
# No specific specs passed
.all-specs: &all-specs
if: $QA_TESTS == ""
# No specific specs in mr pipeline
.all-specs-mr: &all-specs-mr
if: '($CI_MERGE_REQUEST_EVENT_TYPE == "merged_result" || $CI_MERGE_REQUEST_EVENT_TYPE == "detached") && $QA_TESTS == ""'
when: manual
# Triggered by change pattern
.app-changes: &app-changes
if: $APP_CHANGE_TRIGGER == "true"
# Run all tests when framework changes present or explicitly enabled full suite execution
.qa-run-all-tests: &qa-run-all-tests
if: $QA_FRAMEWORK_CHANGES == "true" || $QA_RUN_ALL_TESTS == "true" || $QA_RUN_ALL_E2E_LABEL == "true"
.default-branch: &default-branch
if: $CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH
.if-merge-request: &if-merge-request
if: '$CI_MERGE_REQUEST_EVENT_TYPE == "merged_result" || $CI_MERGE_REQUEST_EVENT_TYPE == "detached"'
.if-merge-request-labels-run-review-app: &if-merge-request-labels-run-review-app
if: '$CI_MERGE_REQUEST_LABELS =~ /pipeline:run-review-app/'
.if-dot-com-ee-schedule-nightly-child-pipeline: &if-dot-com-ee-schedule-nightly-child-pipeline
if: '$CI_SERVER_HOST == "gitlab.com" && $CI_PROJECT_PATH == "gitlab-org/gitlab" && $CI_PIPELINE_SOURCE == "parent_pipeline" && $SCHEDULE_TYPE == "nightly"'
# ------------------------------------------
# Changes patterns
# ------------------------------------------
.ci-review-patterns: &ci-review-patterns
- ".gitlab-ci.yml"
- ".gitlab/ci/frontend.gitlab-ci.yml"
- ".gitlab/ci/build-images.gitlab-ci.yml"
- ".gitlab/ci/review.gitlab-ci.yml"
- ".gitlab/ci/review-apps/**/*"
- "scripts/review_apps/**/*"
- "scripts/trigger-build.rb"
- "{,ee/,jh/}{bin,config}/**/*.rb"
# ------------------------------------------
# Conditions set
# ------------------------------------------
.qa-manual: &qa-manual
when: manual
allow_failure: true
variables:
QA_TESTS: ""
.never-when-qa-run-all-tests-or-no-specific-specs:
- <<: *qa-run-all-tests
when: never
- <<: *all-specs
when: never
.never-when-specific-specs-always-when-qa-run-all-tests:
- *qa-run-all-tests
- <<: *specific-specs
when: manual
allow_failure: true
variables:
QA_TESTS: ""
# ------------------------------------------
# Prepare
# ------------------------------------------
.rules:dont-interrupt:
rules:
- if: $CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH
allow_failure: true
- if: $CI_MERGE_REQUEST_IID
when: manual
allow_failure: true
.review:rules:review-build-cng:
rules:
- when: always
.review:rules:review-delete-deployment:
rules:
- when: on_success
# ------------------------------------------
# Deploy
# ------------------------------------------
.review:rules:review-deploy:
rules:
- when: on_success
.review:rules:trigger-review-stop:
rules:
- when: manual
allow_failure: true
# ------------------------------------------
# Test
# ------------------------------------------
.rules:qa-smoke:
rules:
# always trigger smoke suite if review pipeline got triggered by specific changes in application code
- <<: *app-changes
variables:
QA_TESTS: "" # unset QA_TESTS even if specific tests were inferred from stage label
- *qa-run-all-tests
- if: $QA_SUITES =~ /Test::Instance::Smoke/
- *qa-manual
.rules:qa-blocking:
rules:
- <<: *app-changes
when: never
- !reference [.never-when-qa-run-all-tests-or-no-specific-specs]
- if: $QA_SUITES =~ /Test::Instance::ReviewBlocking/
.rules:qa-blocking-parallel:
rules:
# always trigger blocking suite if review pipeline got triggered by specific changes in application code
- <<: *app-changes
variables:
QA_TESTS: "" # unset QA_TESTS even if specific tests were inferred from stage label
- !reference [.never-when-specific-specs-always-when-qa-run-all-tests]
- if: $QA_SUITES =~ /Test::Instance::ReviewBlocking/
.rules:qa-non-blocking:
rules:
- !reference [.never-when-qa-run-all-tests-or-no-specific-specs]
- if: $QA_SUITES =~ /Test::Instance::ReviewNonBlocking/
.rules:qa-non-blocking-parallel:
rules:
- !reference [.never-when-specific-specs-always-when-qa-run-all-tests]
- *all-specs-mr # set full suite to manual when no specific specs passed in mr
- if: $QA_SUITES =~ /Test::Instance::ReviewNonBlocking/
.review:rules:review-performance:
rules:
- if: '$DAST_RUN == "true"' # Skip this job when DAST is run
when: never
- <<: *if-merge-request-labels-run-review-app # we explicitly don't allow the job to fail in that case
- <<: *if-merge-request # we explicitly don't allow the job to fail in that case
changes: *ci-review-patterns
- when: on_success
allow_failure: true
# ------------------------------------------
# DAST
# ------------------------------------------
.reports:rules:schedule-dast:
rules:
- if: '$DAST_DISABLED || $GITLAB_FEATURES !~ /\bdast\b/'
when: never
- <<: *if-dot-com-ee-schedule-nightly-child-pipeline
# ------------------------------------------
# Prepare/Report
# ------------------------------------------
.rules:prepare-report:
rules:
- when: always
.rules:main-run:
rules:
- *default-branch

View file

@ -2,134 +2,231 @@ review-cleanup:
extends:
- .default-retry
- .review:rules:review-cleanup
image: ${REVIEW_APPS_IMAGE}
image: registry.gitlab.com/gitlab-org/gitlab-build-images:gitlab-helm3-kubectl1.14
stage: prepare
needs: []
environment:
name: review/regular-cleanup
action: access
variables:
GIT_DEPTH: 1
name: review/auto-cleanup
action: stop
before_script:
- source scripts/utils.sh
- !reference [".use-kube-context", before_script]
- source scripts/review_apps/gcp_cleanup.sh
- install_gitlab_gem
- setup_gcloud
- setup_gcp_dependencies
script:
- scripts/review_apps/automated_cleanup.rb --dry-run="${DRY_RUN:-false}" || (scripts/slack review-apps-monitoring "☠️ \`${CI_JOB_NAME}\` failed! ☠️ See ${CI_JOB_URL} - <https://gitlab.com/gitlab-org/quality/engineering-productivity/team/-/blob/main/runbooks/review-apps.md#review-cleanup-job-failed|📗 RUNBOOK 📕>" warning "GitLab Bot" && exit 1);
- ruby -rrubygems scripts/review_apps/automated_cleanup.rb
- gcp_cleanup
.base-before_script: &base-before_script
- source ./scripts/utils.sh
- source ./scripts/review_apps/review-apps.sh
- install_api_client_dependencies_with_apk
review-build-cng:
extends:
- .default-retry
- .review:rules:review-build-cng
image: ${GITLAB_DEPENDENCY_PROXY}ruby:2.7-alpine
stage: review-prepare
before_script:
- source ./scripts/utils.sh
- install_gitlab_gem
needs:
- job: compile-production-assets
artifacts: false
script:
- ./scripts/trigger-build cng
.review-workflow-base:
extends:
- .default-retry
image: registry.gitlab.com/gitlab-org/gitlab-build-images:gitlab-helm3-kubectl1.14
variables:
HOST_SUFFIX: "${CI_ENVIRONMENT_SLUG}"
DOMAIN: "-${CI_ENVIRONMENT_SLUG}.${REVIEW_APPS_DOMAIN}"
GITLAB_HELM_CHART_REF: "v4.6.3"
environment:
name: review/${CI_COMMIT_REF_SLUG}${FREQUENCY}
url: https://gitlab-${CI_ENVIRONMENT_SLUG}.${REVIEW_APPS_DOMAIN}
on_stop: review-stop
auto_stop_in: 48 hours
review-deploy:
extends:
- .review-workflow-base
- .review:rules:review-deploy
stage: review
needs: ["review-build-cng"]
resource_group: "review/${CI_COMMIT_REF_NAME}"
before_script:
- export GITLAB_SHELL_VERSION=$(<GITLAB_SHELL_VERSION)
- export GITALY_VERSION=$(<GITALY_SERVER_VERSION)
- export GITLAB_WORKHORSE_VERSION=$(<GITLAB_WORKHORSE_VERSION)
- echo "${CI_ENVIRONMENT_URL}" > environment_url.txt
- *base-before_script
script:
- check_kube_domain
- ensure_namespace
- install_external_dns
- download_chart
- date
- deploy || (display_deployment_debug && exit 1)
- disable_sign_ups || (delete_release && exit 1)
after_script:
# Run seed-dast-test-data.sh only when DAST_RUN is set to true. This is to pupulate review app with data for DAST scan.
# Set DAST_RUN to true when jobs are manually scheduled.
- if [ "$DAST_RUN" == "true" ]; then source scripts/review_apps/seed-dast-test-data.sh; TRACE=1 trigger_proj_user_creation; fi
artifacts:
paths: [environment_url.txt]
expire_in: 7 days
when: always
.review-stop-base:
extends: .review-workflow-base
environment:
action: stop
dependencies: []
variables:
# We're cloning the repo instead of downloading the script for now
# because some repos are private and CI_JOB_TOKEN cannot access files.
# See https://gitlab.com/gitlab-org/gitlab/issues/191273
GIT_DEPTH: 1
before_script:
- *base-before_script
review-stop-failed-deployment:
extends:
- .review-stop-base
- .review:rules:review-stop-failed-deployment
stage: prepare
script:
- delete_failed_release
review-stop:
extends:
- review-cleanup
- .review-stop-base
- .review:rules:review-stop
environment:
name: review/${CI_COMMIT_REF_SLUG}${SCHEDULE_TYPE} # No separator for SCHEDULE_TYPE so it's compatible as before and looks nice without it
action: stop
resource_group: review/${CI_COMMIT_REF_SLUG}${SCHEDULE_TYPE} # CI_ENVIRONMENT_SLUG is not available here and we want this to be the same as the environment
before_script:
- source ./scripts/utils.sh
- source ./scripts/review_apps/review-apps.sh
- !reference [".use-kube-context", before_script]
stage: post-qa
script:
- retry delete_helm_release
- delete_release
.base-review-checks:
.review-qa-base:
extends:
- .default-retry
image: ${REVIEW_APPS_IMAGE}
stage: prepare
- .use-docker-in-docker
image: registry.gitlab.com/gitlab-org/gitlab-build-images:gitlab-qa-alpine-ruby-2.7
stage: qa
needs: ["review-deploy"]
variables:
QA_ARTIFACTS_DIR: "${CI_PROJECT_DIR}/qa"
QA_CAN_TEST_GIT_PROTOCOL_V2: "false"
QA_DEBUG: "true"
GITLAB_USERNAME: "root"
GITLAB_PASSWORD: "${REVIEW_APPS_ROOT_PASSWORD}"
GITLAB_ADMIN_USERNAME: "root"
GITLAB_ADMIN_PASSWORD: "${REVIEW_APPS_ROOT_PASSWORD}"
GITHUB_ACCESS_TOKEN: "${REVIEW_APPS_QA_GITHUB_ACCESS_TOKEN}"
EE_LICENSE: "${REVIEW_APPS_EE_LICENSE}"
SIGNUP_DISABLED: "true"
before_script:
- source scripts/utils.sh
- setup_gcloud
- !reference [".use-kube-context", before_script]
- export QA_IMAGE="${CI_REGISTRY}/${CI_PROJECT_PATH}/gitlab-ee-qa:${CI_COMMIT_REF_SLUG}"
- export CI_ENVIRONMENT_URL="$(cat environment_url.txt)"
- echo "${CI_ENVIRONMENT_URL}"
- echo "${QA_IMAGE}"
- *base-before_script
- gem install gitlab-qa --no-document ${GITLAB_QA_VERSION:+ --version ${GITLAB_QA_VERSION}}
artifacts:
paths:
- ./qa/gitlab-qa-run-*
expire_in: 7 days
when: always
review-k8s-resources-count-checks:
review-qa-smoke:
extends:
- .base-review-checks
- .review:rules:review-k8s-resources-count-checks
needs:
- job: review-cleanup
optional: true
environment:
name: review/k8s-resources-count-checks
action: verify
- .review-qa-base
- .review:rules:review-qa-smoke
script:
- scripts/review_apps/k8s-resources-count-checks.sh || (scripts/slack review-apps-monitoring "☠️ \`${CI_JOB_NAME}\` failed! ☠️ See ${CI_JOB_URL} - <https://gitlab.com/gitlab-org/quality/engineering-productivity/team/-/blob/main/runbooks/review-apps.md#review-k8s-resources-count-checks-job-failed|📗 RUNBOOK 📕>" warning "GitLab Bot" && exit 1);
- gitlab-qa Test::Instance::Smoke "${QA_IMAGE}" "${CI_ENVIRONMENT_URL}"
review-gcp-quotas-checks:
review-qa-all:
extends:
- .base-review-checks
- .review:rules:review-gcp-quotas-checks
needs: []
environment:
name: review/gcp-quotas-checks
action: verify
- .review-qa-base
- .review:rules:review-qa-all
parallel: 5
script:
- ruby scripts/review_apps/gcp-quotas-checks.rb || (scripts/slack review-apps-monitoring "☠️ \`${CI_JOB_NAME}\` failed! ☠️ See ${CI_JOB_URL} - <https://gitlab.com/gitlab-org/quality/engineering-productivity/team/-/blob/main/runbooks/review-apps.md#review-gcp-quotas-checks-job-failed|📗 RUNBOOK 📕>" warning "GitLab Bot" && exit 1);
- export KNAPSACK_REPORT_PATH=knapsack/master_report.json
- export KNAPSACK_TEST_FILE_PATTERN=qa/specs/features/**/*_spec.rb
- gitlab-qa Test::Instance::Any "${QA_IMAGE}" "${CI_ENVIRONMENT_URL}" -- --format RspecJunitFormatter --out tmp/rspec-${CI_JOB_ID}.xml --format html --out tmp/rspec.htm --color --format documentation
start-review-app-pipeline:
review-performance:
extends:
- .review:rules:start-review-app-pipeline
resource_group: review/${CI_COMMIT_REF_SLUG}${SCHEDULE_TYPE} # CI_ENVIRONMENT_SLUG is not available here and we want this to be the same as the environment
stage: review
needs:
- job: e2e-test-pipeline-generate
- job: build-assets-image
artifacts: false
# We do not want to have ALL global variables passed as trigger variables,
# as they cannot be overridden. See this issue for more context:
#
# https://gitlab.com/gitlab-org/gitlab/-/issues/387183
inherit:
variables:
- CHROME_VERSION
- REGISTRY_GROUP
- REGISTRY_HOST
- REVIEW_APPS_DOMAIN
- REVIEW_APPS_GCP_PROJECT
- REVIEW_APPS_GCP_REGION
- REVIEW_APPS_IMAGE
- RUBY_VERSION
- .default-retry
- .review:rules:review-performance
image:
name: sitespeedio/sitespeed.io
entrypoint: [""]
stage: qa
needs: ["review-deploy"]
before_script:
- export CI_ENVIRONMENT_URL="$(cat environment_url.txt)"
- echo "${CI_ENVIRONMENT_URL}"
- mkdir -p gitlab-exporter
- wget -O ./gitlab-exporter/index.js https://gitlab.com/gitlab-org/gl-performance/raw/master/index.js
- mkdir -p sitespeed-results
script:
- /start.sh --plugins.add ./gitlab-exporter --outputFolder sitespeed-results "${CI_ENVIRONMENT_URL}"
after_script:
- mv sitespeed-results/data/performance.json performance.json
artifacts:
paths:
- sitespeed-results/
reports:
performance: performance.json
expire_in: 31d
# These variables are set in the pipeline schedules.
# They need to be explicitly passed on to the child pipeline.
# https://docs.gitlab.com/ee/ci/pipelines/multi_project_pipelines.html#pass-cicd-variables-to-a-downstream-pipeline-by-using-the-variables-keyword
parallel-spec-reports:
extends:
- .review:rules:review-qa-all
image: ${GITLAB_DEPENDENCY_PROXY}ruby:2.7-alpine
stage: post-qa
needs: ["review-qa-all"]
variables:
# This is needed by `review-build-cng-env` (`.gitlab/ci/review-apps/main.gitlab-ci.yml`).
PARENT_PIPELINE_ID: $CI_PIPELINE_ID
SCHEDULE_TYPE: $SCHEDULE_TYPE
DAST_RUN: $DAST_RUN
SKIP_MESSAGE: Skipping review-app due to mr containing only quarantine changes!
trigger:
strategy: depend
include:
- artifact: review-app-pipeline.yml
job: e2e-test-pipeline-generate
NEW_PARALLEL_SPECS_REPORT: qa/report-new.html
BASE_ARTIFACT_URL: "${CI_PROJECT_URL}/-/jobs/${CI_JOB_ID}/artifacts/file/qa/"
script:
- apk add --update build-base libxml2-dev libxslt-dev && rm -rf /var/cache/apk/*
- gem install nokogiri --no-document
- cd qa/gitlab-qa-run-*/gitlab-*
- ARTIFACT_DIRS=$(pwd |rev| awk -F / '{print $1,$2}' | rev | sed s_\ _/_)
- cd -
- '[[ -f $NEW_PARALLEL_SPECS_REPORT ]] || echo "{}" > ${NEW_PARALLEL_SPECS_REPORT}'
- scripts/merge-html-reports ${NEW_PARALLEL_SPECS_REPORT} ${BASE_ARTIFACT_URL}${ARTIFACT_DIRS} qa/gitlab-qa-run-*/**/rspec.htm
artifacts:
when: always
paths:
- qa/report-new.html
- qa/gitlab-qa-run-*
reports:
junit: qa/gitlab-qa-run-*/**/rspec-*.xml
expire_in: 31d
danger-review:
extends:
- .default-retry
- .ruby-node-cache
- .danger-review-cache
- .review:rules:danger
stage: test
needs: []
before_script:
- source scripts/utils.sh
- bundle_install_script "--with danger"
- yarn_install_script
- run_timed_command "retry yarn install --frozen-lockfile"
script:
# ${DANGER_DANGERFILE} is used by Jihulab for customizing danger support: https://jihulab.com/gitlab-cn/gitlab/-/blob/main-jh/jh/.gitlab-ci.yml
- >
if [ -z "$DANGER_GITLAB_API_TOKEN" ]; then
run_timed_command danger_as_local
# Force danger to skip CI source GitLab and fallback to "local only git repo".
unset GITLAB_CI
# We need to base SHA to help danger determine the base commit for this shallow clone.
run_timed_command "bundle exec danger dry_run --fail-on-errors=true --verbose --base='$CI_MERGE_REQUEST_DIFF_BASE_SHA'"
else
danger_id=$(echo -n ${DANGER_GITLAB_API_TOKEN} | md5sum | awk '{print $1}' | cut -c5-10)
run_timed_command "bundle exec danger --fail-on-errors=true --verbose --danger_id=\"${danger_id}\" --dangerfile=\"${DANGER_DANGERFILE:-Dangerfile}\""
run_timed_command "bundle exec danger --fail-on-errors=true --verbose"
fi
danger-review-local:
extends:
- danger-review
- .review:rules:danger-local
script:
- run_timed_command danger_as_local

File diff suppressed because it is too large Load diff

View file

@ -3,67 +3,56 @@
cache gems:
extends:
- .default-retry
- .ruby-cache
- .rails-cache
- .default-before_script
- .setup:rules:cache-gems
stage: prepare
needs: []
stage: test
needs: ["setup-test-env"]
variables:
BUNDLE_WITHOUT: ""
BUNDLE_WITH: "production:development:test"
BUNDLE_INSTALL_FLAGS: --with=production --with=development --with=test --jobs=2 --path=vendor --retry=3 --quiet
SETUP_DB: "false"
script:
- echo -e "\e[0Ksection_start:`date +%s`:bundle-package[collapsed=true]\r\e[0KPackaging gems"
- bundle config set cache_all true
- run_timed_command "bundle package --all-platforms"
- echo -e "\e[0Ksection_end:`date +%s`:bundle-package\r\e[0K"
- bundle package --all --all-platforms
artifacts:
paths:
- vendor/cache
expire_in: 31d
.predictive-job:
.minimal-job:
extends:
- .default-retry
needs: []
.absolutely-predictive-job:
extends:
- .predictive-job
image: ${GITLAB_DEPENDENCY_PROXY_ADDRESS}alpine:edge
dont-interrupt-me:
extends: .setup:rules:dont-interrupt-me
stage: sync
image: ${GITLAB_DEPENDENCY_PROXY}alpine:edge
interruptible: false
variables:
GIT_STRATEGY: none
dont-interrupt-me:
extends:
- .absolutely-predictive-job
- .setup:rules:dont-interrupt-me
stage: sync
interruptible: false
script:
- echo "This jobs makes sure this pipeline won't be interrupted! See https://docs.gitlab.com/ee/ci/yaml/#interruptible."
gitlab_git_test:
extends:
- .predictive-job
- .minimal-job
- .setup:rules:gitlab_git_test
stage: test
script:
- spec/support/prepare-gitlab-git-test-for-commit --check-for-changes
verify-ruby-3.0:
no_ee_check:
extends:
- .absolutely-predictive-job
- .setup:rules:verify-ruby-3.0
stage: prepare
- .minimal-job
- .setup:rules:no_ee_check
stage: test
script:
- echo 'Please remove label ~"pipeline:run-in-ruby2" so we do test against Ruby 3.0 (default version) before merging the merge request'
- exit 1
- scripts/no-ee-check
verify-tests-yml:
extends:
- .setup:rules:verify-tests-yml
image: ${GITLAB_DEPENDENCY_PROXY_ADDRESS}ruby:${RUBY_VERSION}-alpine3.16
image: ${GITLAB_DEPENDENCY_PROXY}ruby:2.7-alpine
stage: test
needs: []
script:
@ -71,103 +60,37 @@ verify-tests-yml:
- install_tff_gem
- scripts/verify-tff-mapping
verify-approvals:
extends:
- .predictive-job
- .setup:rules:jh-contribution
script:
- source scripts/utils.sh
- install_gitlab_gem
- tooling/bin/find_app_sec_approval
generate-frontend-fixtures-mapping:
extends:
- .setup:rules:generate-frontend-fixtures-mapping
- .use-pg13
- .rails-cache
needs: ["setup-test-env"]
stage: prepare
before_script:
- !reference [.default-before_script, before_script]
- source ./scripts/rspec_helpers.sh
- section_start "gitaly-test-spawn" "Spawning Gitaly"; scripts/gitaly-test-spawn; section_end "gitaly-test-spawn"; # Do not use 'bundle exec' here
script:
- generate_frontend_fixtures_mapping
artifacts:
expire_in: 7d
paths:
- ${FRONTEND_FIXTURES_MAPPING_PATH}
detect-tests:
extends: .rails:rules:detect-tests
image: ${GITLAB_DEPENDENCY_PROXY_ADDRESS}ruby:${RUBY_VERSION}-slim
.detect-test-base:
image: ${GITLAB_DEPENDENCY_PROXY}ruby:2.7
needs: []
stage: prepare
variables:
RSPEC_TESTS_MAPPING_ENABLED: "true"
before_script:
- apt-get update && apt-get install -y curl # Not present in ruby-slim, so we add it manually
script:
- source ./scripts/utils.sh
- source ./scripts/rspec_helpers.sh
- install_gitlab_gem
- install_tff_gem
- install_activesupport_gem
- retrieve_tests_mapping
- retrieve_frontend_fixtures_mapping
- |
if [ -n "$CI_MERGE_REQUEST_IID" ]; then
mkdir -p $(dirname "$RSPEC_CHANGED_FILES_PATH")
tooling/bin/predictive_tests
filter_rspec_matched_foss_tests ${RSPEC_MATCHING_TESTS_PATH} ${RSPEC_MATCHING_TESTS_FOSS_PATH};
filter_rspec_matched_ee_tests ${RSPEC_MATCHING_TESTS_PATH} ${RSPEC_MATCHING_TESTS_EE_PATH};
echoinfo "Changed files: $(cat $RSPEC_CHANGED_FILES_PATH)";
echoinfo "Related FOSS RSpec tests: $(cat $RSPEC_MATCHING_TESTS_FOSS_PATH)";
echoinfo "Related EE RSpec tests: $(cat $RSPEC_MATCHING_TESTS_EE_PATH)";
echoinfo "Related JS files: $(cat $RSPEC_MATCHING_JS_FILES_PATH)";
fi
- 'if [ -n "$CI_MERGE_REQUEST_IID" ]; then tooling/bin/find_tests ${MATCHED_TESTS_FILE}; fi'
- 'if [ -n "$CI_MERGE_REQUEST_IID" ]; then echo "test files affected: $(cat $MATCHED_TESTS_FILE)"; fi'
artifacts:
expire_in: 7d
paths:
- ${FRONTEND_FIXTURES_MAPPING_PATH}
- ${RSPEC_CHANGED_FILES_PATH}
- ${RSPEC_MATCHING_JS_FILES_PATH}
- ${RSPEC_MATCHING_TESTS_EE_PATH}
- ${RSPEC_MATCHING_TESTS_FOSS_PATH}
- ${RSPEC_MATCHING_TESTS_PATH}
- ${RSPEC_VIEWS_INCLUDING_PARTIALS_PATH}
- ${MATCHED_TESTS_FILE}
detect-previous-failed-tests:
detect-tests:
extends:
- detect-tests
- .rails:rules:detect-previous-failed-tests
- .detect-test-base
- .rails:rules:detect-tests
variables:
PREVIOUS_FAILED_TESTS_DIR: tmp/previous_failed_tests/
script:
- source ./scripts/utils.sh
- source ./scripts/rspec_helpers.sh
- retrieve_failed_tests "${PREVIOUS_FAILED_TESTS_DIR}" "oneline" "previous"
artifacts:
expire_in: 7d
paths:
- ${PREVIOUS_FAILED_TESTS_DIR}
RSPEC_TESTS_MAPPING_ENABLED: "true"
MATCHED_TESTS_FILE: tmp/matching_tests.txt
e2e-test-pipeline-generate:
detect-tests as-if-foss:
extends:
- .qa-job-base
- .predictive-job
- .qa:rules:determine-e2e-tests
stage: prepare
- .detect-test-base
- .rails:rules:detect-tests
- .as-if-foss
variables:
ENV_FILE: $CI_PROJECT_DIR/qa_tests_vars.env
COLORIZED_LOGS: "true"
script:
- bundle exec rake "ci:detect_changes[$ENV_FILE]"
- cd $CI_PROJECT_DIR && scripts/generate-e2e-pipeline
artifacts:
expire_in: 1 day
paths:
- '*-pipeline.yml'
MATCHED_TESTS_FILE: tmp/matching_foss_tests.txt
before_script:
- '[ "$FOSS_ONLY" = "1" ] && rm -rf ee/ qa/spec/ee/ qa/qa/specs/features/ee/ qa/qa/ee/ qa/qa/ee.rb'

View file

@ -1,219 +0,0 @@
.static-analysis-base:
extends:
- .default-retry
- .default-before_script
stage: lint
needs: []
variables:
SETUP_DB: "false"
ENABLE_SPRING: "1"
# Disable warnings in browserslist which can break on backports
# https://github.com/browserslist/browserslist/blob/a287ec6/node.js#L367-L384
BROWSERSLIST_IGNORE_OLD_DATA: "true"
GRAPHQL_SCHEMA_APOLLO_FILE: "tmp/tests/graphql/gitlab_schema_apollo.graphql"
update-static-analysis-cache:
extends:
- .static-analysis-base
- .rubocop-job-cache-push
- .shared:rules:update-cache
stage: prepare
script:
# Silence cop offenses for rules with "grace period".
# This will notify Slack if offenses were silenced.
# For the moment we only cache `tmp/rubocop_cache` so we don't need to run all the tasks.
- run_timed_command "fail_on_warnings bundle exec rake rubocop:check:graceful"
static-analysis:
extends:
- .static-analysis-base
- .static-analysis-cache
- .static-analysis:rules:static-analysis
parallel: 2
script:
- yarn_install_script
- fail_on_warnings scripts/static-analysis
static-analysis as-if-foss:
extends:
- static-analysis
- .static-analysis:rules:static-analysis-as-if-foss
- .as-if-foss
static-verification-with-database:
extends:
- .static-analysis-base
- .rubocop-job-cache
- .static-analysis:rules:static-verification-with-database
- .use-pg13
script:
- bundle exec rake lint:static_verification_with_database
variables:
SETUP_DB: "true"
generate-apollo-graphql-schema:
extends:
- .static-analysis-base
- .frontend:rules:default-frontend-jobs
image:
name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:apollo
entrypoint: [""]
needs: ['graphql-schema-dump']
variables:
USE_BUNDLE_INSTALL: "false"
script:
- apollo client:download-schema --config=config/apollo.config.js ${GRAPHQL_SCHEMA_APOLLO_FILE}
artifacts:
name: graphql-schema-apollo
paths:
- "${GRAPHQL_SCHEMA_APOLLO_FILE}"
generate-apollo-graphql-schema as-if-foss:
extends:
- generate-apollo-graphql-schema
- .frontend:rules:eslint-as-if-foss
- .as-if-foss
needs: ['graphql-schema-dump as-if-foss']
eslint:
extends:
- .static-analysis-base
- .yarn-cache
- .frontend:rules:default-frontend-jobs
needs: ['generate-apollo-graphql-schema']
variables:
USE_BUNDLE_INSTALL: "false"
script:
- yarn_install_script
- run_timed_command "yarn run lint:eslint:all"
eslint as-if-foss:
extends:
- eslint
- .frontend:rules:eslint-as-if-foss
- .as-if-foss
needs: ['generate-apollo-graphql-schema as-if-foss']
haml-lint:
extends:
- .static-analysis-base
- .ruby-cache
- .static-analysis:rules:haml-lint
script:
- run_timed_command "bundle exec haml-lint --parallel app/views"
artifacts:
expire_in: 31d
when: always
paths:
- tmp/feature_flags/
haml-lint ee:
extends:
- "haml-lint"
- .static-analysis:rules:haml-lint-ee
script:
- run_timed_command "bundle exec haml-lint --parallel ee/app/views"
rubocop:
extends:
- .static-analysis-base
- .rubocop-job-cache
- .static-analysis:rules:rubocop
needs:
- job: detect-tests
optional: true
variables:
RUBOCOP_TARGET_FILES: "tmp/rubocop_target_files.txt"
script:
- |
# For non-merge request, or when RUN_ALL_RUBOCOP is 'true', run all RuboCop rules
if [ -z "${CI_MERGE_REQUEST_IID}" ] || [ "${RUN_ALL_RUBOCOP}" == "true" ]; then
# Silence cop offenses for rules with "grace period".
# We won't notify Slack if offenses were silenced to avoid frequent messages.
# Job `update-static-analysis-cache` takes care of Slack notifications every 2 hours.
unset CI_SLACK_WEBHOOK_URL
run_timed_command "fail_on_warnings bundle exec rake rubocop:check:graceful"
else
cat "${RSPEC_CHANGED_FILES_PATH}" | ruby -e 'print $stdin.read.split(" ").select { |f| File.exist?(f) }.join(" ")' > "$RUBOCOP_TARGET_FILES"
# Skip running RuboCop if there's no target files
if [ -s "${RUBOCOP_TARGET_FILES}" ]; then
run_timed_command "fail_on_warnings bundle exec rubocop --parallel --force-exclusion $(cat ${RUBOCOP_TARGET_FILES})"
else
echoinfo "Nothing interesting changed for RuboCop. Skipping."
fi
fi
qa:metadata-lint:
extends:
- .static-analysis-base
- .static-analysis:rules:qa:metadata-lint
before_script:
- !reference [.default-before_script, before_script]
- cd qa/
- bundle_install_script
script:
- run_timed_command "bundle exec bin/qa Test::Instance::All http://localhost:3000 --test-metadata-only"
- cd ..
- run_timed_command "./scripts/qa/testcases-check qa/tmp/test-metadata.json"
- run_timed_command "./scripts/qa/quarantine-types-check qa/tmp/test-metadata.json"
variables:
USE_BUNDLE_INSTALL: "false"
SETUP_DB: "false"
QA_EXPORT_TEST_METRICS: "false"
# Disable warnings in browserslist which can break on backports
# https://github.com/browserslist/browserslist/blob/a287ec6/node.js#L367-L384
BROWSERSLIST_IGNORE_OLD_DATA: "true"
artifacts:
expire_in: 31d
when: always
paths:
- qa/tmp/
feature-flags-usage:
extends:
- .static-analysis-base
- .rubocop-job-cache
- .static-analysis:rules:rubocop
script:
# We need to disable the cache for this cop since it creates files under tmp/feature_flags/*.used,
# the cache would prevent these files from being created.
- run_timed_command "fail_on_warnings bundle exec rubocop --only Gitlab/MarkUsedFeatureFlags --cache false"
artifacts:
expire_in: 31d
when: always
paths:
- tmp/feature_flags/
semgrep-appsec-custom-rules:
stage: lint
extends:
- .semgrep-appsec-custom-rules:rules
image: returntocorp/semgrep
needs: []
script:
# Required to avoid a timeout https://github.com/returntocorp/semgrep/issues/5395
- git fetch origin master
# Include/exclude list isn't ideal https://github.com/returntocorp/semgrep/issues/5399
- |
semgrep ci --gitlab-sast --metrics off --config $CUSTOM_RULES_URL \
--include app --include lib --include workhorse \
--exclude '*_test.go' --exclude spec --exclude qa > gl-sast-report.json || true
variables:
CUSTOM_RULES_URL: https://gitlab.com/gitlab-com/gl-security/appsec/sast-custom-rules/-/raw/main/appsec-pings/rules.yml
artifacts:
paths:
- gl-sast-report.json
ping-appsec-for-sast-findings:
stage: lint
image: alpine:latest
extends:
- .ping-appsec-for-sast-findings:rules
variables:
# Project Access Token bot ID for /gitlab-com/gl-security/appsec/sast-custom-rules
BOT_USER_ID: 13559989
needs:
- semgrep-appsec-custom-rules
script:
- apk add jq curl
- scripts/process_custom_semgrep_results.sh

View file

@ -1,24 +1,21 @@
.tests-metadata-state:
image: ${GITLAB_DEPENDENCY_PROXY_ADDRESS}ruby:${RUBY_VERSION}
image: ${GITLAB_DEPENDENCY_PROXY}ruby:2.7
before_script:
- source scripts/utils.sh
artifacts:
expire_in: 31d
paths:
- knapsack/
- rspec/
- crystalball/
when: always
- rspec_flaky/
- rspec_profiling/
- crystalball/packed-mapping.json.gz
retrieve-tests-metadata:
extends:
- .tests-metadata-state
- .test-metadata:rules:retrieve-tests-metadata
# We use a smaller image for this job only (update-tests-metadata compiles some gems)
image: ${GITLAB_DEPENDENCY_PROXY_ADDRESS}ruby:${RUBY_VERSION}-slim
stage: prepare
script:
- apt-get update && apt-get install -y curl # Not present in ruby-slim, so we add it manually
- install_gitlab_gem
- source ./scripts/rspec_helpers.sh
- retrieve_tests_metadata
@ -29,23 +26,22 @@ update-tests-metadata:
- .test-metadata:rules:update-tests-metadata
stage: post-test
dependencies:
- retrieve-tests-metadata
- generate-frontend-fixtures-mapping
- setup-test-env
- rspec migration pg13
- rspec-all frontend_fixture
- rspec unit pg13
- rspec integration pg13
- rspec system pg13
- rspec background_migration pg13
- rspec-ee migration pg13
- rspec-ee unit pg13
- rspec-ee integration pg13
- rspec-ee system pg13
- rspec-ee background_migration pg13
- rspec migration pg12
- rspec frontend_fixture
- rspec-ee frontend_fixture
- rspec unit pg12
- rspec integration pg12
- rspec system pg12
- rspec-ee migration pg12
- rspec-ee unit pg12
- rspec-ee integration pg12
- rspec-ee system pg12
- rspec-ee unit pg12 geo
- rspec-ee integration pg12 geo
- rspec-ee system pg12 geo
script:
- run_timed_command "retry gem install fog-aws mime-types activesupport rspec_profiling postgres-copy --no-document"
- source ./scripts/rspec_helpers.sh
- test -f "${FLAKY_RSPEC_SUITE_REPORT_PATH}" || echo -e "\e[31m" 'Consider add ~"pipeline:run-all-rspec" to run full rspec jobs' "\e[0m"
- update_tests_metadata
- update_tests_mapping

View file

@ -1,129 +0,0 @@
include:
- local: .gitlab/ci/qa-common/main.gitlab-ci.yml
- local: .gitlab/ci/qa-common/rules.gitlab-ci.yml
- local: .gitlab/ci/qa-common/variables.gitlab-ci.yml
.run-tests:
stage: test
image: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images/debian-bullseye-ruby-${RUBY_VERSION}:bundler-2.3-chrome-${CHROME_VERSION}-docker-${DOCKER_VERSION}
services:
- docker:${DOCKER_VERSION}-dind
tags:
- e2e
before_script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- sysctl -n -w fs.inotify.max_user_watches=524288
- echo "SUITE_RAN=true" > suite_status.env
variables:
DOCKER_DRIVER: overlay2
DOCKER_HOST: tcp://docker:2375
QA_GDK_IMAGE: "${CI_REGISTRY}/${CI_PROJECT_PATH}/gitlab-qa-gdk:master"
QA_GENERATE_ALLURE_REPORT: "true"
QA_CAN_TEST_PRAEFECT: "false"
QA_INTERCEPT_REQUESTS: "false"
TEST_LICENSE_MODE: $QA_TEST_LICENSE_MODE
EE_LICENSE: $QA_EE_LICENSE
GITHUB_ACCESS_TOKEN: $QA_GITHUB_ACCESS_TOKEN
GITLAB_QA_ADMIN_ACCESS_TOKEN: $QA_ADMIN_ACCESS_TOKEN
RSPEC_REPORT_OPTS: "--format QA::Support::JsonFormatter --out tmp/rspec-${CI_JOB_ID}.json --format RspecJunitFormatter --out tmp/rspec-${CI_JOB_ID}.xml --format html --out tmp/rspec-${CI_JOB_ID}.htm --color --format documentation"
timeout: 2 hours
artifacts:
when: always
paths:
- test_output
- logs
expire_in: 7 days
reports:
junit: test_output/**/rspec-*.xml
dotenv: suite_status.env
script:
- echo -e "\e[0Ksection_start:`date +%s`:pull_image\r\e[0KPull GDK QA image"
- docker pull ${QA_GDK_IMAGE}
- echo -e "\e[0Ksection_end:`date +%s`:pull_image\r\e[0K"
- echo -e "\e[0Ksection_start:`date +%s`:launch_gdk_and_tests\r\e[0KLaunch GDK and run QA tests"
- cd qa && bundle install --jobs=$(nproc) --retry=3 --quiet
- mkdir -p $CI_PROJECT_DIR/test_output $CI_PROJECT_DIR/logs/gdk $CI_PROJECT_DIR/logs/gitlab
# This command matches the permissions of the user that runs GDK inside the container.
- chown -R 1000:1000 $CI_PROJECT_DIR/test_output $CI_PROJECT_DIR/logs $CI_PROJECT_DIR/qa/knapsack
- |
docker run --rm --name gdk --add-host gdk.test:127.0.0.1 --shm-size=2gb \
--env-file <(bundle exec rake ci:env_var_name_list) \
--volume /var/run/docker.sock:/var/run/docker.sock:z \
--volume $CI_PROJECT_DIR/test_output:/home/gdk/gdk/gitlab/qa/tmp:z \
--volume $CI_PROJECT_DIR/logs/gdk:/home/gdk/gdk/log \
--volume $CI_PROJECT_DIR/logs/gitlab:/home/gdk/gdk/gitlab/log \
--volume $CI_PROJECT_DIR/qa/knapsack:/home/gdk/gdk/gitlab/qa/knapsack \
${QA_GDK_IMAGE} "${CI_COMMIT_SHA}" "$RSPEC_REPORT_OPTS $TEST_GDK_TAGS --tag ~requires_praefect"
# The above image's launch script takes two arguments only - first one is the commit sha and the second one Rspec Args
allow_failure: true
after_script:
- |
if [ "$CI_JOB_STATUS" == "failed" ]; then
echo "SUITE_FAILED=true" >> suite_status.env
fi
download-knapsack-report:
extends:
- .download-knapsack-report
- .rules:download-knapsack
test-on-gdk-smoke:
extends:
- .run-tests
parallel: 2
variables:
TEST_GDK_TAGS: "--tag smoke"
rules:
- when: always
test-on-gdk-full:
extends:
- .run-tests
parallel: 5
rules:
- when: manual
# ==========================================
# Post test stage
# ==========================================
e2e-test-report:
extends:
- .e2e-test-report
- .rules:report:allure-report
variables:
ALLURE_RESULTS_GLOB: test_output/allure-results
upload-knapsack-report:
extends:
- .upload-knapsack-report
- .rules:report:process-results
variables:
QA_KNAPSACK_REPORT_FILE_PATTERN: $CI_PROJECT_DIR/test_output/knapsack/*/*.json
export-test-metrics:
extends:
- .export-test-metrics
- .rules:report:process-results
variables:
QA_METRICS_REPORT_FILE_PATTERN: $CI_PROJECT_DIR/test_output/test-metrics-*.json
relate-test-failures:
extends:
- .relate-test-failures
- .rules:report:process-results
variables:
QA_RSPEC_JSON_FILE_PATTERN: $CI_PROJECT_DIR/test_output/rspec-*.json
generate-test-session:
extends:
- .generate-test-session
- .rules:report:process-results
variables:
QA_RSPEC_JSON_FILE_PATTERN: $CI_PROJECT_DIR/test_output/rspec-*.json
notify-slack:
extends:
- .notify-slack
- .rules:report:process-results
variables:
QA_RSPEC_XML_FILE_PATTERN: $CI_PROJECT_DIR/test_output/rspec-*.xml

View file

@ -0,0 +1,26 @@
untamper-my-lockfile:
image: registry.gitlab.com/gitlab-org/frontend/untamper-my-lockfile:main
stage: test
needs: []
before_script: []
after_script: []
cache: {}
retry: 1
script:
- untamper-my-lockfile --lockfile yarn.lock
rules:
# Create a pipeline if the branch is named 'add-untamper-my-lockfile' in
# order to have an integration check added in the MR that introduces it
- if: $CI_COMMIT_REF_NAME == "add-untamper-my-lockfile"
# Create a pipeline if there are changes in yarn.lock _and_ we are in a
# merge request _or_ branch pipeline.
#
# This ensures that the pipeline isn't run in scheduled jobs for example
#
# Also our best effort to support both branch and MR pipelines. In certain
# projects this might trigger _two_ pipelines. These projects can be fixed
# by adding proper workflow:rules
# https://docs.gitlab.com/ee/ci/yaml/#workflowrules
- if: $CI_PIPELINE_SOURCE == "merge_request_event" || $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH
changes:
- yarn.lock

View file

@ -5,99 +5,3 @@ vendor mail-smtp_pool:
trigger:
include: vendor/gems/mail-smtp_pool/.gitlab-ci.yml
strategy: depend
vendor attr_encrypted:
extends:
- .vendor:rules:attr_encrypted
needs: []
trigger:
include: vendor/gems/attr_encrypted/.gitlab-ci.yml
strategy: depend
vendor microsoft_graph_mailer:
extends:
- .vendor:rules:microsoft_graph_mailer
needs: []
trigger:
include: vendor/gems/microsoft_graph_mailer/.gitlab-ci.yml
strategy: depend
vendor ipynbdiff:
extends:
- .vendor:rules:ipynbdiff
needs: []
trigger:
include: vendor/gems/ipynbdiff/.gitlab-ci.yml
strategy: depend
vendor omniauth-azure-oauth2:
extends:
- .vendor:rules:omniauth-azure-oauth2
needs: []
trigger:
include: vendor/gems/omniauth-azure-oauth2/.gitlab-ci.yml
strategy: depend
vendor omniauth_crowd:
extends:
- .vendor:rules:omniauth_crowd
needs: []
trigger:
include: vendor/gems/omniauth_crowd/.gitlab-ci.yml
strategy: depend
vendor omniauth-gitlab:
extends:
- .vendor:rules:omniauth-gitlab
needs: []
trigger:
include: vendor/gems/omniauth-gitlab/.gitlab-ci.yml
strategy: depend
vendor omniauth-salesforce:
extends:
- .vendor:rules:omniauth-salesforce
needs: []
trigger:
include: vendor/gems/omniauth-salesforce/.gitlab-ci.yml
strategy: depend
vendor devise-pbkdf2-encryptable:
extends:
- .vendor:rules:devise-pbkdf2-encryptable
needs: []
trigger:
include: vendor/gems/devise-pbkdf2-encryptable/.gitlab-ci.yml
strategy: depend
vendor bundler-checksum:
extends:
- .vendor:rules:bundler-checksum
needs: []
trigger:
include: vendor/gems/bundler-checksum/.gitlab-ci.yml
strategy: depend
vendor gitlab_active_record:
extends:
- .vendor:rules:gitlab_active_record
needs: []
trigger:
include: vendor/gems/gitlab_active_record/.gitlab-ci.yml
strategy: depend
vendor cloud_profiler_agent:
extends:
- .vendor:rules:cloud_profiler_agent
needs: []
trigger:
include: vendor/gems/cloud_profiler_agent/.gitlab-ci.yml
strategy: depend
vendor sidekiq-reliable-fetch:
extends:
- .vendor:rules:sidekiq-reliable-fetch
needs: []
trigger:
include: vendor/gems/sidekiq-reliable-fetch/.gitlab-ci.yml
strategy: depend

View file

@ -1,49 +1,32 @@
workhorse:verify:
extends: .workhorse:rules:workhorse
image: ${GITLAB_DEPENDENCY_PROXY_ADDRESS}golang:${GO_VERSION}
image: ${GITLAB_DEPENDENCY_PROXY}golang:1.16
stage: test
needs: []
script:
- go version
- make -C workhorse # test build
- make -C workhorse verify
.workhorse:test:
extends: .workhorse:rules:workhorse
image: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images/debian-${DEBIAN_VERSION}-ruby-${RUBY_VERSION}-golang-${GO_VERSION}-rust-${RUST_VERSION}:rubygems-${RUBYGEMS_VERSION}-git-2.36-exiftool-12.60
services:
- name: registry.gitlab.com/gitlab-org/build/cng/gitaly:latest
# Disable the hooks so we don't have to stub the GitLab API
command: ["/usr/bin/env", "GITALY_TESTING_NO_GIT_HOOKS=1", "/scripts/process-wrapper"]
alias: gitaly
variables:
GITALY_ADDRESS: "tcp://127.0.0.1:8075"
GITALY_ADDRESS: "tcp://gitaly:8075"
stage: test
needs:
- setup-test-env
before_script:
- go version
- scripts/gitaly-test-build
needs: []
script:
- go version
- apt-get update && apt-get -y install libimage-exiftool-perl
- make -C workhorse test
workhorse:test go:
workhorse:test using go 1.15:
extends: .workhorse:test
parallel:
matrix:
- GO_VERSION: ["1.18", "1.19"]
script:
- make -C workhorse test-coverage
coverage: '/\d+.\d+%/'
artifacts:
paths:
- workhorse/coverage.html
image: ${GITLAB_DEPENDENCY_PROXY}golang:1.15
workhorse:test fips:
workhorse:test using go 1.16:
extends: .workhorse:test
parallel:
matrix:
- GO_VERSION: ["1.18", "1.19"]
image: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images/ubi-${UBI_VERSION}-ruby-${RUBY_VERSION}-golang-${GO_VERSION}-rust-${RUST_VERSION}:rubygems-${RUBYGEMS_VERSION}-git-2.36-exiftool-12.60
variables:
FIPS_MODE: 1
workhorse:test race:
extends: .workhorse:test
script:
- make -C workhorse test-race
image: ${GITLAB_DEPENDENCY_PROXY}golang:1.16

View file

@ -1,40 +1,14 @@
# Yamllint of yaml files.
# Yamllint of CI-related yaml and changelogs.
# This uses rules from project root `.yamllint`.
lint-yaml:
extends:
- .default-retry
- .yaml-lint:rules
image: pipelinecomponents/yamllint:latest
stage: lint
needs: []
script:
- yamllint --strict -f colored .
# The jobs below will not use the configuration present in `.yamllint` (it's because of the -d option)
#
# Docs: https://yamllint.readthedocs.io/en/stable/configuration.html#custom-configuration-without-a-config-file
lint-pipeline-yaml:
extends:
- .default-retry
- .lint-pipeline-yaml:rules
image: pipelinecomponents/yamllint:latest
stage: lint
stage: test
needs: []
variables:
LINT_PATHS: .gitlab-ci.yml .gitlab/ci lib/gitlab/ci/templates data/deprecations data/removals data/whats_new
LINT_PATHS: .gitlab-ci.yml .gitlab/ci lib/gitlab/ci/templates changelogs
script:
- 'yamllint -d "{extends: default, rules: {line-length: disable, document-start: disable}}" $LINT_PATHS'
lint-metrics-yaml:
extends:
- .default-retry
- .lint-metrics-yaml:rules
image: pipelinecomponents/yamllint:latest
stage: lint
needs: []
variables:
LINT_PATHS: config/metrics
script:
- 'yamllint --strict -f colored -d "{extends: default, rules: {line-length: disable, document-start: disable, indentation: {spaces: 2, indent-sequences: whatever}}}" $LINT_PATHS'
- '[[ ! -d "ee/" ]] || export LINT_PATHS="$LINT_PATHS ee/changelogs"'
- yamllint -f colored $LINT_PATHS

View file

@ -1,147 +0,0 @@
<!--
HOW TO USE THIS TEMPLATE
To propose an AI experiment, focus on completing the “Experiment” section first. As you refine the idea and gather feedback on your experiment, use the “Feature release” section to define how it will evolve as a Beta or GA capability. It's important that we link experiment to feature release. Feel free to add sections, but keep the existing ones.
You can choose how to get started with this template. For example, the proposal can start as an issue, and then be promoted to an epic to house all the work related to the experiment/prototype and feature release. If you prefer to start with an epic, you have to manually apply the proposal template. Regardless, if the experiment is eventually prioritized for development, the template content will need to appear in a top-level epic so it can be tracked alongside other prioritized AI experiments.
TITLE FORMAT
🤖 [AI Proposal] {Need/outcome} {Beneficiary} {Job/Small Job}
The title should be something that is easily understood that quickly communicates the intent of the project allowing team members to easily understand and recognize the expected work that will be done. A proposal title should combine the beneficiary of the feature/UI, the job it will allow them to accomplish (see https://about.gitlab.com/handbook/product/ux/jobs-to-be-done/#how-to-write-a-jtbd), and their expected outcome when the work is delivered. Well-defined statements are concise without sacrificing the substance of the proposal so that anyone can understand it at a glance. (e.g. {Reduce the effort} {for security teams} {when prioritizing business-critical risks in their assets}).
-->
# Experiment
This section should be completed prior to work on the Experiment beginning.
# [Experiment](https://docs.gitlab.com/ee/policy/alpha-beta-support.html#experiment)
## Problem to be solved
### User problem
_What user problem will this solve?_
### Solution hypothesis
_Why do you believe this AI solution is a good way to solve this problem?_
### Assumption
_What assumptions are you making about this problem and the solution?_
### Personas
_What [personas](https://about.gitlab.com/handbook/product/personas/#list-of-user-personas) have this problem, who is the intended user?_
## Proposal
<!-- Explain the proposed changes, including details around usage and business drivers. -->
### Success
_How will you measure whether this experiment is a success?_
# Feature release
<!-- DO NOT REMOVE THIS SECTION
Although the initial focus is on the “Experiment” section, do not remove this “Feature release” section. It's important that we link experiment to feature release. Fill this section as you progress.
-->
### Main Job story
_What job to be done will this solve?_
<!-- What is the [Main Job story](https://about.gitlab.com/handbook/product/ux/jobs-to-be-done/#how-to-write-a-jtbd) that this proposal was derived from? (e.g. When I am on triage rotation, I want to address all the business-critical risks in my assets, So I can minimize the likelihood of my organization being compromised by a security breach.) -->
## Proposal updates/additions
<!-- Explain any changes or updates to the original proposal from the experiment, including details around usage, business drivers, and reasonings that drove the updates/additions. -->
### Problem validation
_What validation exists that customers have this problem?_
<!-- Refer to https://about.gitlab.com/handbook/product/ux/ux-research/research-in-the-AI-space/#guideline-1-problem-validation --- to help identify and understand user needs -->
### Business objective
_What business objective will be achieved with this proposal?_
<!-- Objectives (from a business point of view) that will be achieved upon completion. (For instance, Increase engagement by making the experience efficient while reducing the chances of users overlooking high-priority items. -->
### Confidence
_Has this proposal been derived from research?_
<!-- How well do we understand the user's problem and their need? Refer to https://about.gitlab.com/handbook/product/ux/product-design/ux-roadmaps/#confidence to assess confidence -->
| Confidence | Research |
| ----------------- | ------------------------------ |
| [High/Medium/Low] | [research/insight issue](Link) |
### Requirements
_What tasks or actions should the user be capable of performing with this feature?_
<!-- Requirements can be taken from existing features or design issues used to build this proposal. Any related issues should be linked with this issue in the Feature/solution issues section below. They are more granular validated needs, goals, and additional details that the proposal encompasses. -->
> ⚠️ Related feature and research issues should be linked in the related issues section (Delete this line when this is done)
#### The user needs to be able to:
- ...
- ...
## Checklist
### Experiment
<details> <summary> Issue information </summary>
- [ ] Add information to the issue body about:
- [ ] The user problem being solved
- [ ] Your assumptions
- [ ] Who it's for, list of personas impacted
- [ ] Your proposal
- [ ] Add relevant designs to the Design Management area of the issue if available
- [ ] Confirm that an unexpected outage of this feature will not negatively impact the application or other features
- [ ] Add a feature flag so that this feature can be quickly disabled if/when needed
- [ ] If this experiment introduces a new service or data store, ensure it is not processing or storing [red data](https://about.gitlab.com/handbook/security/data-classification-standard.html#data-classification-levels) without a security and if needed legal review
- *NOTE*: We recommend using one of the already adopted models or data stores. If you need to use something else, be aware that using other models or data stores will require additional review during the feature stage for operational fitness and compliance.
- [ ] Ensure this issue has the ~wg-ai-integration label to ensure visibility to various teams working on this
</details>
### Feature release
<details> <summary> Issue information </summary>
- [ ] Add information to the issue body about:
- [ ] Your proposal
- [ ] The Job Statement it's expected to satisfy
- [ ] Details about the user problem and provide any research or problem validation
- [ ] List the personas impacted by the proposal.
- [ ] Add all relevant solution validation issues to the Linked items section that shows this proposal will solve the customer problem, or details explaining why it's not possible to provide that validation.
- [ ] Add relevant designs to the Design Management area of the issue.
- [ ] You have adhered to our [Definition of Done](https://docs.gitlab.com/ee/development/contributing/merge_request_workflow.html#definition-of-done) standards
- [ ] Ensure this issue has the ~wg-ai-integration label to ensure visibility to various teams working on this
</details>
<details> <summary> Technical needs </summary>
- [ ] Please consider the operational aspects of the feature you are creating. A list of things to think about is in: https://gitlab.com/gitlab-org/gitlab/-/issues/403859. We will be improving this process in the future: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/117637#note_1353253349.
- [ ] @ mention your [AppSec Stable Counterpart](https://about.gitlab.com/handbook/product/categories/) and read the [AI secure coding guidelines](https://docs.gitlab.com/ee/development/secure_coding_guidelines.html#artificial-intelligence-ai-features)
1. Work estimate and skills needs to build an ML viable feature: To build any ML feature depending on the work, there are many personas that contribute including, Data Scientist, NLP engineer, ML Engineer, MLOps Engineer, ML Infra engineers, and Fullstack engineer to integrate the ML Services with Gitlab. Post-prototype we would assess the skills needed to build a production-grade ML feature for the prototype.
2. Data Limitation: We would like to upfront validate if we have viable data for the feature including whether we can use the DataOps pipeline of ModelOps or create a custom one. We would want to understand the training data, test data, and feedback data to dial up the accuracy and the limitations of the data.
3. Model Limitation: We would want to understand if we can use an open-source pre-trained model, tune and customize it or start a model from scratch as well. Further, we would assess based on the ModelOps model evaluation framework which would be the right model to use based on the use case.
4. Cost, Scalability, Reliability: We would want to estimate the cost of hosting, serving, inference of the model, and the full end-to-end infrastructure including monitoring and observability.
5. Legal and Ethical Framework: We would want to align with legal and ethical framework like any other ModelOps features to cover across the nine principles of responsible ML and any legal support needed.
</details>
<details> <summary> Dependency needs </summary>
- [ ] Please consider the operational aspects of the service you are creating. A list of things to think about is in: https://gitlab.com/gitlab-org/gitlab/-/issues/403859. We will be improving this process in the future: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/117637#note_1353253349.
</details>
<details> <summary> Legal needs </summary>
- [ ] TBD
</details>
## Additional resources
- If you'd like help with technical validation, or would like to discuss UX considerations for AI mention the AI Assisted group using `@gitlab-org/modelops/applied-ml`.
- Read about our [AI Integration strategy](https://internal-handbook.gitlab.io/handbook/product/ai-strategy/ai-integration-effort/)
- Slack channels
- `#wg_ai_integration` - Slack channel for the working group and the high level alignment on getting AI ready for Production (Development, Product, UX, Legal, etc.) But from the other channels fell free to reach out and post progress here
- `#ai_integration_dev_lobby` - Channel for all implementation related topics and discussions of actual AI features (e.g. explain the code)
- `#ai_enablement_team` - Channel for the AI Enablement Team which is building the base for all features (experimentation API, Abstraction Layer, Embeddings, etc.)
/label ~wg-ai-integration
/cc @tmccaslin @hbenson @wayne @pedroms @jmandell
/confidential

View file

@ -1,32 +0,0 @@
<!-- Actionable insights must recommend an action that needs to take place. An actionable insight both defines the insight and clearly calls out action or next step required to improve based on the result of the research observation or data. Actionable insights are tracked over time and will include follow-up. Please follow the tasks outlined in this issue for best results. Learn more in the handbook here: https://about.gitlab.com/handbook/product/ux/ux-research-training/research-insights/#actionable-insights
This issue template is for an actionable insight that requires further exploration.-->
### Insight
<!-- Describe the insight itself: often the problem, finding, or observation.-->
### Supporting evidence
<!-- Describe why the problem is happening, or more details behind the finding or observation. Try to include quotes or specific data collected. Feel free to link the Actionable insight from Dovetail here if applicable instead of retyping details. -->
### Action
<!--Since this is an actionable insight that requires further exploration, ensure the action is algned to that. Describe the next step or action that needs to take place as a result of the research. The action should be clearly defined, achievable, and directly tied back to the insight. Make sure to use directive terminology, such as: conduct, explore, redesign, etc. -->
### Resources
<!--Add resources as links below or as related issues. -->
- :dove: [Dovetail project](Paste URL for Dovetail project here)
- :mag: [Research issue](Paste URL for research issue here)
- :footprints: [Follow-up issue or epic](Paste URL for follow-up issue or epic here)
### Tasks
<!--Fill out these tasks in order to consider an Actionable Insight complete. Actionable Insights are created as confidential by default, but can be made non-confidential if the insight does not include information about competitors from a Competitor Evaluation or any other confidential information. -->
- [ ] Assign this issue to the appropriate Product Manager, Product Designer, or UX Researcher.
- [ ] Add the appropriate `Group` (such as `~"group::source code"`) label to the issue. This helps identify and track actionable insights at the group level.
- [ ] Link this issue back to the original research issue in the GitLab UX Research project and the Dovetail project.
- [ ] Adjust confidentiality of this issue if applicable
/confidential
/label ~"Actionable Insight::Exploration needed"

View file

@ -1,33 +0,0 @@
<!-- Actionable insights must recommend an action that needs to take place. An actionable insight both defines the insight and clearly calls out action or next step required to improve based on the result of the research observation or data. Actionable insights are tracked over time and will include follow-up. Please follow the tasks outlined in this issue for best results. Learn more in the handbook here: https://about.gitlab.com/handbook/product/ux/ux-research-training/research-insights/#actionable-insights
This issue template is for an actionable insight that requires a change in the product.-->
### Insight
<!-- Describe the insight itself: often the problem, finding, or observation.-->
### Supporting evidence
<!-- Describe why the problem is happening, or more details behind the finding or observation. Try to include quotes or specific data collected. Feel free to link the Actionable insight from Dovetail here if applicable instead of retyping details. -->
### Action
<!--Since this is an actionable insight that requires a change in the product, ensure the action is algned to that. Describe the next step or action that needs to take place as a result of the research. The action should be clearly defined, achievable, and directly tied back to the insight. Make sure to use directive terminology, such as: change, update, add/remove, etc. -->
### Resources
<!--Add resources as links below or as related issues. -->
- :dove: [Dovetail project](Paste URL for Dovetail project here)
- :mag: [Research issue](Paste URL for research issue here)
- :footprints: [Follow-up issue or epic](Paste URL for follow-up issue or epic here)
### Tasks
<!--Fill out these tasks in order to consider an Actionable Insight complete. Actionable Insights are created as confidential by default, but can be made non-confidential if the insight does not include information about competitors from a Competitor Evaluation or any other confidential information. -->
- [ ] Assign this issue to the appropriate Product Manager, Product Designer, or UX Researcher.
- [ ] Add the appropriate `Group` (such as `~"group::source code"`) label to the issue. This helps identify and track actionable insights at the group level.
- [ ] Link this issue back to the original research issue in the GitLab UX Research project and the Dovetail project.
- [ ] Adjust confidentiality of this issue if applicable
/confidential
/label ~"Actionable Insight::Product change"
/label ~"SUS"

View file

@ -0,0 +1,28 @@
<!-- Actionable insights must recommend an action that needs to take place. An actionable insight both defines the insight and clearly calls out action or next step required to improve based on the result of the research observation or data. Actionable insights are tracked over time and will include follow-up. Learn more in the handbook here: https://about.gitlab.com/handbook/engineering/ux/ux-research-training/research-insights/#actionable-insights -->
### Insight
<!-- Describe the insight itself: often the problem, finding, or observation. -->
### Supporting evidence
<!-- Describe why the problem is happening, or more details behind the finding or observation. Try to include quotes or specific data collected. Feel free to link the Actionable insight from Dovetail here if applicable instead of retyping details. -->
### Action
<!--Describe the next step or action that needs to take place as a result of the research. The action should be clearly defined, achievable, and directly tied back to the insight. Make sure to use directive terminology, such as: conduct, explore, redesign, etc. -->
### Resources
<!--Add resources as links below or as related issues. -->
- :dove: [Dovetail project](Paste URL for Dovetail project here)
- :mag: [Research issue](Paste URL for research issue here)
- :footprints: [Follow-up issue or epic](Paste URL for follow-up issue or epic here)
### Tasks
- [ ] Assign this issue to the appropriate Product Manager, Product Designer, or UX Researcher.
- [ ] Add the appropriate `Group` (such as `~"group::source code"`) label to the issue. This helps identify and track actionable insights at the group level.
- [ ] Link this issue back to the original research issue in the GitLab UX Research project and the Dovetail project.
/label ~"Actionable Insight"

View file

@ -1,5 +1,4 @@
<!-- Audit Event documentation: See https://docs.gitlab.com/ee/administration/audit_events.html -->
<!-- Streaming Audit Event documentation: See https://docs.gitlab.com/ee/administration/audit_event_streaming.html -->
## Audit need
@ -9,11 +8,6 @@
<!-- Describe the audit event you are proposing should be added, including any details of what should be captured, how, and why. -->
### Streaming-only event or normal event?
<!-- Should this event be a streaming-only audit event or also logged to GitLab's database? Consider the
volume of data that will be generated by the event when answering this. -->
/label ~"Category:Audit Events"
/label ~"type::feature"
/label ~"feature"
/label ~"group::compliance"

View file

@ -1,30 +0,0 @@
<!---
This issue template is for a master pipeline is failing for a flaky reason that cannot be reliably reproduced.
Please read the below documentations for a workflow of triaging and resolving broken master.
- https://about.gitlab.com/handbook/engineering/workflow/#triage-broken-master
- https://gitlab.com/gitlab-org/quality/engineering-productivity/team/-/blob/main/runbooks/master-broken.md
- https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/testing_guide/flaky_tests.md
--->
### Summary
<!-- Link to the failing master build and add the build failure output in the below code block section. -->
### Steps to reproduce
<!-- If the pipeline failure is reproducible, provide steps to recreate the issue locally. Please use an ordered list. -->
Please refer to [Flaky tests documentation](https://docs.gitlab.com/ee/development/testing_guide/flaky_tests.html) to
learn more about how to reproduce them.
### Proposed Resolution
<!-- Describe the proposed change to restore master stability. -->
Please refer to the [Resolution guidance](https://about.gitlab.com/handbook/engineering/workflow/#resolution-of-broken-master) to learn more about resolution of broken master.
Once the flaky failure has been fixed on the default branch, open merge requests to cherry-pick the fix to the active stable branches.
/label ~"type::maintenance" ~"failure::flaky-test" ~"priority::3" ~"severity::3"

View file

@ -1,24 +0,0 @@
<!---
This issue template is for a master pipeline is failing for a non-flaky reason.
Please read the below documentations for a workflow of triaging and resolving broken master.
- https://about.gitlab.com/handbook/engineering/workflow/#triage-broken-master
- https://gitlab.com/gitlab-org/quality/engineering-productivity/team/-/blob/main/runbooks/master-broken.md
--->
### Summary
<!-- Link to the failing master build and add the build failure output in the below code block section. -->
### Steps to reproduce
<!-- If the pipeline failure is reproducible, provide steps to recreate the issue locally. Please use an ordered list. -->
### Proposed Resolution
<!-- Describe the proposed change to restore master stability. -->
Please refer to the [Resolution guidance](https://about.gitlab.com/handbook/engineering/workflow/#resolution-of-broken-master) to learn more about resolution of broken master.
/label ~"master:broken" ~"Engineering Productivity" ~"priority::1" ~"severity::1" ~"type::maintenance" ~"maintenance::pipelines"

View file

@ -2,10 +2,10 @@
Please read this!
Before opening a new issue, make sure to search for keywords in the issues
filtered by the "regression" or "type::bug" label:
filtered by the "regression" or "bug" label:
- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=regression
- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=type::bug
- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=bug
and verify the issue you're about to submit isn't a duplicate.
--->
@ -39,10 +39,7 @@ will also determine whether the bug is fixed in a more recent version. -->
### Output of checks
<!-- If you are reporting a bug on GitLab.com, uncomment below -->
<!-- This bug happens on GitLab.com -->
<!-- /label ~"reproduced on GitLab.com" -->
<!-- If you are reporting a bug on GitLab.com, write: This bug happens on GitLab.com -->
#### Results of GitLab environment info
@ -85,4 +82,4 @@ will also determine whether the bug is fixed in a more recent version. -->
<!-- If you can, link to the line of code that might be responsible for the problem. -->
/label ~"type::bug"
/label ~bug

View file

@ -1,13 +0,0 @@
Before raising an issue to the GitLab issue tracker, please read through our guide for finding help to determine the best place to post:
* https://about.gitlab.com/getting-help/
If you are experiencing an issue when using GitLab.com, your first port of call should be the Community Forum. Your issue may have already been reported there by another user. Please check:
* https://forum.gitlab.com/
If you feel that your issue can be categorized as a reproducible bug or a feature proposal, please use one of the issue templates provided and include as much information as possible.
Thank you for helping to make GitLab a better product.
<!-- template sourced from https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Default.md -->

View file

@ -1,101 +0,0 @@
For guidance on the overall deprecations, removals and breaking changes workflow, please visit [Breaking changes, deprecations, and removing features](https://about.gitlab.com/handbook/product/gitlab-the-product/#deprecations-removals-and-breaking-changes)
<!-- Use this template as a starting point for deprecations. -->
### Deprecation Summary
<!--
This should contain a brief description of the feature or functionality that is deprecated. The description should clearly state the potential impact of the deprecation to end users.
It is recommended that you link to the documentation.
The description of the deprecation should state what actions the user should take to rectify the behavior. If the deprecation is scheduled for an upcoming release, the content should remain in the deprecations documentation page until it has been completed. For example, if a deprecation is announced in 14.9 and scheduled to be completed in 15.0, the same content would be included in the documentation for 14.9, 14.10, and 15.0.
**If this issue proposes a breaking change outside a major release XX.0, you need to get approval from your manager and request collaboration from Product Operations on communication. Be sure to follow the guidance [here](https://about.gitlab.com/handbook/product/gitlab-the-product/#deprecations-removals-and-breaking-changes.)**
-->
### Breaking Change
<!-- Does this MR contain a breaking change? If yes:
- Add the ~"breaking change" label to this issue.
- Add instructions for how users can update their workflow. -->
### Affected Topology
<!--
Who is affected by this deprecation, Self-managed users, SaaS users, or both? This is especially important when nearing the annual major release where breaking changes and removals are typically introduced. These changes might be seen on GitLab.com before the official release date.
-->
### Affected Tier
<!--
Which tier is this feature available in?
* Free
* Premium
* Ultimate
-->
### Checklists
**Labels**
- [ ] This issue is labeled ~deprecation, and with the relevant `~devops::`, `~group::`, and `~Category:` labels.
- [ ] This issue is labeled ~"breaking change" if the removal of the deprecated item will be a [breaking change](https://about.gitlab.com/handbook/product/gitlab-the-product/#examples-of-breaking-changes).
**Timeline**
Please add links to the relevant merge requests.
- As soon as possible, but no later than the third milestone preceding the major release (for example, given the following release schedule: `14.8, 14.9, 14.10, 15.0` `14.8` is the third milestone preceding the major release):
- [ ] A [deprecation announcement entry](https://about.gitlab.com/handbook/marketing/blog/release-posts/#creating-the-announcement) has been created so the deprecation will appear in release posts and on the [general deprecation page](https://docs.gitlab.com/ee/update/deprecations).
- [ ] Documentation has been updated to mark the feature as [deprecated](https://docs.gitlab.com/ee/development/documentation/versions.html#deprecations-and-removals).
- [ ] On or before the major milestone: A [removal entry](https://about.gitlab.com/handbook/marketing/blog/release-posts/#creating-the-announcement-1) has been created so the removal will appear on the [removals by milestones](https://docs.gitlab.com/ee/update/removals) page and be announced in the release post.
- On the major milestone:
- [ ] The deprecated item has been removed.
- [ ] If the removal of the deprecated item is a [breaking change](https://about.gitlab.com/handbook/product/gitlab-the-product/#examples-of-breaking-changes), the merge request is labeled ~"breaking change".
**Mentions**
- [ ] Your stage's stable counterparts have been `@mentioned` on this issue. For example, Customer Support, Customer Success (Technical Account Manager), Product Marketing Manager.
- To see who the stable counterparts are for a product team visit [product categories](https://about.gitlab.com/handbook/product/categories/)
- If there is no stable counterpart listed for Sales/CS please mention `@timtams`
- If there is no stable counterpart listed for Support please mention `@gitlab-com/support/managers`
- If there is no stable counterpart listed for Marketing please mention `@cfoster3`
- [ ] Your GPM has been `@mentioned` so that they are aware of planned deprecations. The goal is to have reviews happen at least two releases before the final removal of the feature or introduction of a breaking change.
### Deprecation Milestone
<!-- In which milestone will this deprecation be announced ? -->
### Planned Removal Milestone
<!-- In which milestone will the feature or functionality be removed and announced? -->
### Links
<!--
Add links to any relevant documentation or code that will provide additional details or clarity regarding the planned change.
This issue is the main SSOT for the deprecations and removals process. Be sure to link all
issues and MRs related to this deprecation/removal to this issue. This can include removal
issues that were created ahead of time, and the MRs doing the actual deprecation/removal work.
-->
<!-- Label reminders - you should have one of each of the following labels.
Use the following resources to find the appropriate labels:
- https://gitlab.com/gitlab-org/gitlab/-/labels
- https://about.gitlab.com/handbook/product/categories/features/
-->
<!-- Populate the Section, Group, and Category -->
/label ~devops:: ~group: ~Category:
<!-- Choose the Pricing Tier(s) -->
/label ~"GitLab Free" ~"GitLab Premium" ~"GitLab Ultimate"
<!-- Identifies that this Issue is related to deprecating a feature -->
/label ~"deprecation"
<!-- Add the ~"breaking change" label to this issue if necessary -->

View file

@ -1,9 +1,5 @@
<!-- Title: Design Sprint -->
This template outlines a sample set-up process, activities and deliverables for running a Remote Design Sprint. The specific activities and deliverables should be customized based on your objectives and timeline.
Please refer to the [Remote Design Sprint Handbook page](#anchor-tag-to-handbook-page) for additional recommendations.
## Design Sprint Focus
* [ ] Have you [determined that a Design Sprint is appropriate for this project](#anchor-tag-to-handbook-page)?
<!-- What is the focus of the [Design Sprint](https://about.gitlab.com/handbook/product/product-processes/#design-sprint)? What problem area will you be solving for and who is the target user? -->
@ -102,7 +98,6 @@ If you enjoy taking notes using post-it notes make sure you have available some
- [ ] Finalise participant list - `decider` and `facilitator`
- [ ] Create [participation form](https://docs.google.com/forms/d/e/1FAIpQLSc0_BNltvRW8yXXaJd8sIKzgDmrSGqILMfkoCJrAj6sFcsMcg/viewform?usp=sf_link) and send to participants (**deadline**: [date]) - `facilitator`
- [ ] Create a dedicated Slack channel and add participants - `facilitator`
- [ ] Promote this issue to an epic - `facilitator`
- [ ] Create issues under the epic for the pre-workshop tasks: Expert interviews ([example](https://gitlab.com/groups/gitlab-org/configure/-/epics/3#note_332412524)), Lightning walkthroughs and How might we.. notetaking assignment ([example](https://gitlab.com/gitlab-org/configure/general/-/issues/52)), Voting How might we... notes assignment ([example](https://gitlab.com/gitlab-org/configure/general/-/issues/54)) - `facilitator`
- [ ] Create sync meetings in calendar and invite all participants (**deadline**: [date]) - `decider` or `facilitator`
@ -146,18 +141,16 @@ Deciding which persona we are focusing on will be part of the Day 1 discussions
* [Delaney (Development Team Lead)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#delaney-development-team-lead)
* [Presley (Product Designer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#presley-product-designer)
* [Sasha (Software Developer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sasha-software-developer)
* [Priyanka (Platform Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#priyanka-platform-engineer)
* [Devon (DevOps Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#devon-devops-engineer)
* [Sidney (Systems Administrator)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sidney-systems-administrator)
* [Sam (Security Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sam-security-analyst)
* [Rachel (Release Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#rachel-release-manager)
* [Alex (Security Operations Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#alex-security-operations-engineer)
* [Simone (Software Engineer in Test)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#simone-software-engineer-in-test)
* [Allison (Application Ops)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#allison-application-ops)
* [Ingrid (Infrastructure Operator)](https://about.gitlab.com/handbook/product/personas/#ingrid-infrastructure-operator)
* [Dakota (Application Development Director)](https://about.gitlab.com/handbook/product/personas/#dakota-application-development-director)
* [Priyanka (Platform Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#priyanka-platform-engineer)
* [Dana (Data Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#dana-data-analyst)
* [Eddie (Content Editor)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#eddie-content-editor)
-->
## Agenda

View file

@ -1,38 +0,0 @@
<!--
* Use this template for documentation issues identified
* by [Vale](https://docs.gitlab.com/ee/development/documentation/testing.html#vale)
* or [markdownlint](https://docs.gitlab.com/ee/development/documentation/testing.html#markdownlint).
* This template is meant to describe work for first-time contributors or
* for work during Hackathons.
*
* Feature development work should not use this template. Use the Feature Request template instead.
-->
## Hi community contributors! :wave:
Do you want to work on this issue?
- **If the issue is unassigned**, in a comment, type `@docs-hackathon I would like to work on this issue` and a writer will assign it to you.
To be fair to others, do not ask for more than three issues at a time.
- **If the issue is assigned to someone already**, choose another issue. Do not open a merge request for this issue if you are not assigned.
## To resolve the issue
[Follow these instructions to create a merge request](https://docs.gitlab.com/ee/development/documentation/workflow.html#how-to-update-the-docs).
- Don't submit your merge request until after the Hackathon has started.
- Try to address the issue in a single merge request.
- Try to stick to the scope of the issue. If you see other improvements that can be made in the file, open a separate merge request.
- When you create the merge request, select the **Documentation** merge request description template.
- In the merge request's description, add a link to this issue.
- Follow the [commit message guidelines](https://docs.gitlab.com/ee/development/contributing/merge_request_workflow.html#commit-messages-guidelines).
Use three to five words for your commit message, start with message with a capital letter, and do **not** end it in a period.
Other commit messages can cause the pipeline to fail.
Thank you again for contributing to the GitLab documentation! :tada:
## Documentation issue
/labels ~"documentation" ~"docs-only" ~"documentation" ~"docs::improvement" ~"type::maintenance" ~"maintenance::refactor" ~"Seeking community contributions" ~"quick win" ~"Technical Writing"

View file

@ -25,7 +25,7 @@
* Include use cases, benefits, and/or goals for this work.
* If adding content: What audience is it intended for? (What roles and scenarios?)
For ideas, see personas at https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/ or the persona labels at
https://gitlab.com/groups/gitlab-org/-/labels?subscribed=&search=persona%3A
https://gitlab.com/groups/gitlab-org/-/labels?utf8=%E2%9C%93&subscribed=&search=persona%3A
-->
### Proposal

View file

@ -1,6 +1,6 @@
<!--Lightweight issue template to encourage Dogfooding and educate team members about the importance of Dogfooding -->
/label ~"dogfooding" ~"group::" ~"section::" ~"Category:"
/label ~"dogfooding" ~"group::" ~"section::" ~"Category::"
## Feature to Dogfood
<!--Link to Description of feature (Documentation, Epic, Opportunity Canvas, etc.) -->

View file

@ -1,80 +0,0 @@
<!-- Before implementing a new empty state solution, make sure to read the
Empty State region docs in Pajamas: https://design.gitlab.com/regions/empty-states -->
## Description
<!-- Describe the solution you're proposing for your empty state region.
Include links to user research (if applicable). -->
## Location
<!-- Provide a link and location of the new empty state solution.
For example: https://gitlab.com/gitlab-org/gitlab-services/design.gitlab.com/-/issues -->
## Use case
<!-- What is the use case for the solution you're proposing?
Read the Empty State docs and select the use case below: https://design.gitlab.com/regions/empty-states -->
- [ ] Blank content
- [ ] Empty search results
- [ ] Configuration required
- [ ] Higher tier
## Checklist
<!-- Follow the steps below that correspond with the use case selected above.
Follow the steps to complete this issue -->
### Blank content
- [ ] The solution follows the `Blank content` specifications [in Pajamas](https://design.gitlab.com/regions/empty-states#blank-content).
- [ ] Follow the instructions from the [`After merge` section](#after-merge) below to add Snowplow tracking.
### Empty search results
- [ ] The solution follows the `Empty search results` specifications [in Pajamas](https://design.gitlab.com/regions/empty-states#empty-search-results).
- [ ] Follow the instructions from the [`After merge` section](#after-merge) below to add Snowplow tracking.
### Configuration required
- [ ] The solution follows the `Configuration required` specifications [in Pajamas](https://design.gitlab.com/regions/empty-states#configuration-required).
- [ ] Ask a [Growth product manager or Designer](https://about.gitlab.com/handbook/engineering/development/growth/#stable-counterparts) to review your solution.
- [ ] Is your solution introducing a new empty states or modifying an existing one?
- [ ] Introducing a new empty state: Follow the instructions from the [`After merge` section](#after-merge) below to add Snowplow tracking.
- [ ] Modifying an existing empty state: Follow the [`Experimentation` process](#experimentation) below. _Note_: If the empty state you want to replace hasn't been updated in a long time, doesn't pitch the value of the feature, or does not contain a next step action CTA, then we recommend you skip the experimentation process to implement and add tracking to your new empty state.
<!-- IF experimentation -->
#### Experimentation
- [ ] Collaborate with a [Growth product manager](https://about.gitlab.com/handbook/engineering/development/growth/#stable-counterparts) to help you determine if you can validate your solution through an experiment on SaaS.
- [ ] If an experiment is possible, create an issue using the [experiment idea template](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Experiment%20Idea) and follow the template intructions. Otherwise, follow the instructions from the [`After merge` section](#after-merge) below to add Snowplow tracking.
- [ ] Ask a [Growth product manager or Designer](https://about.gitlab.com/handbook/engineering/development/growth/#stable-counterparts) to review your experiment set-up.
- [ ] Implement and monitor the experiment following the [implementation guide](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/experiment_guide/gitlab_experiment.md#implement-an-experiment).
- [ ] Review and discuss the findings.
- [ ] Add the findings to the [Growth experimentation knowledge](https://about.gitlab.com/direction/growth/#growth-experiments-knowledge-base---concluded-experiments).
### Higher tier
- [ ] The solution follows the `Higher tier` specifications [in Pajamas](https://design.gitlab.com/regions/empty-states#higher-tier).
- [ ] Ask a Product Manager or Designer from the [Conversion group](https://about.gitlab.com/handbook/engineering/development/growth/conversion/#group-members) to review your solution.
- [ ] Is your solution introducing a new empty states or modifying an existing one?
- [ ] Introducing a new empty state: follow the instructions from the [`After merge` section](#after-merge) below to add Snowplow tracking.
- [ ] Modifying an existing empty state, follow the [`Experimentation` process](#experimentation) below.
<!-- IF experimentation -->
#### Experimentation
- [ ] Collaborate with a [Growth product manager](https://about.gitlab.com/handbook/engineering/development/growth/#stable-counterparts) to help you determine if you can validate your solution through an experiment on SaaS.
- [ ] If an experiment is possible, create an issue using the [experiment idea template](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Experiment%20Idea) and follow the template intructions. Otherwise, follow the instructions from the [`After merge` section](#after-merge) below to add Snowplow tracking.
- [ ] Add a ~"Category:Conversion Experiment" label to the experiment idea issue.
- [ ] Ask a Product Manager or Designer from the [Conversion group](https://about.gitlab.com/handbook/engineering/development/growth/conversion/#group-members) to review your experiment set-up.
- [ ] Implement and monitor the experiment following the [implementation guide](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/experiment_guide/gitlab_experiment.md#implement-an-experiment) .
- [ ] Review and discuss the findings.
- [ ] Add the findings to the [Growth experimentation knowledge](https://about.gitlab.com/direction/growth/#growth-experiments-knowledge-base---concluded-experiments).
## After merge
- [ ] Use the `Snowplow event tracking` [issue template](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Snowplow%20event%20tracking) and open an issue to add Snowplow event tracking to your new empty state solution.
- [ ] Add your ~devops:: and ~group:: labels to the new issue.

View file

@ -1,25 +0,0 @@
<!-- Title suggestion: Experiment Implementation: [description] -->
# Experiment Summary
<!-- Quick rundown of what is being done -->
# Design
<!-- This should include the contexts that determine the reproducibility (stickiness) of an experiment. This means that if you want the same behavior for a user, the context would be user, or if you want all users when viewing a specific project, the context would be the project being viewed, etc. -->
# Rollout strategy
<!-- This is currently called A/B test, which isn't accurate for multi-variants. Let's call this rollout strategy. It should outline the percentages for variants and if there's more than one step to this, each of those steps and the timing for those steps (e.g. 30 days after initial rollout). -->
# Inclusions and exclusions
<!-- These would be the rules for which given context (and are limited to context or resolvable at experiment time details) is included or excluded from the test. An example of this would be to only run an experiment on groups less than N number of days old. -->
# Segmentation
<!-- Rules for always saying context with these criteria always get this variant. For instance, if you want to always give groups less than N number of days old the experiment experience, they are specified here. This is different from the exclusion rules above. -->
# Tracking Details
- [json schema](https://gitlab.com/gitlab-org/iglu/-/blob/master/public/schemas/com.gitlab/gitlab_experiment/jsonschema/0-3-0) used in `gitlab-experiment` tracking.
- see [event schema](https://docs.gitlab.com/ee/development/snowplow/index.html#event-schema) for a guide.
| sequence | activity | category | action | label | property | value |
| -------- | -------- | ------ | ----- | ------- | -------- | ----- |
| | | | | | | |

View file

@ -1,120 +0,0 @@
<!-- Title suggestion: [Experiment Rollout] feature-flag-name - description of experiment -->
## Summary
This issue tracks the rollout and status of an experiment through to removal.
1. Feature flag name: `<feature-flag-name>`
1. Epic or issue link: `<issue or epic link>`
This is an experiment rollout issue
using the scoped [experiment label](https://about.gitlab.com/handbook/engineering/development/growth/experimentation/#experiment-rollout-issue).
As well as defining the experiment rollout and cleanup, this issue incorporates the relevant
[`Feature Flag Roll Out`](https://gitlab.com/gitlab-org/gitlab/-/edit/master/.gitlab/issue_templates/Feature%20Flag%20Roll%20Out.md) steps.
## Owners
- Team: `group::TEAM_NAME`
- Most appropriate slack channel to reach out to: `#g_TEAM_NAME`
- Best individual to reach out to: NAME
- Product manager (PM): NAME
### Stakeholders
<!--
Are there any other stages or teams involved that need to be kept in the loop?
- PM: Name
- Group: `group::TEAM_NAME`
- The Support Team
- The Delivery Team
-->
## Expectations
### What are we expecting to happen?
<!-- Describe the expected outcome when rolling out this experiment. -->
### What might happen if this goes wrong?
<!-- Any MRs that need to be rolled back? Communication that needs to happen? What are some things you can think of that could go wrong - data loss or broken pages? -->
### What can we monitor to detect problems with this?
<!-- Which dashboards from https://dashboards.gitlab.net are most relevant? -->
## Tracked data
<!-- brief description or link to issue or Sisense dashboard -->
Note: you can use the [CXL calculator](https://cxl.com/ab-test-calculator/) to determine if your experiment has reached significance. The calculator includes an estimate for how much longer an experiment must run for before reaching significance.
## Rollout plan
<!-- Add an overview and method for modifying the feature flag -->
- Runtime in days, or until we expect to reach statistical significance: `30`
- We will roll this out behind a feature flag and expose this to `<rollout-percentage>`% of actors to start then ramp it up from there.
`/chatops run feature set <feature-flag-name> <rollout-percentage> --actors`
### Status
#### Preferred workflow
The issue should be assigned to the Product manager (PM) or Engineer (Eng) as follows:
1. PM determines and manages the status of the experiment (assign this issue to the PM)
1. PM asks for initial rollout on production, or changes to the status (assign to an Eng)
1. Eng changes the status using `chatops` (reassign to the PM)
1. When concluded, PM updates the 'Roll Out Steps' and adds a milestone (assigns to an Eng)
The current status and history can be viewed using the:
- [API](https://gitlab.com/api/v4/experiments) (GitLab team members)
- [Feature flag log](https://gitlab.com/gitlab-com/gl-infra/feature-flag-log/-/issues?scope=all&utf8=%E2%9C%93&state=all) (GitLab team members)
- [Experiment rollout board](https://gitlab.com/groups/gitlab-org/-/boards/1352542)
In this rollout issue, ensure the scoped `experiment::` label is kept accurate.
### Experiment Results
<!-- update when experiment in/validated, set the scoped `~experiment::` status accordingly -->
## Roll Out Steps
- [ ] [Confirm that end-to-end tests pass with the feature flag enabled](https://docs.gitlab.com/ee/development/testing_guide/end_to_end/feature_flags.html#confirming-that-end-to-end-tests-pass-with-a-feature-flag-enabled). If there are failing tests, contact the relevant [stable counterpart in the Quality department](https://about.gitlab.com/handbook/engineering/quality/#individual-contributors) to collaborate in updating the tests or confirming that the failing tests are not caused by the changes behind the enabled feature flag.
- [ ] Enable on staging (`/chatops run feature set <feature-flag-name> true --staging`)
- [ ] Test on staging
- [ ] Ensure that documentation has been updated
- [ ] Enable on GitLab.com for individual groups/projects listed above and verify behaviour (`/chatops run feature set --project=gitlab-org/gitlab <feature-flag-name> true`)
- [ ] Coordinate a time to enable the flag with the SRE oncall and release managers
- In `#production` mention `@sre-oncall` and `@release-managers`. Once an SRE on call and Release Manager on call confirm, you can proceed with the rollout
- [ ] Announce on the issue an estimated time this will be enabled on GitLab.com
- [ ] Enable on GitLab.com by running chatops command in `#production` (`/chatops run feature set <feature-flag-name> true`)
- [ ] Cross post chatops Slack command to `#support_gitlab-com` ([more guidance when this is necessary in the dev docs](https://docs.gitlab.com/ee/development/feature_flags/controls.html#where-to-run-commands)) and in your team channel
- [ ] Announce on the issue that the flag has been enabled
- [ ] Remove experiment code and feature flag and add changelog entry - a separate [cleanup issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Experiment%20Successful%20Cleanup) might be required
- [ ] After the flag removal is deployed, [clean up the feature flag](https://docs.gitlab.com/ee/development/feature_flags/controls.html#cleaning-up) by running chatops command in `#production` channel
- [ ] Assign to the product manager to update the [knowledge base](https://about.gitlab.com/direction/growth/#growth-insights-knowledge-base) (if applicable)
## Rollback Steps
- [ ] This feature can be disabled by running the following Chatops command:
```
/chatops run feature set <feature-flag-name> false
```
## Experiment Successful Cleanup Concerns
_Items to be considered if candidate experience is to become a permanent part of GitLab_
<!--
Add a list of items raised during MR review or otherwise that may need further thought/consideration
before becoming permanent parts of the product.
Example: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/70451#note_727246104
-->
/label ~"feature flag" ~"devops::growth" ~"growth experiment" ~"experiment-rollout" ~Engineering ~"workflow::scheduling" ~"experiment::pending"
/milestone %"Next 1-3 releases"

View file

@ -10,16 +10,11 @@ The changes need to become an official part of the product.
- [ ] Determine whether the feature should apply to SaaS and/or self-managed
- [ ] Determine whether the feature should apply to EE - and which tiers - and/or Core
- [ ] Determine if tracking should be kept as is, removed, or modified.
- [ ] Determine if any UX experiences need to be "polished" i.e. updated to further improve the end user experience. This task should be completed by the designated UX counterpart.
- [ ] (placeholder for UX polish work that needs to be completed for this cleanup issue to be considered completed)
- [ ] Ensure any relevant documentation has been updated.
- [ ] Determine whether there are other concerns that need to be considered before removing the feature flag.
- These are typically captured in the `Experiment Successful Cleanup Concerns` section of the rollout issue.
- [ ] Consider changes to any `feature_category:` introduced by the experiment if ownership is changing (PM for Growth and PM for the new category as DRIs)
- [ ] Check to see if the experiment introduced new design assets. Add them to the appropriate repos and document them if needed.
- [ ] Optional: Migrate experiment to a default enabled [feature flag](https://docs.gitlab.com/ee/development/feature_flags) for one milestone and add a changelog. Converting to a feature flag can be skipped at the ICs discretion if risk is deemed low with consideration to both SaaS and (if applicable) self managed
- [ ] In the next milestone, [remove the feature flag](https://docs.gitlab.com/ee/development/feature_flags/controls.html#cleaning-up) if applicable
- [ ] After the flag removal is deployed, [clean up the feature/experiment feature flags](https://docs.gitlab.com/ee/development/feature_flags/controls.html#cleaning-up) by running chatops command in `#production` channel
- [ ] Ensure the corresponding [Experiment Rollout](https://gitlab.com/groups/gitlab-org/-/boards/1352542?label_name[]=devops%3A%3Agrowth&label_name[]=growth%20experiment&label_name[]=experiment-rollout) issue is updated
- [ ] Ensure the corresponding [Experiment Tracking](https://gitlab.com/groups/gitlab-org/-/boards/1352542?label_name[]=devops%3A%3Agrowth&label_name[]=growth%20experiment&label_name[]=experiment%20tracking) issue is updated
/label ~"type::maintenance" ~"workflow::scheduling" ~"growth experiment" ~"feature flag"
/label ~"feature" ~"feature::maintenance" ~"workflow::scheduling" ~"growth experiment" ~"feature flag"

View file

@ -0,0 +1,25 @@
<!-- Title suggestion: Experiment: [description] -->
# Experiment Summary
<!-- Quick rundown of what is being done -->
# Design
<!-- This should include the contexts that determine the reproducibility (stickiness) of an experiment. This means that if you want the same behavior for a user, the context would be user, or if you want all users when viewing a specific project, the context would be the project being viewed, etc. -->
# Rollout strategy
<!-- This is currently called A/B test, which isn't accurate for multi-variants. Let's call this rollout strategy. It should outline the percentages for variants and if there's more than one step to this, each of those steps and the timing for those steps (e.g. 30 days after initial rollout). -->
# Inclusions and exclusions
<!-- These would be the rules for which given context (and are limited to context or resolvable at experiment time details) is included or excluded from the test. An example of this would be to only run an experiment on groups less than N number of days old. -->
# Segmentation
<!-- Rules for always saying context with these criteria always get this variant. For instance, if you want to always give groups less than N number of days old the experiment experience, they are specified here. This is different from the exclusion rules above. -->
# Tracking Details
- [json schema](https://gitlab.com/gitlab-org/iglu/-/blob/master/public/schemas/com.gitlab/gitlab_experiment/jsonschema/0-3-0) used in `gitlab-experiment` tracking.
- see [taxonomy](https://docs.gitlab.com/ee/development/snowplow/index.html#structured-event-taxonomy) for a guide.
| activity | category | action | label | context | property | value |
| -------- | -------- | ------ | ----- | ------- | -------- | ----- |
| | | | | json schema | | |

View file

@ -1,51 +0,0 @@
<!-- Title suggestion: [Feature flag] Cleanup <feature-flag-name> -->
## Summary
This issue is to cleanup the `<feature-flag-name>` feature flag, after the feature flag has been enabled by default for an appropriate amount of time in production.
<!-- Short description of what the feature is about and link to relevant other issues. -->
## Owners
- Team: NAME_OF_TEAM
- Most appropriate slack channel to reach out to: `#g_TEAM_NAME`
- Best individual to reach out to: NAME
- PM: NAME
## Stakeholders
<!--
Are there any other stages or teams involved that need to be kept in the loop?
- Name of a PM
- The Support Team
- The Delivery Team
-->
## Expectations
### What might happen if this goes wrong?
<!-- Any MRs that need to be rolled back? Communication that needs to happen? What are some things you can think of that could go wrong - data loss or broken pages? -->
### Cleaning up the feature flag
<!-- The checklist here is to help stakeholders keep track of the feature flag status -->
- [ ] Create a merge request to remove `<feature-flag-name>` feature flag. Ask for review and merge it.
- [ ] Remove all references to the feature flag from the codebase.
- [ ] Remove the YAML definitions for the feature from the repository.
- [ ] Create [a changelog entry](https://docs.gitlab.com/ee/development/feature_flags/#changelog).
- [ ] Ensure that the cleanup MR has been deployed to both production and canary.
If the merge request was deployed before [the code cutoff](https://about.gitlab.com/handbook/engineering/releases/#self-managed-releases-1),
the feature can be officially announced in a release blog post.
- [ ] `/chatops run auto_deploy status <merge-commit-of-cleanup-mr>`
- [ ] Close [the feature issue](ISSUE LINK) to indicate the feature will be released in the current milestone.
- [ ] If not already done, clean up the feature flag from all environments by running these chatops command in `#production` channel:
- [ ] `/chatops run feature delete <feature-flag-name> --dev`
- [ ] `/chatops run feature delete <feature-flag-name> --staging`
- [ ] `/chatops run feature delete <feature-flag-name>`
- [ ] Close this rollout issue.
/label ~"feature flag" ~"type::maintenance" ~"maintenance::removal"

View file

@ -0,0 +1,28 @@
<!-- Title suggestion: [Feature flag] Remove FEATURE_FLAG_NAME -->
## Feature
The `:feature_name` feature flag was previously [enabled by default](URL) and should be removed.
## Owners
- Group: ~"group::GROUP_NAME"
- Slack channel: `#g_GROUP_NAME`
- DRI: USERNAME
- PM: USERNAME
**Removal**
This is an __important__ phase, that should be either done in the next Milestone or as soon as possible. For the cleanup phase, please follow our documentation on how to [clean up the feature flag](https://docs.gitlab.com/ee/development/feature_flags/controls.html#cleaning-up).
- [ ] Remove `:feature_name` feature flag
- [ ] Remove all references to the feature flag from the codebase
- [ ] Remove the YAML definitions for the feature from the repository
- [ ] Create a Changelog Entry
- [ ] Clean up the feature flag from all environments by running this chatops command in `#production` channel `/chatops run feature delete some_feature`.
- [ ] Close this issue after the feature flag is removed from the codebase.
/label ~"feature flag" ~"technical debt"
/assign DRI

View file

@ -1,17 +1,8 @@
<!-- Title suggestion: [Feature flag] Enable description of feature -->
<!--
Set the main issue link: The main issue is the one that describes the problem to solve,
the one this feature flag is being added for. For example:
[main-issue]: https://gitlab.com/gitlab-org/gitlab/-/issues/123456
-->
[main-issue]: MAIN-ISSUE-LINK
## Summary
This issue is to rollout [the feature][main-issue] on production,
This issue is to rollout [the feature](ISSUE LINK) on production,
that is currently behind the `<feature-flag-name>` feature flag.
<!-- Short description of what the feature is about and link to relevant other issues. -->
@ -33,23 +24,30 @@ Are there any other stages or teams involved that need to be kept in the loop?
- The Delivery Team
-->
## The Rollout Plan
- Partial Rollout on GitLab.com with testing groups
- Rollout on GitLab.com for a certain period (How long)
- Percentage Rollout on GitLab.com
- Rollout Feature for everyone as soon as it's ready
<!-- Which dashboards from https://dashboards.gitlab.net are most relevant? Sentry errors reports can also be useful to review -->
## Testing Groups/Projects/Users
<!-- If applicable, any groups/projects that are happy to have this feature turned on early. Some organizations may wish to test big changes they are interested in with a small subset of users ahead of time for example. -->
- `gitlab-org/gitlab` project
- `gitlab-org`/`gitlab-com` groups
- ...
## Expectations
### What are we expecting to happen?
<!-- Describe the expected outcome when rolling out this feature -->
### When is the feature viable?
<!-- What are the settings we need to configure in order to have this feature viable? -->
<!--
Example below:
1. Enable service ping collection
`ApplicationSetting.first.update(usage_ping_enabled: true)`
-->
### What might happen if this goes wrong?
<!-- Should the feature flag be turned off? Any MRs that need to be rolled back? Communication that needs to happen? What are some things you can think of that could go wrong - data loss or broken pages? -->
@ -57,76 +55,45 @@ Example below:
### What can we monitor to detect problems with this?
<!-- Which dashboards from https://dashboards.gitlab.net are most relevant? -->
_Consider mentioning checks for 5xx errors or other anomalies like an increase in redirects
(302 HTTP response status)_
### What can we check for monitoring production after rollouts?
_Consider adding links to check for Sentry errors, Production logs for 5xx, 302s, etc._
## Rollout Steps
Note: Please make sure to run the chatops commands in the slack channel that gets impacted by the command.
### Rollout on non-production environments
- [ ] Verify the MR with the feature flag is merged to master.
- Verify that the feature MRs have been deployed to non-production environments with:
- [ ] Ensure that the feature MRs have been deployed to non-production environments.
- [ ] `/chatops run auto_deploy status <merge-commit-of-your-feature>`
- [ ] Enable the feature globally on non-production environments.
- [ ] `/chatops run feature set <feature-flag-name> true --dev --staging --staging-ref`
- If the feature flag causes QA end-to-end tests to fail:
- [ ] Disable the feature flag on staging to avoid blocking [deployments](https://about.gitlab.com/handbook/engineering/deployments-and-releases/deployments/).
- [ ] `/chatops run feature set <feature-flag-name> true --dev`
- [ ] `/chatops run feature set <feature-flag-name> true --staging`
- [ ] Verify that the feature works as expected. Posting the QA result in this issue is preferable.
The best environment to validate the feature in is [staging-canary](https://about.gitlab.com/handbook/engineering/infrastructure/environments/#staging-canary)
as this is the first environment deployed to. Note you will need to make sure you are configured to use canary as outlined [here](https://about.gitlab.com/handbook/engineering/infrastructure/environments/canary-stage/)
when accessing the staging environment in order to make sure you are testing appropriately.
For assistance with QA end-to-end test failures, please reach out via the `#quality` Slack channel. Note that QA test failures on staging-ref [don't block deployments](https://about.gitlab.com/handbook/engineering/infrastructure/environments/staging-ref/#how-to-use-staging-ref).
### Preparation before production rollout
### Specific rollout on production
For visibility, all `/chatops` commands that target production should be executed in the `#production` slack channel and cross-posted (with the command results) to the responsible team's slack channel (`#g_TEAM_NAME`).
- Ensure that the feature MRs have been deployed to both production and canary.
- [ ] Ensure that the feature MRs have been deployed to both production and canary.
- [ ] `/chatops run auto_deploy status <merge-commit-of-your-feature>`
- Depending on the [type of actor](https://docs.gitlab.com/ee/development/feature_flags/#feature-actors) you are using, pick one of these options:
- If you're using **project-actor**, you must enable the feature on these entries:
- [ ] `/chatops run feature set --project=gitlab-org/gitlab,gitlab-org/gitlab-foss,gitlab-com/www-gitlab-com <feature-flag-name> true`
- If you're using **group-actor**, you must enable the feature on these entries:
- [ ] `/chatops run feature set --group=gitlab-org,gitlab-com <feature-flag-name> true`
- If you're using **user-actor**, you must enable the feature on these entries:
- [ ] `/chatops run feature set --user=<your-username> <feature-flag-name> true`
- [ ] Verify that the feature works on the specific entries. Posting the QA result in this issue is preferable.
### Preparation before global rollout
- [ ] Set a milestone to the rollout issue to signal for enabling and removing the feature flag when it is stable.
- [ ] Check if the feature flag change needs to be accompanied with a
[change management issue](https://about.gitlab.com/handbook/engineering/infrastructure/change-management/#feature-flags-and-the-change-management-process).
Cross link the issue here if it does.
- [ ] Ensure that you or a representative in development can be available for at least 2 hours after feature flag updates in production.
If a different developer will be covering, or an exception is needed, please inform the oncall SRE by using the `@sre-oncall` Slack alias.
- [ ] Ensure that documentation has been updated ([More info](https://docs.gitlab.com/ee/development/documentation/feature_flags.html#features-that-became-enabled-by-default)).
- [ ] Leave a comment on [the feature issue][main-issue] announcing estimated time when this feature flag will be enabled on GitLab.com.
- [ ] Ensure that any breaking changes have been announced following the [release post process](https://about.gitlab.com/handbook/marketing/blog/release-posts/#deprecations-removals-and-breaking-changes) to ensure GitLab customers are aware.
- [ ] Notify `#support_gitlab-com` and your team channel ([more guidance when this is necessary in the dev docs](https://docs.gitlab.com/ee/development/feature_flags/controls.html#communicate-the-change)).
- [ ] Ensure that the feature flag rollout plan is reviewed by another developer familiar with the domain.
- [ ] Announce on [the feature issue](ISSUE LINK) an estimated time this will be enabled on GitLab.com.
- [ ] If the feature flag in code has [an actor](https://docs.gitlab.com/ee/development/feature_flags/#feature-actors), enable it on GitLab.com for [testing groups/projects](#testing-groupsprojectsusers).
- [ ] `/chatops run feature set --<actor-type>=<actor> <feature-flag-name> true`
- [ ] Verify that the feature works as expected. Posting the QA result in this issue is preferable.
### Global rollout on production
For visibility, all `/chatops` commands that target production should be executed in the `#production` slack channel and cross-posted (with the command results) to the responsible team's slack channel (`#g_TEAM_NAME`).
- [ ] [Incrementally roll out](https://docs.gitlab.com/ee/development/feature_flags/controls.html#process) the feature.
- [ ] Between every step wait for at least 15 minutes and monitor the appropriate graphs on https://dashboards.gitlab.net.
- If the feature flag in code has [an actor](https://docs.gitlab.com/ee/development/feature_flags/#feature-actors), perform **actor-based** rollout.
- [ ] `/chatops run feature set <feature-flag-name> <rollout-percentage> --actors`
- If the feature flag in code does **NOT** have [an actor](https://docs.gitlab.com/ee/development/feature_flags/#feature-actors), perform time-based rollout (**random** rollout).
- [ ] `/chatops run feature set <feature-flag-name> <rollout-percentage> --random`
- [ ] `/chatops run feature set <feature-flag-name> <rollout-percentage>`
- Enable the feature globally on production environment.
- [ ] `/chatops run feature set <feature-flag-name> true`
- [ ] Observe appropriate graphs on https://dashboards.gitlab.net and verify that services are not affected.
- [ ] Leave a comment on [the feature issue][main-issue] announcing that the feature has been globally enabled.
- [ ] Announce on [the feature issue](ISSUE LINK) that the feature has been globally enabled.
- [ ] Cross-post chatops slack command to `#support_gitlab-com`.
([more guidance when this is necessary in the dev docs](https://docs.gitlab.com/ee/development/feature_flags/controls.html#communicate-the-change)) and in your team channel
- [ ] Wait for [at least one day for the verification term](https://about.gitlab.com/handbook/product-development-flow/feature-flag-lifecycle/#including-a-feature-behind-feature-flag-in-the-final-release).
### (Optional) Release the feature with the feature flag
@ -137,20 +104,12 @@ To do so, follow these steps:
- [ ] Create a merge request with the following changes. Ask for review and merge it.
- [ ] Set the `default_enabled` attribute in [the feature flag definition](https://docs.gitlab.com/ee/development/feature_flags/#feature-flag-definition-and-validation) to `true`.
- [ ] Review [what warrants a changelog entry](https://docs.gitlab.com/ee/development/changelog.html#what-warrants-a-changelog-entry) and decide if [a changelog entry](https://docs.gitlab.com/ee/development/feature_flags/#changelog) is needed.
- [ ] Ensure that the default-enabling MR has been included in the release package.
If the merge request was deployed before [the monthly release was tagged](https://about.gitlab.com/handbook/engineering/releases/#self-managed-releases-1),
- [ ] Create [a changelog entry](https://docs.gitlab.com/ee/development/feature_flags/#changelog).
- [ ] Ensure that the above MR has been deployed to both production and canary.
If the merge request was deployed before [the code cutoff](https://about.gitlab.com/handbook/engineering/releases/#self-managed-releases-1),
the feature can be officially announced in a release blog post.
- [ ] `/chatops run release check <merge-request-url> <milestone>`
- [ ] Consider cleaning up the feature flag from all environments by running these chatops command in `#production` channel. Otherwise these settings may override the default enabled.
- [ ] `/chatops run feature delete <feature-flag-name> --dev --staging --staging-ref --production`
- [ ] Close [the feature issue][main-issue] to indicate the feature will be released in the current milestone.
- [ ] Set the next milestone to this rollout issue for scheduling [the flag removal](#release-the-feature).
- [ ] (Optional) You can [create a separate issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Feature%20Flag%20Cleanup) for scheduling the steps below to [Release the feature](#release-the-feature).
- [ ] Set the title to "[Feature flag] Cleanup `<feature-flag-name>`".
- [ ] Execute the `/copy_metadata <this-rollout-issue-link>` quick action to copy the labels from this rollout issue.
- [ ] Link this rollout issue as a related issue.
- [ ] Close this rollout issue.
- [ ] `/chatops run auto_deploy status <merge-commit>`
- [ ] Close [the feature issue](ISSUE LINK) to indicate the feature will be released in the current milestone.
**WARNING:** This approach has the downside that it makes it difficult for us to
[clean up](https://docs.gitlab.com/ee/development/feature_flags/controls.html#cleaning-up) the flag.
@ -166,20 +125,20 @@ the [clean up](https://docs.gitlab.com/ee/development/feature_flags/controls.htm
should be done as soon as possible to permanently enable the feature and reduce complexity in the
codebase.
You can either [create a follow-up issue for Feature Flag Cleanup](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Feature%20Flag%20Cleanup) or use the checklist below in this same issue.
<!-- The checklist here is to help stakeholders keep track of the feature flag status -->
- [ ] Create a merge request to remove `<feature-flag-name>` feature flag. Ask for review and merge it.
- [ ] Remove all references to the feature flag from the codebase.
- [ ] Remove the YAML definitions for the feature from the repository.
- [ ] Create [a changelog entry](https://docs.gitlab.com/ee/development/feature_flags/#changelog).
- [ ] Ensure that the cleanup MR has been included in the release package.
If the merge request was deployed before [the monthly release was tagged](https://about.gitlab.com/handbook/engineering/releases/#self-managed-releases-1),
- [ ] Ensure that the above MR has been deployed to both production and canary.
If the merge request was deployed before [the code cutoff](https://about.gitlab.com/handbook/engineering/releases/#self-managed-releases-1),
the feature can be officially announced in a release blog post.
- [ ] `/chatops run release check <merge-request-url> <milestone>`
- [ ] Close [the feature issue][main-issue] to indicate the feature will be released in the current milestone.
- [ ] `/chatops run auto_deploy status <merge-commit>`
- [ ] Close [the feature issue](ISSUE LINK) to indicate the feature will be released in the current milestone.
- [ ] Clean up the feature flag from all environments by running these chatops command in `#production` channel:
- [ ] `/chatops run feature delete <feature-flag-name> --dev --staging --staging-ref --production`
- [ ] `/chatops run feature delete <feature-flag-name> --dev`
- [ ] `/chatops run feature delete <feature-flag-name> --staging`
- [ ] `/chatops run feature delete <feature-flag-name>`
- [ ] Close this rollout issue.
## Rollback Steps
@ -190,11 +149,5 @@ You can either [create a follow-up issue for Feature Flag Cleanup](https://gitla
/chatops run feature set <feature-flag-name> false
```
<!-- A feature flag can also be used for rolling out a bug fix or a maintenance work.
In this scenario, labels must be related to it, for example; ~"type::feature", ~"type::bug" or ~"type::maintenance".
Please use /copy_metadata to copy the labels from the issue you're rolling out. -->
/label ~group::
/label ~"feature flag"
/assign me
/due in 1 month
/assign DRI

View file

@ -5,15 +5,7 @@
<!-- Use this section to explain the feature and how it will work. It can be helpful to add technical details, design proposals, and links to related epics or issues. -->
<!-- Consider adding related issues and epics to this issue. You can also reference the Feature Proposal Template (https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Feature%20proposal%20-%20detailed.md) for additional details to consider adding to this issue. Additionally, as a data oriented organization, when your feature exits planning breakdown, consider adding the `What does success look like, and how can we measure that?` section.
-->
<!-- Label reminders
Use the following resources to find the appropriate labels:
- Use only one tier label choosing the lowest tier this is intended for
- https://gitlab.com/gitlab-org/gitlab/-/labels
- https://about.gitlab.com/handbook/product/categories/features/
-->
/label ~"group::" ~"section::" ~"Category::" ~"GitLab Core"/~"GitLab Premium"/~"GitLab Ultimate"
/label ~group:: ~section:: ~Category:
/label ~"GitLab Free" ~"GitLab Premium" ~"GitLab Ultimate"
/label ~"type::feature" ~"feature::addition" ~documentation
-->

View file

@ -1,6 +1,4 @@
<!-- This issue template can be used as a great starting point for feature requests. The section "Release notes" can be used as a summary of the feature and is also required if you want to have your release post blog MR auto generated using the release post item generator: https://about.gitlab.com/handbook/marketing/blog/release-posts/#release-post-item-generator. The remaining sections are the backbone for every feature in GitLab.
The goal of this template is brevity for quick/smaller iterations. For a more thorough list of considerations for larger features or feature sets, you can leverage the detailed [feature proposal](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Feature%20proposal%20-%20detailed.md). -->
<!-- This issue template can be used a great starting point for feature requests. The last section "Release notes" can be used as a summary of the feature and is also required if you want to have your release post blog MR auto generated using the release post item generator: https://about.gitlab.com/handbook/marketing/blog/release-posts/#release-post-item-generator. The remaining sections are the backbone for every feature in GitLab. -->
### Release notes
@ -14,9 +12,22 @@ The goal of this template is brevity for quick/smaller iterations. For a more th
<!-- Use this section to explain the feature and how it will work. It can be helpful to add technical details, design proposals, and links to related epics or issues. -->
/label ~"feature" ~"group::" ~"section::" ~"Category::" ~"GitLab Free"/~"GitLab Premium"/~"GitLab Ultimate"
<!--- Use the following resources to find the appropriate labels:
- https://gitlab.com/gitlab-org/gitlab/-/labels
- https://about.gitlab.com/handbook/product/categories/features/
Consider adding related issues and epics to this issue. You can also reference the Feature Proposal Template (https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Feature%20proposal%20-%20detailed.md) for additional details to consider adding to this issue. Additionally, as a data oriented organization, when your feature exits planning breakdown, consider adding the `What does success look like, and how can we measure that?` section.
Other sections to consider adding:
### Intended users
<!-- Who will use this feature? If known, include any of the following: types of users (e.g. Developer), personas, or specific company roles (e.g. Release Manager). It's okay to write "Unknown" and fill this field in later.
Who will use this feature? If known, include any of the following: types of users (e.g. Developer), personas, or specific company roles (e.g. Release Manager). It's okay to write "Unknown" and fill this field in later.
Personas are described at https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/
@ -25,34 +36,71 @@ Personas are described at https://about.gitlab.com/handbook/marketing/product-ma
* [Delaney (Development Team Lead)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#delaney-development-team-lead)
* [Presley (Product Designer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#presley-product-designer)
* [Sasha (Software Developer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sasha-software-developer)
* [Priyanka (Platform Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#priyanka-platform-engineer)
* [Devon (DevOps Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#devon-devops-engineer)
* [Sidney (Systems Administrator)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sidney-systems-administrator)
* [Sam (Security Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sam-security-analyst)
* [Rachel (Release Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#rachel-release-manager)
* [Alex (Security Operations Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#alex-security-operations-engineer)
* [Simone (Software Engineer in Test)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#simone-software-engineer-in-test)
* [Allison (Application Ops)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#allison-application-ops)
* [Ingrid (Infrastructure Operator)](https://about.gitlab.com/handbook/product/personas/#ingrid-infrastructure-operator)
* [Dakota (Application Development Director)](https://about.gitlab.com/handbook/product/personas/#dakota-application-development-director)
* [Priyanka (Platform Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#priyanka-platform-engineer)
* [Dana (Data Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#dana-data-analyst)
* [Eddie (Content Editor)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#eddie-content-editor)
-->
### Feature Usage Metrics
<!-- How are you going to track usage of this feature? Think about user behavior and their interaction with the product. What indicates someone is getting value from it?
### User experience goal
Create tracking issue using the Snowplow event tracking template. See https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Snowplow%20event%20tracking.md
What is the single user experience workflow this problem addresses?
For example, "The user should be able to use the UI/API/.gitlab-ci.yml with GitLab to <perform a specific task>"
https://about.gitlab.com/handbook/engineering/ux/ux-research-training/user-story-mapping/
-->
<!-- Label reminders
Use the following resources to find the appropriate labels:
- Use only one tier label choosing the lowest tier this is intended for
- https://gitlab.com/gitlab-org/gitlab/-/labels
- https://about.gitlab.com/handbook/product/categories/features/
-->
### Further details
/label ~group:: ~section:: ~Category:
/label ~"GitLab Free" ~"GitLab Premium" ~"GitLab Ultimate"
/label ~"type::feature" ~documentation ~direction
Include use cases, benefits, goals, or any other details that will help us understand the problem better.
### Permissions and Security
<!-- What permissions are required to perform the described actions? Are they consistent with the existing permissions as documented for users, groups, and projects as appropriate? Is the proposed behavior consistent between the UI, API, and other access methods (e.g. email replies)?
Consider adding checkboxes and expectations of users with certain levels of membership https://docs.gitlab.com/ee/user/permissions.html
* [ ] Add expected impact to members with no access (0)
* [ ] Add expected impact to Guest (10) members
* [ ] Add expected impact to Reporter (20) members
* [ ] Add expected impact to Developer (30) members
* [ ] Add expected impact to Maintainer (40) members
* [ ] Add expected impact to Owner (50) members
### Documentation
See the Feature Change Documentation Workflow https://docs.gitlab.com/ee/development/documentation/workflow.html#for-a-product-change
* Add all known Documentation Requirements in this section. See https://docs.gitlab.com/ee/development/documentation/workflow.html
* If this feature requires changing permissions, update the permissions document. See https://docs.gitlab.com/ee/user/permissions.html
### Availability & Testing
This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier.
What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing?
Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance.
* Unit test changes
* Integration test changes
* End-to-end test change
See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning
### What does success look like, and how can we measure that?
Define both the success metrics and acceptance criteria. Note that success metrics indicate the desired business outcomes, while acceptance criteria indicate when the solution is working correctly. If there is no way to measure success, link to an issue that will implement a way to measure this.
### What is the type of buyer?
What is the buyer persona for this feature? See https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/buyer-persona/
In which enterprise tier should this feature go? See https://about.gitlab.com/handbook/product/pricing/#three-tiers
### Is this a cross-stage feature?
Communicate if this change will affect multiple Stage Groups or product areas. We recommend always start with the assumption that a feature request will have an impact into another Group. Loop in the most relevant PM and Product Designer from that Group to provide strategic support to help align the Group's broader plan and vision, as well as to avoid UX and technical debt. https://about.gitlab.com/handbook/product/#cross-stage-features -->
/label ~documentation
/label ~direction

View file

@ -19,25 +19,24 @@ Personas are described at https://about.gitlab.com/handbook/marketing/product-ma
* [Delaney (Development Team Lead)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#delaney-development-team-lead)
* [Presley (Product Designer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#presley-product-designer)
* [Sasha (Software Developer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sasha-software-developer)
* [Priyanka (Platform Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#priyanka-platform-engineer)
* [Devon (DevOps Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#devon-devops-engineer)
* [Sidney (Systems Administrator)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sidney-systems-administrator)
* [Sam (Security Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sam-security-analyst)
* [Rachel (Release Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#rachel-release-manager)
* [Alex (Security Operations Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#alex-security-operations-engineer)
* [Simone (Software Engineer in Test)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#simone-software-engineer-in-test)
* [Allison (Application Ops)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#allison-application-ops)
* [Ingrid (Infrastructure Operator)](https://about.gitlab.com/handbook/product/personas/#ingrid-infrastructure-operator)
* [Dakota (Application Development Director)](https://about.gitlab.com/handbook/product/personas/#dakota-application-development-director)
* [Priyanka (Platform Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#priyanka-platform-engineer)
* [Dana (Data Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#dana-data-analyst)
* [Eddie (Content Editor)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#eddie-content-editor)
-->
### User experience goal
<!-- What is the single user experience workflow this problem addresses?
For example, "The user should be able to use the UI/API/.gitlab-ci.yml with GitLab to <perform a specific task>"
https://about.gitlab.com/handbook/product/ux/ux-research-training/user-story-mapping/ -->
https://about.gitlab.com/handbook/engineering/ux/ux-research-training/user-story-mapping/ -->
### Proposal
@ -56,12 +55,7 @@ Consider adding checkboxes and expectations of users with certain levels of memb
* [ ] Add expected impact to Reporter (20) members
* [ ] Add expected impact to Developer (30) members
* [ ] Add expected impact to Maintainer (40) members
* [ ] Add expected impact to Owner (50) members
Please consider performing a threat model for the code changes that are introduced as part of this feature. To get started, refer to our Threat Modeling handbook page https://about.gitlab.com/handbook/security/threat_modeling/#threat-modeling.
Don't hesitate to reach out to the Application Security Team (`@gitlab-com/gl-security/appsec`) to discuss any security concerns.
-->
* [ ] Add expected impact to Owner (50) members -->
### Documentation
@ -81,9 +75,7 @@ Please list the test areas (unit, integration and end-to-end) that needs to be a
* Integration test changes
* End-to-end test change
See the Quality Engineering quad planning and test planning processes and reach out to your counterpart Software Engineer in Test for assistance.
Quad Planning: https://about.gitlab.com/handbook/engineering/quality/quality-engineering/quad-planning
Test Planning: https://about.gitlab.com/handbook/engineering/quality/quality-engineering/test-engineering/#test-planning -->
See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning -->
### Available Tier
@ -94,20 +86,12 @@ Test Planning: https://about.gitlab.com/handbook/engineering/quality/quality-eng
* Ultimate/Gold
-->
### Feature Usage Metrics
<!-- How are you going to track usage of this feature? Think about user behavior and their interaction with the product. What indicates someone is getting value from it?
Create tracking issue using the Snowplow event tracking template. See https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Snowplow%20event%20tracking.md
-->
### What does success look like, and how can we measure that?
<!--
Define both the success metrics and acceptance criteria. Note that success metrics indicate the desired business outcomes, while acceptance criteria indicate when the solution is working correctly. If there is no way to measure success, link to an issue that will implement a way to measure this.
Create tracking issue using the Snowplow event tracking template. See https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Snowplow%20event%20tracking.md
Create tracking issue using the the Snowplow event tracking template. See https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Snowplow%20event%20tracking.md
-->
### What is the type of buyer?
@ -119,16 +103,15 @@ In which enterprise tier should this feature go? See https://about.gitlab.com/ha
<!-- Communicate if this change will affect multiple Stage Groups or product areas. We recommend always start with the assumption that a feature request will have an impact into another Group. Loop in the most relevant PM and Product Designer from that Group to provide strategic support to help align the Group's broader plan and vision, as well as to avoid UX and technical debt. https://about.gitlab.com/handbook/product/#cross-stage-features -->
### What is the competitive advantage or differentiation for this feature?
### Links / references
<!-- Label reminders - you should have one of each of the following labels.
Use the following resources to find the appropriate labels:
- Use only one tier label choosing the lowest tier this is intended for
- https://gitlab.com/gitlab-org/gitlab/-/labels
- https://about.gitlab.com/handbook/product/categories/features/
-->
/label ~group:: ~section:: ~Category:
/label ~"GitLab Free" ~"GitLab Premium" ~"GitLab Ultimate"
/label ~"type::feature" ~documentation ~direction
/label ~devops:: ~group: ~Category:
/label ~"GitLab Free"/~"GitLab Premium"/~"GitLab Ultimate"
/label ~feature
/label ~documentation
/label ~direction

View file

@ -1,59 +0,0 @@
<!-- This issue template can be used as a starting point for a UX Issue. This is not a feature request, rather an issue that is being created for a product designer to solve a problem.
The goal of this template is to ensure we have captured all the information available to the product designer so they can approach the problem creatively and efficiently. Please add links to SSOT if this informatin exists elsewhere. -->
### Who will use this solution?
<!-- If known, include any of the following: types of users (e.g. Developer), personas, or specific company roles (e.g. Release Manager). It's okay to write "Unknown" and fill this field in later.
Personas are described at https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/
* [Cameron (Compliance Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#cameron-compliance-manager)
* [Parker (Product Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#parker-product-manager)
* [Delaney (Development Team Lead)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#delaney-development-team-lead)
* [Presley (Product Designer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#presley-product-designer)
* [Sasha (Software Developer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sasha-software-developer)
* [Priyanka (Platform Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#priyanka-platform-engineer)
* [Sidney (Systems Administrator)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sidney-systems-administrator)
* [Sam (Security Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sam-security-analyst)
* [Rachel (Release Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#rachel-release-manager)
* [Alex (Security Operations Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#alex-security-operations-engineer)
* [Simone (Software Engineer in Test)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#simone-software-engineer-in-test)
* [Allison (Application Ops)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#allison-application-ops)
* [Ingrid (Infrastructure Operator)](https://about.gitlab.com/handbook/product/personas/#ingrid-infrastructure-operator)
* [Dakota (Application Development Director)](https://about.gitlab.com/handbook/product/personas/#dakota-application-development-director)
* [Dana (Data Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#dana-data-analyst)
* [Eddie (Content Editor)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#eddie-content-editor)
-->
### What problem do they have?
### When do they have the problem?
### Where in the app do they have the problem and at what frequency (if known)?
### Why will a design help them?
### What is the JTBD and/or Tasks?
### Is this problem supported by user research (please link relevant research issue/s)?
### Known technical constraints
### How does this help the business?
/label ~"group::" ~"section::" ~"Category:" ~UX

View file

@ -18,7 +18,7 @@ If your Model's pluralized form is non-standard, i.e. it doesn't just end in `s`
-->
## Replicate Cool Widgets - Repository
## Replicate Cool Widgets
This issue is for implementing Geo replication and verification of Cool Widgets.
@ -34,11 +34,10 @@ There are three main sections below. It is a good idea to structure your merge r
It is also a good idea to first open a proof-of-concept merge request. It can be helpful for working out kinks and getting initial support and feedback from the Geo team. As an example, see the [Proof of Concept to replicate Pipeline Artifacts](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/56423).
You can look into the following example for implementing replication/verification for a new Git repository type:
- [Add snippet repository verification](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/56596)
### Modify database schemas to prepare to add Geo support for Cool Widgets
You might do this section in its own merge request, but it is not required.
#### Add the registry table to track replication and verification state
Geo secondary sites have a [Geo tracking database](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/geo.md#tracking-database) independent of the main database. It is used to track the replication and verification state of all replicables. Every Model has a corresponding "registry" table in the Geo tracking database.
@ -46,7 +45,7 @@ Geo secondary sites have a [Geo tracking database](https://gitlab.com/gitlab-org
- [ ] Create the migration file in `ee/db/geo/migrate`:
```shell
bin/rails generate migration CreateCoolWidgetRegistry --database geo
bin/rails generate geo_migration CreateCoolWidgetRegistry
```
- [ ] Replace the contents of the migration file with the following. Note that we cannot add a foreign key constraint on `cool_widget_id` because the `cool_widgets` table is in a different database. The application code must handle logic such as propagating deletions.
@ -54,8 +53,14 @@ Geo secondary sites have a [Geo tracking database](https://gitlab.com/gitlab-org
```ruby
# frozen_string_literal: true
class CreateCoolWidgetRegistry < Gitlab::Database::Migration[2.1]
def change
class CreateCoolWidgetRegistry < ActiveRecord::Migration[6.0]
include Gitlab::Database::MigrationHelpers
disable_ddl_transaction!
def up
unless table_exists?(:cool_widget_registry)
ActiveRecord::Base.transaction do
create_table :cool_widget_registry, id: :bigserial, force: :cascade do |t|
t.bigint :cool_widget_id, null: false
t.datetime_with_timezone :created_at, null: false
@ -73,58 +78,147 @@ Geo secondary sites have a [Geo tracking database](https://gitlab.com/gitlab-org
t.boolean :missing_on_primary, default: false, null: false
t.binary :verification_checksum
t.binary :verification_checksum_mismatched
t.text :verification_failure, limit: 255
t.text :last_sync_failure, limit: 255
t.string :verification_failure, limit: 255 # rubocop:disable Migration/PreventStrings see https://gitlab.com/gitlab-org/gitlab/-/issues/323806
t.string :last_sync_failure, limit: 255 # rubocop:disable Migration/PreventStrings see https://gitlab.com/gitlab-org/gitlab/-/issues/323806
t.index :cool_widget_id, name: :index_cool_widget_registry_on_cool_widget_id, unique: true
t.index :retry_at
t.index :state
# To optimize performance of CoolWidgetRegistry.verification_failed_batch
t.index :verification_retry_at,
name: :cool_widget_registry_failed_verification,
order: "NULLS FIRST",
where: "((state = 2) AND (verification_state = 3))"
t.index :verification_retry_at, name: :cool_widget_registry_failed_verification, order: "NULLS FIRST", where: "((state = 2) AND (verification_state = 3))"
# To optimize performance of CoolWidgetRegistry.needs_verification_count
t.index :verification_state,
name: :cool_widget_registry_needs_verification,
where: "((state = 2) AND (verification_state = ANY (ARRAY[0, 3])))"
t.index :verification_state, name: :cool_widget_registry_needs_verification, where: "((state = 2) AND (verification_state = ANY (ARRAY[0, 3])))"
# To optimize performance of CoolWidgetRegistry.verification_pending_batch
t.index :verified_at,
name: :cool_widget_registry_pending_verification,
order: "NULLS FIRST",
where: "((state = 2) AND (verification_state = 0))"
t.index :verified_at, name: :cool_widget_registry_pending_verification, order: "NULLS FIRST", where: "((state = 2) AND (verification_state = 0))"
end
end
end
end
def down
drop_table :cool_widget_registry
end
end
```
- [ ] If deviating from the above example, then be sure to order columns according to [our guidelines](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/ordering_table_columns.md).
- [ ] Run Geo tracking database migrations:
```shell
bin/rake geo:db:migrate
```
- [ ] Be sure to commit the relevant changes in `ee/db/geo/schema.rb`
### Add verification state fields on the Geo primary site
The Geo primary site needs to checksum every replicable in order for secondaries to verify their own checksums. To do this, Geo requires fields on the Model. There are two ways to add the necessary verification state fields. If the table is large and wide, then it may be a good idea to add verification state fields to a separate table (Option 2). Consult a database expert if needed.
#### Add verification state fields to the model table (Option 1)
- [ ] Create the migration file in `db/migrate`:
```shell
bin/rails generate migration AddVerificationStateToCoolWidgets
```
- [ ] Replace the contents of the migration file with:
```ruby
# frozen_string_literal: true
class AddVerificationStateToCoolWidgets < ActiveRecord::Migration[6.0]
def change
change_table(:cool_widgets) do |t|
t.integer :verification_state, default: 0, limit: 2, null: false
t.column :verification_started_at, :datetime_with_timezone
t.integer :verification_retry_count, limit: 2, null: false
t.column :verification_retry_at, :datetime_with_timezone
t.column :verified_at, :datetime_with_timezone
t.binary :verification_checksum, using: 'verification_checksum::bytea'
t.text :verification_failure # rubocop:disable Migration/AddLimitToTextColumns
end
end
end
```
- [ ] If deviating from the above example, then be sure to order columns according to [our guidelines](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/ordering_table_columns.md).
- [ ] Add the new table to the [database dictionary](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/database/database_dictionary.md) defined in [`ee/db/geo/docs/`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/db/geo/docs):
```yaml
table_name: cool_widget_registry
description: Description example
introduced_by_url: Merge request link
milestone: Milestone example
feature_categories:
- Feature category example
classes:
- Class example
gitlab_schema: gitlab_geo
```
- [ ] Run Geo tracking database migrations:
- [ ] If `cool_widgets` is a high-traffic table, follow [the database documentation to use `with_lock_retries`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/migration_style_guide.md#when-to-use-the-helper-method)
- [ ] Adding a `text` column also [requires](../database/strings_and_the_text_data_type.md#add-a-text-column-to-an-existing-table) setting a limit. Create the migration file in `db/migrate`:
```shell
bin/rake db:migrate:geo
bin/rails generate migration AddVerificationFailureLimitToCoolWidgets
```
- [ ] Be sure to commit the relevant changes in `ee/db/geo/structure.sql` and the file under `ee/db/geo/schema_migrations`
- [ ] Replace the contents of the migration file with:
### Add verification state to the Model
```ruby
# frozen_string_literal: true
The Geo primary site needs to checksum every replicable so secondaries can verify their own checksums. To do this, Geo requires the Model to have an associated table to track verification state.
class AddVerificationFailureLimitToCoolWidgets < ActiveRecord::Migration[6.0]
include Gitlab::Database::MigrationHelpers
disable_ddl_transaction!
CONSTRAINT_NAME = 'cool_widget_verification_failure_text_limit'
def up
add_text_limit :cool_widget, :verification_failure, 255, constraint_name: CONSTRAINT_NAME
end
def down
remove_check_constraint(:cool_widget, CONSTRAINT_NAME)
end
end
```
- [ ] Add indexes on verification fields to ensure verification can be performed efficiently. Some or all of these indexes can be omitted if the table is guaranteed to be small. Ask a database expert if you are considering omitting indexes. Create the migration file in `db/migrate`:
```shell
bin/rails generate migration AddVerificationIndexesToCoolWidgets
```
- [ ] Replace the contents of the migration file with:
```ruby
# frozen_string_literal: true
class AddVerificationIndexesToCoolWidgets < ActiveRecord::Migration[6.0]
include Gitlab::Database::MigrationHelpers
VERIFICATION_STATE_INDEX_NAME = "index_cool_widgets_on_verification_state"
PENDING_VERIFICATION_INDEX_NAME = "index_cool_widgets_pending_verification"
FAILED_VERIFICATION_INDEX_NAME = "index_cool_widgets_failed_verification"
NEEDS_VERIFICATION_INDEX_NAME = "index_cool_widgets_needs_verification"
disable_ddl_transaction!
def up
add_concurrent_index :cool_widgets, :verification_state, name: VERIFICATION_STATE_INDEX_NAME
add_concurrent_index :cool_widgets, :verified_at, where: "(verification_state = 0)", order: { verified_at: 'ASC NULLS FIRST' }, name: PENDING_VERIFICATION_INDEX_NAME
add_concurrent_index :cool_widgets, :verification_retry_at, where: "(verification_state = 3)", order: { verification_retry_at: 'ASC NULLS FIRST' }, name: FAILED_VERIFICATION_INDEX_NAME
add_concurrent_index :cool_widgets, :verification_state, where: "(verification_state = 0 OR verification_state = 3)", name: NEEDS_VERIFICATION_INDEX_NAME
end
def down
remove_concurrent_index_by_name :cool_widgets, VERIFICATION_STATE_INDEX_NAME
remove_concurrent_index_by_name :cool_widgets, PENDING_VERIFICATION_INDEX_NAME
remove_concurrent_index_by_name :cool_widgets, FAILED_VERIFICATION_INDEX_NAME
remove_concurrent_index_by_name :cool_widgets, NEEDS_VERIFICATION_INDEX_NAME
end
end
```
- [ ] Run database migrations:
```shell
bin/rake db:migrate
```
- [ ] Be sure to commit the relevant changes in `db/structure.sql`
#### Add verification state fields to a separate table (Option 2)
- [ ] Create the migration file in `db/migrate`:
@ -137,39 +231,39 @@ The Geo primary site needs to checksum every replicable so secondaries can verif
```ruby
# frozen_string_literal: true
class CreateCoolWidgetStates < Gitlab::Database::Migration[2.1]
class CreateCoolWidgetStates < ActiveRecord::Migration[6.0]
include Gitlab::Database::MigrationHelpers
VERIFICATION_STATE_INDEX_NAME = "index_cool_widget_states_on_verification_state"
PENDING_VERIFICATION_INDEX_NAME = "index_cool_widget_states_pending_verification"
FAILED_VERIFICATION_INDEX_NAME = "index_cool_widget_states_failed_verification"
NEEDS_VERIFICATION_INDEX_NAME = "index_cool_widget_states_needs_verification"
enable_lock_retries!
disable_ddl_transaction!
def up
unless table_exists?(:cool_widget_states)
with_lock_retries do
create_table :cool_widget_states, id: false do |t|
t.datetime_with_timezone :verification_started_at
t.references :cool_widget, primary_key: true, null: false, foreign_key: { on_delete: :cascade }
t.integer :verification_state, default: 0, limit: 2, null: false
t.column :verification_started_at, :datetime_with_timezone
t.datetime_with_timezone :verification_retry_at
t.datetime_with_timezone :verified_at
t.references :cool_widget, primary_key: true, default: nil, index: false, foreign_key: { on_delete: :cascade }
t.integer :verification_state, default: 0, limit: 2, null: false
t.integer :verification_retry_count, default: 0, limit: 2, null: false
t.integer :verification_retry_count, limit: 2
t.binary :verification_checksum, using: 'verification_checksum::bytea'
t.text :verification_failure, limit: 255
t.text :verification_failure
t.index :verification_state, name: VERIFICATION_STATE_INDEX_NAME
t.index :verified_at,
where: "(verification_state = 0)",
order: { verified_at: 'ASC NULLS FIRST' },
name: PENDING_VERIFICATION_INDEX_NAME
t.index :verification_retry_at,
where: "(verification_state = 3)",
order: { verification_retry_at: 'ASC NULLS FIRST' },
name: FAILED_VERIFICATION_INDEX_NAME
t.index :verification_state,
where: "(verification_state = 0 OR verification_state = 3)",
name: NEEDS_VERIFICATION_INDEX_NAME
t.index :verified_at, where: "(verification_state = 0)", order: { verified_at: 'ASC NULLS FIRST' }, name: PENDING_VERIFICATION_INDEX_NAME
t.index :verification_retry_at, where: "(verification_state = 3)", order: { verification_retry_at: 'ASC NULLS FIRST' }, name: FAILED_VERIFICATION_INDEX_NAME
t.index :verification_state, where: "(verification_state = 0 OR verification_state = 3)", name: NEEDS_VERIFICATION_INDEX_NAME
end
end
end
add_text_limit :cool_widget_states, :verification_failure, 255
end
def down
drop_table :cool_widget_states
@ -178,31 +272,13 @@ The Geo primary site needs to checksum every replicable so secondaries can verif
```
- [ ] If deviating from the above example, then be sure to order columns according to [our guidelines](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/ordering_table_columns.md).
- [ ] If `cool_widgets` is a high-traffic table, follow [the database documentation to use `with_lock_retries`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/migration_style_guide.md#when-to-use-the-helper-method)
- [ ] Add the new table to the [database dictionary](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/database/database_dictionary.md) defined in [`db/docs/`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/db/docs):
```yaml
---
table_name: cool_widget_states
description: Separate table for cool widget verification states
introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/XXXXX
milestone: 'XX.Y'
feature_categories:
- geo_replication
classes:
- Geo::CoolWidgetState
gitlab_schema: gitlab_main
```
- [ ] Run database migrations:
```shell
bin/rake db:migrate
```
- [ ] Be sure to commit the relevant changes in `db/structure.sql` and the file under `db/schema_migrations`
- [ ] Be sure to commit the relevant changes in `db/structure.sql`
That's all of the required database changes.
@ -210,14 +286,7 @@ That's all of the required database changes.
#### Step 1. Implement replication and verification
- [ ] Add the following lines to the `cool_widget` model to accomplish some important tasks:
- Include `::Geo::ReplicableModel` in the `CoolWidget` class, and specify the Replicator class `with_replicator Geo::CoolWidgetReplicator`.
- Include the `::Geo::VerifiableModel` concern.
- Delegate verification related methods to the `cool_widget_state` model.
- For verification, override some scopes to use the `cool_widget_states` table instead of the model table.
- Implement the `verification_state_object` method to return the object that holds
the verification details
- Override some methods to use the `cool_widget_states` table in verification-related queries.
- [ ] Include `Gitlab::Geo::ReplicableModel` in the `CoolWidget` class, and specify the Replicator class `with_replicator Geo::CoolWidgetReplicator`.
Pay some attention to method `pool_repository`. Not every repository type uses repository pooling. As Geo prefers to use repository snapshotting, it can lead to data loss. Make sure to overwrite `pool_repository` so it returns nil for repositories that do not have pools.
@ -227,59 +296,28 @@ That's all of the required database changes.
# frozen_string_literal: true
class CoolWidget < ApplicationRecord
...
include ::Geo::ReplicableModel
include ::Geo::VerifiableModel
delegate(*::Geo::VerificationState::VERIFICATION_METHODS, to: :cool_widget_state)
include ::Gitlab::Geo::ReplicableModel
include ::Gitlab::Geo::VerificationState
with_replicator Geo::CoolWidgetReplicator
has_one :cool_widget_state, autosave: false, inverse_of: :cool_widget, class_name: 'Geo::CoolWidgetState'
after_save :save_verification_details
mount_uploader :file, CoolWidgetUploader
# Override the `all` default if not all records can be replicated. For an
# example of an existing Model that needs to do this, see
# `EE::MergeRequestDiff`.
# scope :available_replicables, -> { all }
scope :available_verifiables, -> { joins(:cool_widget_state) }
scope :checksummed, -> {
joins(:cool_widget_state).where.not(cool_widget_states: { verification_checksum: nil })
}
scope :not_checksummed, -> {
joins(:cool_widget_state).where(cool_widget_states: { verification_checksum: nil })
}
scope :with_verification_state, ->(state) {
joins(:cool_widget_state)
.where(cool_widget_states: { verification_state: verification_state_value(state) })
}
def verification_state_object
cool_widget_state
end
...
class_methods do
extend ::Gitlab::Utils::Override
...
# @param primary_key_in [Range, CoolWidget] arg to pass to primary_key_in scope
# @return [ActiveRecord::Relation<CoolWidget>] everything that should be synced
# to this node, restricted by primary key
def replicables_for_current_secondary(primary_key_in)
# @return [ActiveRecord::Relation<CoolWidget>] everything that should be synced to this node, restricted by primary key
def self.replicables_for_current_secondary(primary_key_in)
# This issue template does not help you write this method.
#
# This method is called only on Geo secondary sites. It is called when
# we want to know which records to replicate. This is not easy to automate
# because for example:
#
# * The "selective sync" feature allows admins to choose which namespaces
# to replicate, per secondary site. Most Models are scoped to a
# * The "selective sync" feature allows admins to choose which namespaces # to replicate, per secondary site. Most Models are scoped to a
# namespace, but the nature of the relationship to a namespace varies
# between Models.
# * The "selective sync" feature allows admins to choose which shards to
@ -292,34 +330,48 @@ That's all of the required database changes.
# Search the codebase for examples, and consult a Geo expert if needed.
end
override :verification_state_table_class
def verification_state_table_class
CoolWidgetState
end
end
# Geo checks this method in FrameworkRepositorySyncService to avoid
# snapshotting repositories using object pools
def pool_repository
nil
end
def cool_widget_state
super || build_cool_widget_state
end
...
end
```
- [ ] Implement `CoolWidget.replicables_for_current_secondary` above.
- [ ] Ensure `CoolWidget.replicables_for_current_secondary` is well-tested. Search the codebase for `replicables_for_current_secondary` to find examples of parameterized table specs. You may need to add more `FactoryBot` traits.
- [ ] Add the following shared examples to `ee/spec/models/ee/cool_widget_spec.rb`:
- [ ] If you are using a separate table `cool_widget_states` to track verification state on the Geo primary site, then:
- [ ] Do not include `::Gitlab::Geo::VerificationState` on the `CoolWidget` class.
- [ ] Add the following lines to the `cool_widget_state.rb` model:
```ruby
include_examples 'a replicable model with a separate table for verification state' do
let(:verifiable_model_record) { build(:cool_widget) } # add extra params if needed to make sure the record is in `Geo::ReplicableModel.verifiables` scope
let(:unverifiable_model_record) { build(:cool_widget) } # add extra params if needed to make sure the record is NOT included in `Geo::ReplicableModel.verifiables` scope
class CoolWidgetState < ApplicationRecord
...
self.primary_key = :cool_widget_id
include ::Gitlab::Geo::VerificationState
belongs_to :cool_widget, inverse_of: :cool_widget_state
...
end
```
- [ ] Add the following lines to the `cool_widget` model:
```ruby
class CoolWidget < ApplicationRecord
...
has_one :cool_widget_state, inverse_of: :cool_widget
delegate :verification_retry_at, :verification_retry_at=,
:verified_at, :verified_at=,
:verification_checksum, :verification_checksum=,
:verification_failure, :verification_failure=,
:verification_retry_count, :verification_retry_count=,
to: :cool_widget_state
...
end
```
@ -331,18 +383,23 @@ That's all of the required database changes.
module Geo
class CoolWidgetReplicator < Gitlab::Geo::Replicator
include ::Geo::RepositoryReplicatorStrategy
extend ::Gitlab::Utils::Override
def self.model
::CoolWidget
end
def repository
model_record.repository
end
def self.git_access_class
::Gitlab::GitAccessCoolWidget
end
def self.no_repo_message
git_access_class.error_message(:no_repo)
# The feature flag follows the format `geo_#{replicable_name}_replication`,
# so here it would be `geo_cool_widget_replication`
def self.replication_enabled_by_default?
false
end
override :verification_feature_flag_enabled?
@ -354,19 +411,6 @@ That's all of the required database changes.
true
end
override :housekeeping_enabled?
def self.housekeeping_enabled?
# Remove this method if the new Git repository type supports git
# repository housekeeping and the ::CoolWidget#git_garbage_collect_worker_klass
# is implemented. If the data type requires any action to be performed
# before running the housekeeping override the `before_housekeeping` method
# (see `RepositoryReplicatorStrategy#before_housekeeping`)
false
end
def repository
model_record.repository
end
end
end
```
@ -390,10 +434,7 @@ That's all of the required database changes.
```
- [ ] Make sure a Geo secondary site can request and download Cool Widgets on the Geo primary site. You may need to make some changes to `Gitlab::GitAccessCoolWidget`. For example, see [this change for Group-level Wikis](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/54914/diffs?commit_id=0f2b36f66697b4addbc69bd377ee2818f648dd33).
- [ ] Make sure a Geo secondary site can replicate Cool Widgets where repository does not exist on the Geo primary site. The only way to know about this is to parse the error text. You may need to make some changes to `Gitlab::CoolWidgetReplicator.no_repo_message` to return the proper error message. For example, see [this change for Group-level Wikis](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/74133).
- [ ] Generate the feature flag definition files by running the feature flag commands and following the command prompts:
- [ ] Generate the feature flag definition file by running the feature flag command and following the command prompts:
```shell
bin/feature-flag --ee geo_cool_widget_replication --type development --group 'group::geo'
@ -407,6 +448,7 @@ That's all of the required database changes.
::Geo::PackageFileReplicator,
::Geo::CoolWidgetReplicator
]
end
```
- [ ] Create `ee/spec/replicators/geo/cool_widget_replicator_spec.rb` and perform the necessary setup to define the `model_record` variable for the shared examples:
@ -416,7 +458,7 @@ That's all of the required database changes.
require 'spec_helper'
RSpec.describe Geo::CoolWidgetReplicator, feature_category: :geo_replication do
RSpec.describe Geo::CoolWidgetReplicator do
let(:model_record) { build(:cool_widget) }
include_examples 'a repository replicator'
@ -429,8 +471,7 @@ That's all of the required database changes.
```ruby
# frozen_string_literal: true
module Geo
class CoolWidgetRegistry < Geo::BaseRegistry
class Geo::CoolWidgetRegistry < Geo::BaseRegistry
include ::Geo::ReplicableRegistry
include ::Geo::VerifiableRegistry
@ -439,11 +480,10 @@ That's all of the required database changes.
belongs_to :cool_widget, class_name: 'CoolWidget'
end
end
```
- [ ] Update `REGISTRY_CLASSES` in `ee/app/workers/geo/secondary/registry_consistency_worker.rb`.
- [ ] Add a custom factory name if needed in `def model_class_factory_name` in `ee/spec/support/helpers/ee/geo_helpers.rb`.
- [ ] Update `def model_class_factory_name` in `ee/spec/services/geo/registry_consistency_service_spec.rb`.
- [ ] Update `it 'creates missing registries for each registry class'` in `ee/spec/workers/geo/secondary/registry_consistency_worker_spec.rb`.
- [ ] Add `cool_widget_registry` to `ActiveSupport::Inflector.inflections` in `config/initializers_before_autoloader/000_inflections.rb`.
- [ ] Create `ee/spec/factories/geo/cool_widget_registry.rb`:
@ -453,7 +493,7 @@ That's all of the required database changes.
FactoryBot.define do
factory :geo_cool_widget_registry, class: 'Geo::CoolWidgetRegistry' do
cool_widget # This association should have data, like a file or repository
cool_widget
state { Geo::CoolWidgetRegistry.state_value(:pending) }
trait :synced do
@ -465,7 +505,6 @@ That's all of the required database changes.
state { Geo::CoolWidgetRegistry.state_value(:failed) }
last_synced_at { 1.day.ago }
retry_count { 2 }
retry_at { 2.hours.from_now }
last_sync_failure { 'Random error' }
end
@ -474,12 +513,6 @@ That's all of the required database changes.
last_synced_at { 1.day.ago }
retry_count { 0 }
end
trait :verification_succeeded do
verification_checksum { 'e079a831cab27bcda7d81cd9b48296d0c3dd92ef' }
verification_state { Geo::CoolWidgetRegistry.verification_state_value(:verification_succeeded) }
verified_at { 5.days.ago }
end
end
end
```
@ -491,7 +524,7 @@ That's all of the required database changes.
require 'spec_helper'
RSpec.describe Geo::CoolWidgetRegistry, :geo, type: :model, feature_category: :geo_replication do
RSpec.describe Geo::CoolWidgetRegistry, :geo, type: :model do
let_it_be(:registry) { create(:geo_cool_widget_registry) }
specify 'factory is valid' do
@ -503,73 +536,6 @@ That's all of the required database changes.
end
```
- [ ] Add the following to `ee/spec/factories/cool_widgets.rb`:
```ruby
# frozen_string_literal: true
FactoryBot.modify do
factory :cool_widget do
trait :verification_succeeded do
with_file
verification_checksum { 'abc' }
verification_state { CoolWidget.verification_state_value(:verification_succeeded) }
end
trait :verification_failed do
with_file
verification_failure { 'Could not calculate the checksum' }
verification_state { CoolWidget.verification_state_value(:verification_failed) }
end
end
end
```
If there is not an existing factory for the object in `spec/factories/cool_widgets.rb`, wrap the traits in `FactoryBot.create` instead of `FactoryBot.modify`.
- [ ] Make sure the factory also allows setting a `project` attribute. If the model does not have a direct relation to a project, you can use a `transient` attribute. Check out `spec/factories/merge_request_diffs.rb` for an example.
- [ ] Following [the example of Merge Request Diffs](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63309) add a `Geo::CoolWidgetState` model in `ee/app/models/geo/cool_widget_state.rb`:
``` ruby
# frozen_string_literal: true
module Geo
class CoolWidgetState < ApplicationRecord
include ::Geo::VerificationStateDefinition
self.primary_key = :cool_widget_id
belongs_to :cool_widget, inverse_of: :cool_widget_state
validates :verification_failure, length: { maximum: 255 }
validates :verification_state, :cool_widget, presence: true
end
end
```
- [ ] Add a `factory` for `cool_widget_state`, in `ee/spec/factories/geo/cool_widget_states.rb`:
``` ruby
# frozen_string_literal: true
FactoryBot.define do
factory :geo_cool_widget_state, class: 'Geo::CoolWidgetState' do
cool_widget
trait :checksummed do
verification_checksum { 'abc' }
end
trait :checksum_failure do
verification_failure { 'Could not calculate the checksum' }
end
end
end
```
- [ ] Add `[:geo_cool_widget_state, any]` to `skipped` in `spec/models/factories_spec.rb`
#### Step 2. Implement metrics gathering
Metrics are gathered by `Geo::MetricsUpdateWorker`, persisted in `GeoNodeStatus` for display in the UI, and sent to Prometheus:
@ -590,21 +556,41 @@ Metrics are gathered by `Geo::MetricsUpdateWorker`, persisted in `GeoNodeStatus`
- [ ] Add the same fields to `GET /geo_nodes/status` example response in
`ee/spec/fixtures/api/schemas/public_api/v4/geo_node_status.json`.
- [ ] Add the following fields to the `Sidekiq metrics` table in `doc/administration/monitoring/prometheus/gitlab_metrics.md`:
```markdown
| `geo_cool_widgets` | Gauge | XX.Y | Number of Cool Widgets on primary | `url` |
| `geo_cool_widgets_checksum_total` | Gauge | XX.Y | Number of Cool Widgets to checksum on primary | `url` |
| `geo_cool_widgets_checksummed` | Gauge | XX.Y | Number of Cool Widgets that successfully calculated the checksum on primary | `url` |
| `geo_cool_widgets_checksum_failed` | Gauge | XX.Y | Number of Cool Widgets that failed to calculate the checksum on primary | `url` |
| `geo_cool_widgets_synced` | Gauge | XX.Y | Number of syncable Cool Widgets synced on secondary | `url` |
| `geo_cool_widgets_failed` | Gauge | XX.Y | Number of syncable Cool Widgets failed to sync on secondary | `url` |
| `geo_cool_widgets_registry` | Gauge | XX.Y | Number of Cool Widgets in the registry | `url` |
| `geo_cool_widgets_verification_total` | Gauge | XX.Y | Number of Cool Widgets to attempt to verify on secondary | `url` |
| `geo_cool_widgets_verified` | Gauge | XX.Y | Number of Cool Widgets successfully verified on secondary | `url` |
| `geo_cool_widgets_verification_failed` | Gauge | XX.Y | Number of Cool Widgets that failed verification on secondary | `url` |
```
- [ ] Run the rake task `geo:dev:ssf_metrics` and commit the changes to `ee/config/metrics/object_schemas/geo_node_usage.json`
- `geo_cool_widgets`
- `geo_cool_widgets_checksum_total`
- `geo_cool_widgets_checksummed`
- `geo_cool_widgets_checksum_failed`
- `geo_cool_widgets_synced`
- `geo_cool_widgets_failed`
- `geo_cool_widgets_registry`
- `geo_cool_widgets_verification_total`
- `geo_cool_widgets_verified`
- `geo_cool_widgets_verification_failed`
- [ ] Add the following to the parameterized table in the `context 'Replicator stats' do` block in `ee/spec/models/geo_node_status_spec.rb`:
Cool Widget replication and verification metrics should now be available in the API, the `Admin > Geo > Sites` view, and Prometheus.
```ruby
Geo::CoolWidgetReplicator | :cool_widget | :geo_cool_widget_registry
```
- [ ] Add the following to `spec/factories/cool_widgets.rb`:
```ruby
trait(:verification_succeeded) do
with_file
verification_checksum { 'abc' }
verification_state { CoolWidget.verification_state_value(:verification_succeeded) }
end
trait(:verification_failed) do
with_file
verification_failure { 'Could not calculate the checksum' }
verification_state { CoolWidget.verification_state_value(:verification_failed) }
end
```
- [ ] Make sure the factory also allows setting a `project` attribute. If the model does not have a direct relation to a project, you can use a `transient` attribute. Check out `spec/factories/merge_request_diffs.rb` for an example.
Cool Widget replication and verification metrics should now be available in the API, the `Admin > Geo > Nodes` view, and Prometheus.
#### Step 3. Implement the GraphQL API
@ -616,9 +602,8 @@ The GraphQL API is used by `Admin > Geo > Replication Details` views, and is dir
field :cool_widget_registries, ::Types::Geo::CoolWidgetRegistryType.connection_type,
null: true,
resolver: ::Resolvers::Geo::CoolWidgetRegistriesResolver,
description: 'Find Cool Widget registries on this Geo node. '\
'Ignored if `geo_cool_widget_replication` feature flag is disabled.',
alpha: { milestone: '15.5' } # Update the milestone
description: 'Find Cool Widget registries on this Geo node',
feature_flag: :geo_cool_widget_replication
```
- [ ] Add the new `cool_widget_registries` field name to the `expected_fields` array in `ee/spec/graphql/types/geo/geo_node_type_spec.rb`.
@ -645,7 +630,7 @@ The GraphQL API is used by `Admin > Geo > Replication Details` views, and is dir
require 'spec_helper'
RSpec.describe Resolvers::Geo::CoolWidgetRegistriesResolver, feature_category: :geo_replication do
RSpec.describe Resolvers::Geo::CoolWidgetRegistriesResolver do
it_behaves_like 'a Geo registries resolver', :geo_cool_widget_registry
end
```
@ -669,7 +654,7 @@ The GraphQL API is used by `Admin > Geo > Replication Details` views, and is dir
require 'spec_helper'
RSpec.describe Geo::CoolWidgetRegistryFinder, feature_category: :geo_replication do
RSpec.describe Geo::CoolWidgetRegistryFinder do
it_behaves_like 'a framework registry finder', :geo_cool_widget_registry
end
```
@ -683,15 +668,13 @@ The GraphQL API is used by `Admin > Geo > Replication Details` views, and is dir
module Geo
# rubocop:disable Graphql/AuthorizeTypes because it is included
class CoolWidgetRegistryType < BaseObject
graphql_name 'CoolWidgetRegistry'
include ::Types::Geo::RegistryType
graphql_name 'CoolWidgetRegistry'
description 'Represents the Geo replication and verification state of a cool_widget'
field :cool_widget_id, GraphQL::Types::ID, null: false, description: 'ID of the Cool Widget.'
field :cool_widget_id, GraphQL::ID_TYPE, null: false, description: 'ID of the Cool Widget'
end
# rubocop:enable Graphql/AuthorizeTypes
end
end
```
@ -703,7 +686,7 @@ The GraphQL API is used by `Admin > Geo > Replication Details` views, and is dir
require 'spec_helper'
RSpec.describe GitlabSchema.types['CoolWidgetRegistry'], feature_category: :geo_replication do
RSpec.describe GitlabSchema.types['CoolWidgetRegistry'] do
it_behaves_like 'a Geo registry type'
it 'has the expected fields (other than those included in RegistryType)' do
@ -733,49 +716,6 @@ The GraphQL API is used by `Admin > Geo > Replication Details` views, and is dir
Individual Cool Widget replication and verification data should now be available via the GraphQL API.
#### Step 4. Handle batch destroy
If batch destroy logic is implemented for a replicable, then that logic must be "replicated" by Geo secondaries. The easiest way to do this is use `Geo::BatchEventCreateWorker` to bulk insert a delete event for each replicable.
For example, if `FastDestroyAll` is used, then you may be able to [use `begin_fast_destroy` and `finalize_fast_destroy` hooks, like we did for uploads](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/69763).
Or if a special service is used to batch delete records and their associated data, then you probably need to [hook into that service, like we did for job artifacts](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/79530).
As illustrated by the above two examples, batch destroy logic cannot be handled automatically by Geo secondaries without restricting the way other teams perform batch destroys. It is up to you to produce `Geo::BatchEventCreateWorker` attributes before the records are deleted, and then enqueue `Geo::BatchEventCreateWorker` after the records are deleted.
- [ ] Ensure that any batch destroy of this replicable is replicated to secondary sites
- [ ] Regardless of implementation details, please verify in specs that when the parent object is removed, the new `Geo::Event` records are created:
```ruby
describe '#destroy' do
subject { create(:cool_widget) }
context 'when running in a Geo primary node' do
let_it_be(:primary) { create(:geo_node, :primary) }
let_it_be(:secondary) { create(:geo_node) }
it 'logs an event to the Geo event log when bulk removal is used', :sidekiq_inline do
stub_current_geo_node(primary)
expect { subject.project.destroy! }.to change(Geo::Event.where(replicable_name: :cool_widget, event_name: :deleted), :count).by(1)
payload = Geo::Event.where(replicable_name: :cool_widget, event_name: :deleted).last.payload
expect(payload['model_record_id']).to eq(subject.id)
expect(payload['blob_path']).to eq(subject.relative_path)
expect(payload['uploader_class']).to eq('CoolWidgetUploader')
end
end
end
```
### Code Review
When requesting review from database reviewers:
- [ ] Include a comment mentioning that the change is based on a documented template.
- [ ] `replicables_for_current_secondary` and `available_replicables` may differ per Model. If their queries are new, then add [query plans](https://docs.gitlab.com/ee/development/database_review.html#query-plans) to the MR description. An easy place to gather SQL queries is your GDK's `log/test.log` when running tests of these methods.
### Release Geo support of Cool Widgets
- [ ] In the rollout issue you created when creating the feature flag, modify the Roll Out Steps:
@ -783,18 +723,34 @@ When requesting review from database reviewers:
- [ ] Add a step to `Test replication and verification of Cool Widgets on a non-GDK-deployment. For example, using GitLab Environment Toolkit`.
- [ ] Add a step to `Ping the Geo PM and EM to coordinate testing`. For example, you might add steps to generate Cool Widgets, and then a Geo engineer may take it from there.
- [ ] In `ee/config/feature_flags/development/geo_cool_widget_replication.yml`, set `default_enabled: true`
- [ ] In `ee/app/graphql/types/geo/geo_node_type.rb`, remove the `alpha` option for the released type:
- [ ] In `ee/app/replicators/geo/cool_widget_replicator.rb`, delete the `self.replication_enabled_by_default?` method:
```ruby
module Geo
class CoolWidgetReplicator < Gitlab::Geo::Replicator
...
# REMOVE THIS METHOD
def self.replication_enabled_by_default?
false
end
# REMOVE THIS METHOD
...
end
end
```
- [ ] In `ee/app/graphql/types/geo/geo_node_type.rb`, remove the `feature_flag` option for the released type:
```ruby
field :cool_widget_registries, ::Types::Geo::CoolWidgetRegistryType.connection_type,
null: true,
resolver: ::Resolvers::Geo::CoolWidgetRegistriesResolver,
description: 'Find Cool Widget registries on this Geo node. '\
'Ignored if `geo_cool_widget_replication` feature flag is disabled.',
alpha: { milestone: '15.5' } # Update the milestone
description: 'Find Cool Widget registries on this Geo node',
feature_flag: :geo_cool_widget_replication # REMOVE THIS LINE
```
- [ ] Run `bundle exec rake gitlab:graphql:compile_docs` after the step above to regenerate the GraphQL docs.
- [ ] Add a row for Cool Widgets to the `Data types` table in [Geo data types support](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/administration/geo/replication/datatypes.md#data-types)
- [ ] Add a row for Cool Widgets to the `Limitations on replication/verification` table in [Geo data types support](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/administration/geo/replication/datatypes.md#limitations-on-replicationverification). If the row already exists, then update it to show that Replication and Verification is released in the current version.

View file

@ -18,7 +18,7 @@ If your Model's pluralized form is non-standard, i.e. it doesn't just end in `s`
-->
## Replicate Cool Widgets - Blob
## Replicate Cool Widgets
This issue is for implementing Geo replication and verification of Cool Widgets.
@ -34,13 +34,10 @@ There are three main sections below. It is a good idea to structure your merge r
It is also a good idea to first open a proof-of-concept merge request. It can be helpful for working out kinks and getting initial support and feedback from the Geo team. As an example, see the [Proof of Concept to replicate Pipeline Artifacts](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/56423).
You can look into the following examples of MRs for implementing replication/verification for a new blob type:
- [Add db changes](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60935) and [add verification for MR diffs using SSF](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63309)
- [Verify Terraform state versions](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/58800)
- [Verify LFS objects](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63981)
### Modify database schemas to prepare to add Geo support for Cool Widgets
You might do this section in its own merge request, but it is not required.
#### Add the registry table to track replication and verification state
Geo secondary sites have a [Geo tracking database](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/geo.md#tracking-database) independent of the main database. It is used to track the replication and verification state of all replicables. Every Model has a corresponding "registry" table in the Geo tracking database.
@ -48,7 +45,7 @@ Geo secondary sites have a [Geo tracking database](https://gitlab.com/gitlab-org
- [ ] Create the migration file in `ee/db/geo/migrate`:
```shell
bin/rails generate migration CreateCoolWidgetRegistry --database geo
bin/rails generate geo_migration CreateCoolWidgetRegistry
```
- [ ] Replace the contents of the migration file with the following. Note that we cannot add a foreign key constraint on `cool_widget_id` because the `cool_widgets` table is in a different database. The application code must handle logic such as propagating deletions.
@ -56,8 +53,14 @@ Geo secondary sites have a [Geo tracking database](https://gitlab.com/gitlab-org
```ruby
# frozen_string_literal: true
class CreateCoolWidgetRegistry < Gitlab::Database::Migration[2.1]
def change
class CreateCoolWidgetRegistry < ActiveRecord::Migration[6.0]
include Gitlab::Database::MigrationHelpers
disable_ddl_transaction!
def up
unless table_exists?(:cool_widget_registry)
ActiveRecord::Base.transaction do
create_table :cool_widget_registry, id: :bigserial, force: :cascade do |t|
t.bigint :cool_widget_id, null: false
t.datetime_with_timezone :created_at, null: false
@ -73,60 +76,147 @@ Geo secondary sites have a [Geo tracking database](https://gitlab.com/gitlab-org
t.boolean :checksum_mismatch, default: false, null: false
t.binary :verification_checksum
t.binary :verification_checksum_mismatched
t.text :verification_failure, limit: 255
t.text :last_sync_failure, limit: 255
t.string :verification_failure, limit: 255 # rubocop:disable Migration/PreventStrings see https://gitlab.com/gitlab-org/gitlab/-/issues/323806
t.string :last_sync_failure, limit: 255 # rubocop:disable Migration/PreventStrings see https://gitlab.com/gitlab-org/gitlab/-/issues/323806
t.index :cool_widget_id, name: :index_cool_widget_registry_on_cool_widget_id, unique: true
t.index :retry_at
t.index :state
# To optimize performance of CoolWidgetRegistry.verification_failed_batch
t.index :verification_retry_at,
name: :cool_widget_registry_failed_verification,
order: "NULLS FIRST",
where: "((state = 2) AND (verification_state = 3))"
t.index :verification_retry_at, name: :cool_widget_registry_failed_verification, order: "NULLS FIRST", where: "((state = 2) AND (verification_state = 3))"
# To optimize performance of CoolWidgetRegistry.needs_verification_count
t.index :verification_state,
name: :cool_widget_registry_needs_verification,
where: "((state = 2) AND (verification_state = ANY (ARRAY[0, 3])))"
t.index :verification_state, name: :cool_widget_registry_needs_verification, where: "((state = 2) AND (verification_state = ANY (ARRAY[0, 3])))"
# To optimize performance of CoolWidgetRegistry.verification_pending_batch
t.index :verified_at,
name: :cool_widget_registry_pending_verification,
order: "NULLS FIRST",
where: "((state = 2) AND (verification_state = 0))"
t.index :verified_at, name: :cool_widget_registry_pending_verification, order: "NULLS FIRST", where: "((state = 2) AND (verification_state = 0))"
end
end
end
end
def down
drop_table :cool_widget_registry
end
end
```
- [ ] If deviating from the above example, then be sure to order columns according to [our guidelines](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/ordering_table_columns.md).
- [ ] Run Geo tracking database migrations:
```shell
bin/rake geo:db:migrate
```
- [ ] Be sure to commit the relevant changes in `ee/db/geo/schema.rb`
### Add verification state fields on the Geo primary site
The Geo primary site needs to checksum every replicable in order for secondaries to verify their own checksums. To do this, Geo requires fields on the Model. There are two ways to add the necessary verification state fields. If the table is large and wide, then it may be a good idea to add verification state fields to a separate table (Option 2). Consult a database expert if needed.
#### Add verification state fields to the model table (Option 1)
- [ ] Create the migration file in `db/migrate`:
```shell
bin/rails generate migration AddVerificationStateToCoolWidgets
```
- [ ] Replace the contents of the migration file with:
```ruby
# frozen_string_literal: true
class AddVerificationStateToCoolWidgets < ActiveRecord::Migration[6.0]
def change
change_table(:cool_widgets) do |t|
t.integer :verification_state, default: 0, limit: 2, null: false
t.column :verification_started_at, :datetime_with_timezone
t.integer :verification_retry_count, limit: 2, null: false
t.column :verification_retry_at, :datetime_with_timezone
t.column :verified_at, :datetime_with_timezone
t.binary :verification_checksum, using: 'verification_checksum::bytea'
t.text :verification_failure # rubocop:disable Migration/AddLimitToTextColumns
end
end
end
```
- [ ] If deviating from the above example, then be sure to order columns according to [our guidelines](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/ordering_table_columns.md).
- [ ] Add the new table to the [database dictionary](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/database/database_dictionary.md) defined in [`ee/db/geo/docs/`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/db/geo/docs):
```yaml
table_name: cool_widget_registry
description: Description example
introduced_by_url: Merge request link
milestone: Milestone example
feature_categories:
- Feature category example
classes:
- Class example
gitlab_schema: gitlab_geo
```
- [ ] Run Geo tracking database migrations:
- [ ] If `cool_widgets` is a high-traffic table, follow [the database documentation to use `with_lock_retries`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/migration_style_guide.md#when-to-use-the-helper-method)
- [ ] Adding a `text` column also [requires](../database/strings_and_the_text_data_type.md#add-a-text-column-to-an-existing-table) setting a limit. Create the migration file in `db/migrate`:
```shell
bin/rake db:migrate:geo
bin/rails generate migration AddVerificationFailureLimitToCoolWidgets
```
- [ ] Be sure to commit the relevant changes in `ee/db/geo/structure.sql` and the file created under `ee/db/geo/schema_migrations`
- [ ] Replace the contents of the migration file with:
### Add verification state fields on the Geo primary site
```ruby
# frozen_string_literal: true
The Geo primary site needs to checksum every replicable so secondaries can verify their own checksums. To do this, Geo requires fields on the Model. Add verification state fields to a separate table. Consult a database expert if needed.
class AddVerificationFailureLimitToCoolWidgets < ActiveRecord::Migration[6.0]
include Gitlab::Database::MigrationHelpers
#### Add verification state fields to a new table
disable_ddl_transaction!
CONSTRAINT_NAME = 'cool_widget_verification_failure_text_limit'
def up
add_text_limit :cool_widget, :verification_failure, 255, constraint_name: CONSTRAINT_NAME
end
def down
remove_check_constraint(:cool_widget, CONSTRAINT_NAME)
end
end
```
- [ ] Add indexes on verification fields to ensure verification can be performed efficiently. Some or all of these indexes can be omitted if the table is guaranteed to be small. Ask a database expert if you are considering omitting indexes. Create the migration file in `db/migrate`:
```shell
bin/rails generate migration AddVerificationIndexesToCoolWidgets
```
- [ ] Replace the contents of the migration file with:
```ruby
# frozen_string_literal: true
class AddVerificationIndexesToCoolWidgets < ActiveRecord::Migration[6.0]
include Gitlab::Database::MigrationHelpers
VERIFICATION_STATE_INDEX_NAME = "index_cool_widgets_on_verification_state"
PENDING_VERIFICATION_INDEX_NAME = "index_cool_widgets_pending_verification"
FAILED_VERIFICATION_INDEX_NAME = "index_cool_widgets_failed_verification"
NEEDS_VERIFICATION_INDEX_NAME = "index_cool_widgets_needs_verification"
disable_ddl_transaction!
def up
add_concurrent_index :cool_widgets, :verification_state, name: VERIFICATION_STATE_INDEX_NAME
add_concurrent_index :cool_widgets, :verified_at, where: "(verification_state = 0)", order: { verified_at: 'ASC NULLS FIRST' }, name: PENDING_VERIFICATION_INDEX_NAME
add_concurrent_index :cool_widgets, :verification_retry_at, where: "(verification_state = 3)", order: { verification_retry_at: 'ASC NULLS FIRST' }, name: FAILED_VERIFICATION_INDEX_NAME
add_concurrent_index :cool_widgets, :verification_state, where: "(verification_state = 0 OR verification_state = 3)", name: NEEDS_VERIFICATION_INDEX_NAME
end
def down
remove_concurrent_index_by_name :cool_widgets, VERIFICATION_STATE_INDEX_NAME
remove_concurrent_index_by_name :cool_widgets, PENDING_VERIFICATION_INDEX_NAME
remove_concurrent_index_by_name :cool_widgets, FAILED_VERIFICATION_INDEX_NAME
remove_concurrent_index_by_name :cool_widgets, NEEDS_VERIFICATION_INDEX_NAME
end
end
```
- [ ] Run database migrations:
```shell
bin/rake db:migrate
```
- [ ] Be sure to commit the relevant changes in `db/structure.sql`
#### Add verification state fields to a separate table (Option 2)
- [ ] Create the migration file in `db/migrate`:
@ -139,43 +229,39 @@ The Geo primary site needs to checksum every replicable so secondaries can verif
```ruby
# frozen_string_literal: true
class CreateCoolWidgetStates < Gitlab::Database::Migration[2.1]
class CreateCoolWidgetStates < ActiveRecord::Migration[6.0]
include Gitlab::Database::MigrationHelpers
VERIFICATION_STATE_INDEX_NAME = "index_cool_widget_states_on_verification_state"
PENDING_VERIFICATION_INDEX_NAME = "index_cool_widget_states_pending_verification"
FAILED_VERIFICATION_INDEX_NAME = "index_cool_widget_states_failed_verification"
NEEDS_VERIFICATION_INDEX_NAME = "index_cool_widget_states_needs_verification"
enable_lock_retries!
disable_ddl_transaction!
def up
unless table_exists?(:cool_widget_states)
with_lock_retries do
create_table :cool_widget_states, id: false do |t|
t.datetime_with_timezone :verification_started_at
t.references :cool_widget, primary_key: true, null: false, foreign_key: { on_delete: :cascade }
t.integer :verification_state, default: 0, limit: 2, null: false
t.column :verification_started_at, :datetime_with_timezone
t.datetime_with_timezone :verification_retry_at
t.datetime_with_timezone :verified_at
t.references :cool_widget,
primary_key: true,
default: nil,
index: false,
foreign_key: { on_delete: :cascade }
t.integer :verification_state, default: 0, limit: 2, null: false
t.integer :verification_retry_count, default: 0, limit: 2, null: false
t.integer :verification_retry_count, limit: 2
t.binary :verification_checksum, using: 'verification_checksum::bytea'
t.text :verification_failure, limit: 255
t.text :verification_failure
t.index :verification_state, name: VERIFICATION_STATE_INDEX_NAME
t.index :verified_at,
where: "(verification_state = 0)",
order: { verified_at: 'ASC NULLS FIRST' },
name: PENDING_VERIFICATION_INDEX_NAME
t.index :verification_retry_at,
where: "(verification_state = 3)",
order: { verification_retry_at: 'ASC NULLS FIRST' },
name: FAILED_VERIFICATION_INDEX_NAME
t.index :verification_state,
where: "(verification_state = 0 OR verification_state = 3)",
name: NEEDS_VERIFICATION_INDEX_NAME
t.index :verified_at, where: "(verification_state = 0)", order: { verified_at: 'ASC NULLS FIRST' }, name: PENDING_VERIFICATION_INDEX_NAME
t.index :verification_retry_at, where: "(verification_state = 3)", order: { verification_retry_at: 'ASC NULLS FIRST' }, name: FAILED_VERIFICATION_INDEX_NAME
t.index :verification_state, where: "(verification_state = 0 OR verification_state = 3)", name: NEEDS_VERIFICATION_INDEX_NAME
end
end
end
add_text_limit :cool_widget_states, :verification_failure, 255
end
def down
drop_table :cool_widget_states
@ -184,31 +270,13 @@ The Geo primary site needs to checksum every replicable so secondaries can verif
```
- [ ] If deviating from the above example, then be sure to order columns according to [our guidelines](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/ordering_table_columns.md).
- [ ] If `cool_widgets` is a high-traffic table, follow [the database documentation to use `with_lock_retries`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/migration_style_guide.md#when-to-use-the-helper-method)
- [ ] Add the new table to the [database dictionary](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/database/database_dictionary.md) defined in [`db/docs/`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/db/docs):
```yaml
---
table_name: cool_widget_states
description: Separate table for cool widget verification states
introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/XXXXX
milestone: 'XX.Y'
feature_categories:
- geo_replication
classes:
- Geo::CoolWidgetState
gitlab_schema: gitlab_main
```
- [ ] Run database migrations:
```shell
bin/rake db:migrate
```
- [ ] Be sure to commit the relevant changes in `db/structure.sql` and the file under `db/schema_migrations`
- [ ] Be sure to commit the relevant changes in `db/structure.sql`
That's all of the required database changes.
@ -216,14 +284,7 @@ That's all of the required database changes.
#### Step 1. Implement replication and verification
- [ ] Add the following lines to the `cool_widget` model to accomplish some important tasks:
- Include `::Geo::ReplicableModel` in the `CoolWidget` class, and specify the Replicator class `with_replicator Geo::CoolWidgetReplicator`.
- Include the `::Geo::VerifiableModel` concern.
- Delegate verification related methods to the `cool_widget_state` model.
- For verification, override some scopes to use the `cool_widget_states` table instead of the model table.
- Implement the `verification_state_object` method to return the object that holds
the verification details
- Override some methods to use the `cool_widget_states` table in verification-related queries.
- [ ] Include `Gitlab::Geo::ReplicableModel` in the `CoolWidget` class, and specify the Replicator class `with_replicator Geo::CoolWidgetReplicator`.
At this point the `CoolWidget` class should look like this:
@ -231,61 +292,28 @@ That's all of the required database changes.
# frozen_string_literal: true
class CoolWidget < ApplicationRecord
...
include ::Geo::ReplicableModel
include ::Geo::VerifiableModel
delegate(*::Geo::VerificationState::VERIFICATION_METHODS, to: :cool_widget_state)
include ::Gitlab::Geo::ReplicableModel
include ::Gitlab::Geo::VerificationState
with_replicator Geo::CoolWidgetReplicator
mount_uploader :file, CoolWidgetUploader
has_one :cool_widget_state, autosave: false, inverse_of: :cool_widget, class_name: 'Geo::CoolWidgetState'
after_save :save_verification_details
# Override the `all` default if not all records can be replicated. For an
# example of an existing Model that needs to do this, see
# `EE::MergeRequestDiff`.
# scope :available_replicables, -> { all }
scope :available_verifiables, -> { joins(:cool_widget_state) }
scope :checksummed, -> {
joins(:cool_widget_state).where.not(cool_widget_states: { verification_checksum: nil })
}
scope :not_checksummed, -> {
joins(:cool_widget_state).where(cool_widget_states: { verification_checksum: nil })
}
scope :with_verification_state, ->(state) {
joins(:cool_widget_state)
.where(cool_widget_states: { verification_state: verification_state_value(state) })
}
def verification_state_object
cool_widget_state
end
...
class_methods do
extend ::Gitlab::Utils::Override
...
# @param primary_key_in [Range, CoolWidget] arg to pass to primary_key_in scope
# @return [ActiveRecord::Relation<CoolWidget>] everything that should be synced
# to this node, restricted by primary key
def replicables_for_current_secondary(primary_key_in)
# @return [ActiveRecord::Relation<CoolWidget>] everything that should be synced to this node, restricted by primary key
def self.replicables_for_current_secondary(primary_key_in)
# This issue template does not help you write this method.
#
# This method is called only on Geo secondary sites. It is called when
# we want to know which records to replicate. This is not easy to automate
# because for example:
#
# * The "selective sync" feature allows admins to choose which namespaces
# to replicate, per secondary site. Most Models are scoped to a
# * The "selective sync" feature allows admins to choose which namespaces # to replicate, per secondary site. Most Models are scoped to a
# namespace, but the nature of the relationship to a namespace varies
# between Models.
# * The "selective sync" feature allows admins to choose which shards to
@ -297,29 +325,42 @@ That's all of the required database changes.
#
# Search the codebase for examples, and consult a Geo expert if needed.
end
override :verification_state_table_class
def verification_state_table_class
CoolWidgetState
end
end
def cool_widget_state
super || build_cool_widget_state
end
...
end
```
- [ ] Implement `CoolWidget.replicables_for_current_secondary` above.
- [ ] Ensure `CoolWidget.replicables_for_current_secondary` is well-tested. Search the codebase for `replicables_for_current_secondary` to find examples of parameterized table specs. You may need to add more `FactoryBot` traits.
- [ ] Add the following shared examples to `ee/spec/models/ee/cool_widget_spec.rb`:
- [ ] If you are using a separate table `cool_widget_states` to track verification state on the Geo primary site, then:
- [ ] Do not include `::Gitlab::Geo::VerificationState` on the `CoolWidget` class.
- [ ] Add the following lines to the `cool_widget_state.rb` model:
```ruby
include_examples 'a replicable model with a separate table for verification state' do
let(:verifiable_model_record) { build(:cool_widget) } # add extra params if needed to make sure the record is in `Geo::ReplicableModel.verifiables` scope
let(:unverifiable_model_record) { build(:cool_widget) } # add extra params if needed to make sure the record is NOT included in `Geo::ReplicableModel.verifiables` scope
class CoolWidgetState < ApplicationRecord
...
self.primary_key = :cool_widget_id
include ::Gitlab::Geo::VerificationState
belongs_to :cool_widget, inverse_of: :cool_widget_state
...
end
```
- [ ] Add the following lines to the `cool_widget` model:
```ruby
class CoolWidget < ApplicationRecord
...
has_one :cool_widget_state, inverse_of: :cool_widget
delegate :verification_retry_at, :verification_retry_at=,
:verified_at, :verified_at=,
:verification_checksum, :verification_checksum=,
:verification_failure, :verification_failure=,
:verification_retry_count, :verification_retry_count=,
to: :cool_widget_state
...
end
```
@ -331,7 +372,6 @@ That's all of the required database changes.
module Geo
class CoolWidgetReplicator < Gitlab::Geo::Replicator
include ::Geo::BlobReplicatorStrategy
extend ::Gitlab::Utils::Override
def self.model
::CoolWidget
@ -341,6 +381,12 @@ That's all of the required database changes.
model_record.file
end
# The feature flag follows the format `geo_#{replicable_name}_replication`,
# so here it would be `geo_cool_widget_replication`
def self.replication_enabled_by_default?
false
end
override :verification_feature_flag_enabled?
def self.verification_feature_flag_enabled?
# We are adding verification at the same time as replication, so we
@ -349,11 +395,12 @@ That's all of the required database changes.
# (see `VerifiableReplicator.verification_enabled?`)
true
end
end
end
```
- [ ] Generate the feature flag definition file by running the feature flag commands and following the command prompts:
- [ ] Generate the feature flag definition file by running the feature flag command and following the command prompts:
```shell
bin/feature-flag --ee geo_cool_widget_replication --type development --group 'group::geo'
@ -367,6 +414,7 @@ That's all of the required database changes.
::Geo::PackageFileReplicator,
::Geo::CoolWidgetReplicator
]
end
```
- [ ] Create `ee/spec/replicators/geo/cool_widget_replicator_spec.rb` and perform the necessary setup to define the `model_record` variable for the shared examples:
@ -376,7 +424,7 @@ That's all of the required database changes.
require 'spec_helper'
RSpec.describe Geo::CoolWidgetReplicator, feature_category: :geo_replication do
RSpec.describe Geo::CoolWidgetReplicator do
let(:model_record) { build(:cool_widget) }
include_examples 'a blob replicator'
@ -389,8 +437,7 @@ That's all of the required database changes.
```ruby
# frozen_string_literal: true
module Geo
class CoolWidgetRegistry < Geo::BaseRegistry
class Geo::CoolWidgetRegistry < Geo::BaseRegistry
include ::Geo::ReplicableRegistry
include ::Geo::VerifiableRegistry
@ -399,11 +446,10 @@ That's all of the required database changes.
belongs_to :cool_widget, class_name: 'CoolWidget'
end
end
```
- [ ] Update `REGISTRY_CLASSES` in `ee/app/workers/geo/secondary/registry_consistency_worker.rb`.
- [ ] Add a custom factory name if needed in `def model_class_factory_name` in `ee/spec/support/helpers/ee/geo_helpers.rb`.
- [ ] Update `def model_class_factory_name` in `ee/spec/services/geo/registry_consistency_service_spec.rb`.
- [ ] Update `it 'creates missing registries for each registry class'` in `ee/spec/workers/geo/secondary/registry_consistency_worker_spec.rb`.
- [ ] Add `cool_widget_registry` to `ActiveSupport::Inflector.inflections` in `config/initializers_before_autoloader/000_inflections.rb`.
- [ ] Create `ee/spec/factories/geo/cool_widget_registry.rb`:
@ -413,7 +459,7 @@ That's all of the required database changes.
FactoryBot.define do
factory :geo_cool_widget_registry, class: 'Geo::CoolWidgetRegistry' do
cool_widget # This association should have data, like a file or repository
cool_widget
state { Geo::CoolWidgetRegistry.state_value(:pending) }
trait :synced do
@ -425,7 +471,6 @@ That's all of the required database changes.
state { Geo::CoolWidgetRegistry.state_value(:failed) }
last_synced_at { 1.day.ago }
retry_count { 2 }
retry_at { 2.hours.from_now }
last_sync_failure { 'Random error' }
end
@ -434,12 +479,6 @@ That's all of the required database changes.
last_synced_at { 1.day.ago }
retry_count { 0 }
end
trait :verification_succeeded do
verification_checksum { 'e079a831cab27bcda7d81cd9b48296d0c3dd92ef' }
verification_state { Geo::CoolWidgetRegistry.verification_state_value(:verification_succeeded) }
verified_at { 5.days.ago }
end
end
end
```
@ -451,7 +490,7 @@ That's all of the required database changes.
require 'spec_helper'
RSpec.describe Geo::CoolWidgetRegistry, :geo, type: :model, feature_category: :geo_replication do
RSpec.describe Geo::CoolWidgetRegistry, :geo, type: :model do
let_it_be(:registry) { create(:geo_cool_widget_registry) }
specify 'factory is valid' do
@ -463,80 +502,6 @@ That's all of the required database changes.
end
```
- [ ] Add the following to `spec/factories/cool_widgets.rb`:
```ruby
# frozen_string_literal: true
FactoryBot.modify do
factory :cool_widget do
trait :verification_succeeded do
with_file
verification_checksum { 'abc' }
verification_state { CoolWidget.verification_state_value(:verification_succeeded) }
end
trait :verification_failed do
with_file
verification_failure { 'Could not calculate the checksum' }
verification_state { CoolWidget.verification_state_value(:verification_failed) }
end
end
end
```
If there is not an existing factory for the object in `spec/factories/cool_widgets.rb`, wrap the traits in `FactoryBot.create` instead of `FactoryBot.modify`
[ ] Make sure the factory supports the `:remote_store` trait. If not, add something like
```ruby
trait :remote_store do
file_store { CoolWidget::FileUploader::Store::REMOTE }
end
```
- [ ] Make sure the factory also allows setting a `project` attribute. If the model does not have a direct relation to a project, you can use a `transient` attribute. Check out `spec/factories/merge_request_diffs.rb` for an example.
- [ ] Following [the example of Merge Request Diffs](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63309) add a `Geo::CoolWidgetState` model in `ee/app/models/geo/cool_widget_state.rb`:
``` ruby
# frozen_string_literal: true
module Geo
class CoolWidgetState < ApplicationRecord
include ::Geo::VerificationStateDefinition
self.primary_key = :cool_widget_id
belongs_to :cool_widget, inverse_of: :cool_widget_state
validates :verification_failure, length: { maximum: 255 }
validates :verification_state, :cool_widget, presence: true
end
end
```
- [ ] Add a `factory` for `cool_widget_state`, in `ee/spec/factories/geo/cool_widget_states.rb`:
``` ruby
# frozen_string_literal: true
FactoryBot.define do
factory :geo_cool_widget_state, class: 'Geo::CoolWidgetState' do
cool_widget
trait :checksummed do
verification_checksum { 'abc' }
end
trait :checksum_failure do
verification_failure { 'Could not calculate the checksum' }
end
end
end
```
- [ ] Add `[:cool_widget, :remote_store]` and `[:geo_cool_widget_state, any]` to `skipped` in `spec/models/factories_spec.rb`
#### Step 2. Implement metrics gathering
Metrics are gathered by `Geo::MetricsUpdateWorker`, persisted in `GeoNodeStatus` for display in the UI, and sent to Prometheus:
@ -557,22 +522,41 @@ Metrics are gathered by `Geo::MetricsUpdateWorker`, persisted in `GeoNodeStatus`
- [ ] Add the same fields to `GET /geo_nodes/status` example response in
`ee/spec/fixtures/api/schemas/public_api/v4/geo_node_status.json`.
- [ ] Add the following fields to the `Sidekiq metrics` table in `doc/administration/monitoring/prometheus/gitlab_metrics.md`:
- `geo_cool_widgets`
- `geo_cool_widgets_checksum_total`
- `geo_cool_widgets_checksummed`
- `geo_cool_widgets_checksum_failed`
- `geo_cool_widgets_synced`
- `geo_cool_widgets_failed`
- `geo_cool_widgets_registry`
- `geo_cool_widgets_verification_total`
- `geo_cool_widgets_verified`
- `geo_cool_widgets_verification_failed`
- [ ] Add the following to the parameterized table in the `context 'Replicator stats' do` block in `ee/spec/models/geo_node_status_spec.rb`:
```markdown
| `geo_cool_widgets` | Gauge | XX.Y | Number of Cool Widgets on primary | `url` |
| `geo_cool_widgets_checksum_total` | Gauge | XX.Y | Number of Cool Widgets to checksum on primary | `url` |
| `geo_cool_widgets_checksummed` | Gauge | XX.Y | Number of Cool Widgets that successfully calculated the checksum on primary | `url` |
| `geo_cool_widgets_checksum_failed` | Gauge | XX.Y | Number of Cool Widgets that failed to calculate the checksum on primary | `url` |
| `geo_cool_widgets_synced` | Gauge | XX.Y | Number of syncable Cool Widgets synced on secondary | `url` |
| `geo_cool_widgets_failed` | Gauge | XX.Y | Number of syncable Cool Widgets failed to sync on secondary | `url` |
| `geo_cool_widgets_registry` | Gauge | XX.Y | Number of Cool Widgets in the registry | `url` |
| `geo_cool_widgets_verification_total` | Gauge | XX.Y | Number of Cool Widgets to attempt to verify on secondary | `url` |
| `geo_cool_widgets_verified` | Gauge | XX.Y | Number of Cool Widgets successfully verified on secondary | `url` |
| `geo_cool_widgets_verification_failed` | Gauge | XX.Y | Number of Cool Widgets that failed verification on secondary | `url` |
```ruby
Geo::CoolWidgetReplicator | :cool_widget | :geo_cool_widget_registry
```
- [ ] Run the rake task `geo:dev:ssf_metrics` and commit the changes to `ee/config/metrics/object_schemas/geo_node_usage.json`
Cool Widget replication and verification metrics should now be available in the API, the `Admin > Geo > Sites` view, and Prometheus.
- [ ] Add the following to `spec/factories/cool_widgets.rb`:
```ruby
trait(:verification_succeeded) do
with_file
verification_checksum { 'abc' }
verification_state { CoolWidget.verification_state_value(:verification_succeeded) }
end
trait(:verification_failed) do
with_file
verification_failure { 'Could not calculate the checksum' }
verification_state { CoolWidget.verification_state_value(:verification_failed) }
end
```
- [ ] Make sure the factory also allows setting a `project` attribute. If the model does not have a direct relation to a project, you can use a `transient` attribute. Check out `spec/factories/merge_request_diffs.rb` for an example.
Cool Widget replication and verification metrics should now be available in the API, the `Admin > Geo > Nodes` view, and Prometheus.
#### Step 3. Implement the GraphQL API
@ -584,9 +568,8 @@ The GraphQL API is used by `Admin > Geo > Replication Details` views, and is dir
field :cool_widget_registries, ::Types::Geo::CoolWidgetRegistryType.connection_type,
null: true,
resolver: ::Resolvers::Geo::CoolWidgetRegistriesResolver,
description: 'Find Cool Widget registries on this Geo node. '\
'Ignored if `geo_cool_widget_replication` feature flag is disabled.',
alpha: { milestone: '15.5' } # Update the milestone
description: 'Find Cool Widget registries on this Geo node',
feature_flag: :geo_cool_widget_replication
```
- [ ] Add the new `cool_widget_registries` field name to the `expected_fields` array in `ee/spec/graphql/types/geo/geo_node_type_spec.rb`.
@ -613,7 +596,7 @@ The GraphQL API is used by `Admin > Geo > Replication Details` views, and is dir
require 'spec_helper'
RSpec.describe Resolvers::Geo::CoolWidgetRegistriesResolver, feature_category: :geo_replication do
RSpec.describe Resolvers::Geo::CoolWidgetRegistriesResolver do
it_behaves_like 'a Geo registries resolver', :geo_cool_widget_registry
end
```
@ -637,7 +620,7 @@ The GraphQL API is used by `Admin > Geo > Replication Details` views, and is dir
require 'spec_helper'
RSpec.describe Geo::CoolWidgetRegistryFinder, feature_category: :geo_replication do
RSpec.describe Geo::CoolWidgetRegistryFinder do
it_behaves_like 'a framework registry finder', :geo_cool_widget_registry
end
```
@ -651,15 +634,13 @@ The GraphQL API is used by `Admin > Geo > Replication Details` views, and is dir
module Geo
# rubocop:disable Graphql/AuthorizeTypes because it is included
class CoolWidgetRegistryType < BaseObject
graphql_name 'CoolWidgetRegistry'
include ::Types::Geo::RegistryType
graphql_name 'CoolWidgetRegistry'
description 'Represents the Geo replication and verification state of a cool_widget'
field :cool_widget_id, GraphQL::Types::ID, null: false, description: 'ID of the Cool Widget.'
field :cool_widget_id, GraphQL::ID_TYPE, null: false, description: 'ID of the Cool Widget'
end
# rubocop:enable Graphql/AuthorizeTypes
end
end
```
@ -671,7 +652,7 @@ The GraphQL API is used by `Admin > Geo > Replication Details` views, and is dir
require 'spec_helper'
RSpec.describe GitlabSchema.types['CoolWidgetRegistry'], feature_category: :geo_replication do
RSpec.describe GitlabSchema.types['CoolWidgetRegistry'] do
it_behaves_like 'a Geo registry type'
it 'has the expected fields (other than those included in RegistryType)' do
@ -701,49 +682,6 @@ The GraphQL API is used by `Admin > Geo > Replication Details` views, and is dir
Individual Cool Widget replication and verification data should now be available via the GraphQL API.
#### Step 4. Handle batch destroy
If batch destroy logic is implemented for a replicable, then that logic must be "replicated" by Geo secondaries. The easiest way to do this is use `Geo::BatchEventCreateWorker` to bulk insert a delete event for each replicable.
For example, if `FastDestroyAll` is used, then you may be able to [use `begin_fast_destroy` and `finalize_fast_destroy` hooks, like we did for uploads](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/69763).
Or if a special service is used to batch delete records and their associated data, then you probably need to [hook into that service, like we did for job artifacts](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/79530).
As illustrated by the above two examples, batch destroy logic cannot be handled automatically by Geo secondaries without restricting the way other teams perform batch destroys. It is up to you to produce `Geo::BatchEventCreateWorker` attributes before the records are deleted, and then enqueue `Geo::BatchEventCreateWorker` after the records are deleted.
- [ ] Ensure that any batch destroy of this replicable is replicated to secondary sites
- [ ] Regardless of implementation details, please verify in specs that when the parent object is removed, the new `Geo::Event` records are created:
```ruby
describe '#destroy' do
subject { create(:cool_widget) }
context 'when running in a Geo primary node' do
let_it_be(:primary) { create(:geo_node, :primary) }
let_it_be(:secondary) { create(:geo_node) }
it 'logs an event to the Geo event log when bulk removal is used', :sidekiq_inline do
stub_current_geo_node(primary)
expect { subject.project.destroy! }.to change(Geo::Event.where(replicable_name: :cool_widget, event_name: :deleted), :count).by(1)
payload = Geo::Event.where(replicable_name: :cool_widget, event_name: :deleted).last.payload
expect(payload['model_record_id']).to eq(subject.id)
expect(payload['blob_path']).to eq(subject.relative_path)
expect(payload['uploader_class']).to eq('CoolWidgetUploader')
end
end
end
```
### Code Review
When requesting review from database reviewers:
- [ ] Include a comment mentioning that the change is based on a documented template.
- [ ] `replicables_for_current_secondary` and `available_replicables` may differ per Model. If their queries are new, then add [query plans](https://docs.gitlab.com/ee/development/database_review.html#query-plans) to the MR description. An easy place to gather SQL queries is your GDK's `log/test.log` when running tests of these methods.
### Release Geo support of Cool Widgets
- [ ] In the rollout issue you created when creating the feature flag, modify the Roll Out Steps:
@ -751,18 +689,34 @@ When requesting review from database reviewers:
- [ ] Add a step to `Test replication and verification of Cool Widgets on a non-GDK-deployment. For example, using GitLab Environment Toolkit`.
- [ ] Add a step to `Ping the Geo PM and EM to coordinate testing`. For example, you might add steps to generate Cool Widgets, and then a Geo engineer may take it from there.
- [ ] In `ee/config/feature_flags/development/geo_cool_widget_replication.yml`, set `default_enabled: true`
- [ ] In `ee/app/graphql/types/geo/geo_node_type.rb`, remove the `alpha` option for the released type:
- [ ] In `ee/app/replicators/geo/cool_widget_replicator.rb`, delete the `self.replication_enabled_by_default?` method:
```ruby
module Geo
class CoolWidgetReplicator < Gitlab::Geo::Replicator
...
# REMOVE THIS METHOD
def self.replication_enabled_by_default?
false
end
# REMOVE THIS METHOD
...
end
end
```
- [ ] In `ee/app/graphql/types/geo/geo_node_type.rb`, remove the `feature_flag` option for the released type:
```ruby
field :cool_widget_registries, ::Types::Geo::CoolWidgetRegistryType.connection_type,
null: true,
resolver: ::Resolvers::Geo::CoolWidgetRegistriesResolver,
description: 'Find Cool Widget registries on this Geo node. '\
'Ignored if `geo_cool_widget_replication` feature flag is disabled.',
alpha: { milestone: '15.5' } # Update the milestone
description: 'Find Cool Widget registries on this Geo node',
feature_flag: :geo_cool_widget_replication # REMOVE THIS LINE
```
- [ ] Run `bundle exec rake gitlab:graphql:compile_docs` after the step above to regenerate the GraphQL docs.
- [ ] Add a row for Cool Widgets to the `Data types` table in [Geo data types support](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/administration/geo/replication/datatypes.md#data-types)
- [ ] Add a row for Cool Widgets to the `Limitations on replication/verification` table in [Geo data types support](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/administration/geo/replication/datatypes.md#limitations-on-replicationverification). If the row already exists, then update it to show that Replication and Verification is released in the current version.

View file

@ -1,30 +0,0 @@
## Summary
<!-- Summarize the bug encountered concisely. -->
## Steps to reproduce
<!-- Describe how one can reproduce the issue - this is very important. Please use an ordered list. -->
## What is the current *bug* behavior?
<!-- Describe what actually happens. -->
## What is the expected *correct* behavior?
<!-- Describe what you should see instead. -->
## Relevant logs and/or screenshots
<!-- Paste any relevant logs - please use code blocks (```) to format console output, logs, and code
as it's tough to read otherwise. -->
## Possible fixes
<!-- If you can, link to the line of code that might be responsible for the problem. -->
<!-- Please add a label for the type of bug as per https://about.gitlab.com/handbook/engineering/metrics/#work-type-classification -->
/label ~"type::bug"
/label ~"group::global search"
/label ~"workflow::solution validation"
/milestone %Backlog

View file

@ -1,13 +0,0 @@
## Problem to solve
<!-- What problem do we solve? Try to define the who/what/why of the opportunity as a user story. For example, "As a (who), I want (what), so I can (why/value)." -->
## Proposal
<!-- Use this section to explain the feature and how it will work. It can be helpful to add technical details, design proposals, and links to related epics or issues. -->
<!-- Please add a label for the type of feature as per https://about.gitlab.com/handbook/engineering/metrics/#work-type-classification -->
/label ~"type::feature"
/label ~"group::global search"
/label ~"workflow::solution validation"
/milestone %Backlog

View file

@ -1,11 +0,0 @@
## Background
## Proposal
<!-- Use this section to explain the feature and how it will work. It can be helpful to add technical details, design proposals, and links to related epics or issues. -->
<!-- Please add a label for the type of maintenance as per https://about.gitlab.com/handbook/engineering/metrics/#work-type-classification -->
/label ~"type::maintenance"
/label ~"group::global search"
/label ~"workflow::solution validation"
/milestone %Backlog

View file

@ -37,20 +37,13 @@ Add details for required items and delete others.
## Implementation plan
<!--
Steps and the parts of the code that will need to get updated.
The plan can also call-out responsibilities for other team members or teams and
can be split into smaller MRs to simplify the code review process.
Steps and the parts of the code that will need to get updated. The plan can also
call-out responsibilities for other team members or teams.
e.g.:
- MR 1: Part 1
- [ ] ~frontend Step 1
- [ ] ~frontend Step 2
- MR 2: Part 2
- [ ] ~backend Step 1
- [ ] ~backend Step 2
- MR 3: Part 3
- [ ] ~frontend Step 1
- [ ] `@person` Step 1a
- [ ] ~frontend Step 2
-->
@ -66,17 +59,5 @@ Other settings you might want to include when creating the issue.
# /epic &
-->
## Verification steps
<!--
Add verification steps to help GitLab team members test the implementation. This is particularly useful
during the MR review and the ~"workflow::verification" step. You may not know exactly what the
verification steps should be during issue refinement, so you can always come back later to add
them.
1. Check-out the corresponding branch
1. ...
1. Profit!
-->
/label ~"workflow::refinement"
/milestone %Backlog

View file

@ -1,56 +0,0 @@
<!--
Triage of infradev Issues is desired to occur asynchronously.
For maximum efficiency, please ensure the following, so that your infradev issues can gain maximum traction.
https://about.gitlab.com/handbook/engineering/workflow/#a-guide-to-creating-effective-infradev-issues
-->
## Summary
<!--
Clearly state the scope of the problem, and how it affects GitLab.com
-->
## Impact
<!--
- Quantify the effect of the problem to help ensure that correct prioritization occurs.
- Include costs to availability. The Incident Budget Explorer dashboard can help here.
- Include the number of times alerts have fired owing to the problem, how much time was spent dealing with the problem, and how many people were involved.
- Link to affected incidents, and cross-reference them as related issues.
- Include screenshots of visualization from Grafana or Kibana.
- Always include a permalink to the source of the screenshot so that others can investigate further.
-->
## Recommendation
<!--
Provide a clear, unambiguous, self-contained solution to the problem.
-->
## Verification
<!--
Provide a method for validating that the original issue still exists.
Having a way of checking validity can save on a great deal of back-and-forth discussion between Infradev Triage participants including Engineering Managers, Directors and Product Managers and make space for other non-resolved issues to get scheduled sooner.
Ideally, provide a link to a Thanos query or an ELK query and clear instructions on how to interpret the results to determine whether the problem is still occurring.
-->
<!--
Workflow and other relevant labels
/label ~"severity::"
/label ~"priority::"
/label ~"group::"
/label ~"devops::"
See also:
- https://about.gitlab.com/handbook/engineering/quality/issue-triage/#availability
- https://about.gitlab.com/handbook/product/categories/
- https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml
-->
/label ~"infradev"
/label ~"type::bug"

View file

@ -1,26 +0,0 @@
<!-- This template is used for proposing changes to the left sidebar contextual navigation. This could include additions, removals, or general changes to overall hierarchy.-->
### Proposal
<!-- Use this section to explain the proposed changes, including details around usage and business drivers. -->
#### Other locations that were considered
<!-- Include other design patterns or places you considered for this feature besides navigation. -->
### Checklist
- [ ] Review the handbook page for [navigation changes](https://about.gitlab.com/handbook/product/ux/navigation/#when-to-consider-making-a-change-to-the-navigation)
- [ ] Add relevant information to the issue description detailing your proposal, including usage and business drivers.
- [ ] List at least two other places you considered to introduce your feature
- [ ] Add relevant designs to the Design Management area of the issue
- [ ] Ensure your UI suggestion align with the [Documentation Style Guide](https://docs.gitlab.com/ee/development/documentation/styleguide/)
- [ ] Engage ~"Technical Writing". They can help craft a term that best describes the feature(s) youre proposing.
- [ ] Follow the [product development workflow](https://about.gitlab.com/handbook/product-development-flow/#validation-phase-2-problem-validation) validation process to ensure you are solving a well understood problem and that the proposed change is understandable and non-disruptive to users. Navigation-specific research is mandatory for additions or when restructuring.
- [ ] Engage the [Foundations Product Manager](https://about.gitlab.com/handbook/product/categories/#foundations-group) for approval. The Foundations DRI (@cdybenko) will work with UX partners in product design, research, and technical writing, as applicable.
- [ ] Consider whether you need to [communicate the change somehow](https://design.gitlab.com/patterns/navigation#messaging-changes-to-users), or if you will have an interim period in the UI where your item will live in more than one place.
- [ ] Ensure engineers are familiar with the [implementation steps for navigation](https://docs.gitlab.com/ee/development/navigation_sidebar.html#navigation-sidebar).
/label ~UX ~"UI text" ~"documentation" ~"Category:Navigation & Settings" ~navigation ~type::ignore
/label ~"Nav request::Start"

View file

@ -0,0 +1,68 @@
<!-- Please title your issue with the following format: "Project Name | Issue Tracker". -->
## Background
<!--
Please add information here about why your project is considering a migration to GitLab, or why it decided to do so. Include any initial announcements that have been / were made about the decision or status.
-->
### Goals
<!-- What are some of the goals of your migration to GitLab? Delete this section if you don't want to enumerate goals. -->
## Quick Facts
<!-- Please complete as many items in this list as possible. If you're not sure yet, add "TBD" (To be Decided) or "Unknown" -->
* **Timeline.** -
* **Product.** - SaaS-Ultimate/Self-Managed-Ultimate or Community Edition
* **Project's License.** What kind of OSI-approved license does your project use?
## Current Tooling and Replacements
<!--
Please fill in the table to give an overview of your current tooling. Here's a description of what to include in each column:
- Tool: which tool or platform you are currently using
- Feature: which particular feature you are using in that tool or platform
- GitLab feature: equivalent GitLab feature (the GitLab team can help fill this in, as well as the info in the next column)
- GitLab edition: in which GitLab edition (CE or EE) is this feature available?
Here's an example of a replacements overview from one of the projects which migrated to GitLab: https://gitlab.com/gitlab-org/gitlab/-/issues/25657#gitlab-replacements
Consider deleting the table below if you are unable to expand upon your current tooling.
-->
| Tool | Feature | GitLab feature | GitLab edition |
| --- | --- | --- | --- |
| | | | |
## Collaborators
<!-- Please add names of collaborators in the format: Name, Title, Role (what will you be helping to do, or how should you be involved), GitLab username -->
## Related Issues
<!-- Add any related issues that are important for your project by adding the title of the issue and a link to it (preferably as an embedded link). You will probably keep editing this section as the migration progresses, so don't worry if it's mostly blank for now.
Here is an example of what this list might look like once populated: https://gitlab.com/gitlab-org/gitlab-foss/-/issues/55039#outstanding-issues
-->
### Blockers
* [ ] ADD_LINK_TO_ISSUE_HERE
### Urgent
* [ ]
### Important but not urgent
* [ ]
### Nice to have
* [ ]
------
/label ~"Open Source Partners"
/cc @nuritzi @greg

View file

@ -1,54 +0,0 @@
<!--
## Implementation Issue To-Do list
(_NOTE: This section can be removed when the issue is ready for creation_)
- [ ] Ensure that issue title is concise yet descriptive
- [ ] Add `Frontend :` or `Backend: ` per group [naming conventions](https://about.gitlab.com/handbook/engineering/development/ops/verify/pipeline-authoring/#splitting-issues)
- [ ] Ensure the issue containing the feature or change proposal and related discussions is linked as related to this implementation issue.
- [ ] Aside from default labeling, please make sure to include relevant labels for `type::`, `workflow::`, and `~frontend`/`~backend` labeling.
- [ ] Issues with user-facing changes should include the `~UX` label.
-->
## Summary
## Proposal
## Additional details
<!--
_NOTE: If the issue has addressed all of these questions, this separate section can be removed._
-->
Some relevant technical details, if applicable, such as:
- Does this need a ~"feature flag"?
- Does there need to be an associated ~"instrumentation" issue created related to this work?
- Is there an example response showing the data structure that should be returned (new endpoints only)?
- What permissions should be used?
- Is this EE or CE?
- [ ] EE
- [ ] CE
- Additional comments:
## Implementation Table
<!--
_NOTE: If the issue is not part of an epic, the implementation table can be removed. If it is part of an epic, make sure that the implementation table below mirrors the corresponding epic's implementation table content._
-->
| Group | Issue Link |
| ------ | ------ |
| ~backend | :point_left: You are here |
| ~frontend | [#123123](url) |
<!--
## Documentation
_NOTE: This section is optional, but can be used for easy access to any relevant documentation URLs._
-->
## Links/References
/label ~"group::pipeline authoring" ~"Category:Pipeline Composition" ~"section::ops" ~"devops::verify" ~"workflow::planning breakdown"

View file

@ -1,7 +1,3 @@
<!-- This template is used as a starting point for understanding and articulating a customer problem.
Learn more about it in the handbook: https://about.gitlab.com/handbook/product-development-flow/#validation-phase-2-problem-validation
-->
## Problem Statement
<!-- What is the problem we hope to validate? Reference how to write a real customer problem statement at https://productcoalition.com/how-to-write-a-good-customer-problem-statement-a815f80189ba for guidance. -->
@ -42,15 +38,4 @@ Learn more about it in the handbook: https://about.gitlab.com/handbook/product-d
For example, if the solution will take a product manager, designer, and engineer two weeks of effort - you may quantify this as 1.5 (based on 0.5 months x 3 people). -->
## Definition of Done
- [ ] The problem is well understood by the PM to have an understanding summarized in a RICE score
- [ ] The problem is well understood by the PM to decide if they want to move forward with this idea or drop it
- [ ] The problem is well described and detailed with necessary requirements for product design to understand the problem
- [ ] The problem is well described and detailed with necessary requirements for engineering to understand the problem
## Research Issue
<!-- Link to the Problem Validation Research issue that will be executed by the UX Researcher. https://gitlab.com/gitlab-org/ux-research/ -->
/label ~"workflow::validation backlog" ~devops:: ~category: ~group::

View file

@ -2,7 +2,7 @@
<!--
Please describe the engineering productivity problem that needs to be solved backed by charts from
https://about.gitlab.com/handbook/engineering/quality/engineering-productivity/#engineering-productivity-metrics.
https://about.gitlab.com/handbook/engineering/quality/engineering-productivity-team/#engineering-productivity-team-metrics.
-->
### Problem identification checklist

Some files were not shown because too many files have changed in this diff Show more