debian-mirror-gitlab/doc/development/rake_tasks.md

301 lines
9.2 KiB
Markdown
Raw Normal View History

2021-01-29 00:20:46 +05:30
---
stage: none
group: unassigned
2021-02-22 17:27:13 +05:30
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
2021-01-29 00:20:46 +05:30
---
2014-09-02 18:07:02 +05:30
# Rake tasks for developers
2020-05-24 23:13:21 +05:30
Rake tasks are available for developers and others contributing to GitLab.
2014-09-02 18:07:02 +05:30
2020-05-24 23:13:21 +05:30
## Set up database with developer seeds
Note that if your database user does not have advanced privileges, you must create the database manually before running this command.
2014-09-02 18:07:02 +05:30
2020-03-13 15:44:24 +05:30
```shell
2014-09-02 18:07:02 +05:30
bundle exec rake setup
```
2018-03-17 18:26:18 +05:30
The `setup` task is an alias for `gitlab:setup`.
2019-10-12 21:52:04 +05:30
This tasks calls `db:reset` to create the database, and calls `db:seed_fu` to seed the database.
2021-01-29 00:20:46 +05:30
`db:setup` calls `db:seed` but this does nothing.
2015-04-26 12:48:37 +05:30
2020-05-24 23:13:21 +05:30
### Environment variables
2019-12-26 22:10:19 +05:30
**MASS_INSERT**: Create millions of users (2m), projects (5m) and its
relations. It's highly recommended to run the seed with it to catch slow queries
while developing. Expect the process to take up to 20 extra minutes.
2020-05-24 23:13:21 +05:30
See also [Mass inserting Rails models](mass_insert.md).
**LARGE_PROJECTS**: Create large projects (through import) from a predefined set of URLs.
2019-12-26 22:10:19 +05:30
2019-07-31 22:56:46 +05:30
### Seeding issues for all or a given project
You can seed issues for all or a given project with the `gitlab:seed:issues`
task:
```shell
# All projects
bin/rake gitlab:seed:issues
# A specific project
bin/rake "gitlab:seed:issues[group-path/project-path]"
```
By default, this seeds an average of 2 issues per week for the last 5 weeks per
project.
2019-09-30 21:07:59 +05:30
#### Seeding issues for Insights charts **(ULTIMATE)**
2019-07-31 22:56:46 +05:30
You can seed issues specifically for working with the
2019-09-04 21:01:54 +05:30
[Insights charts](../user/group/insights/index.md) with the
2019-07-31 22:56:46 +05:30
`gitlab:seed:insights:issues` task:
```shell
# All projects
bin/rake gitlab:seed:insights:issues
# A specific project
bin/rake "gitlab:seed:insights:issues[group-path/project-path]"
```
By default, this seeds an average of 10 issues per week for the last 52 weeks
2021-02-22 17:27:13 +05:30
per project. All issues are also randomly labeled with team, type, severity,
2019-07-31 22:56:46 +05:30
and priority.
2020-03-13 15:44:24 +05:30
#### Seeding groups with sub-groups
You can seed groups with sub-groups that contain milestones/projects/issues
with the `gitlab:seed:group_seed` task:
```shell
bin/rake "gitlab:seed:group_seed[subgroup_depth, username]"
```
Group are additionally seeded with epics if GitLab instance has epics feature available.
2020-04-22 19:07:51 +05:30
#### Seeding custom metrics for the monitoring dashboard
A lot of different types of metrics are supported in the monitoring dashboard.
To import these metrics, you can run:
```shell
bundle exec rake 'gitlab:seed:development_metrics[your_project_id]'
```
2017-09-10 17:25:29 +05:30
### Automation
If you're very sure that you want to **wipe the current database** and refill
2021-02-22 17:27:13 +05:30
seeds, you can set the `FORCE` environment variable to `yes`:
2017-09-10 17:25:29 +05:30
2020-03-13 15:44:24 +05:30
```shell
2021-02-22 17:27:13 +05:30
FORCE=yes bundle exec rake setup
2017-09-10 17:25:29 +05:30
```
2021-02-22 17:27:13 +05:30
This will skip the action confirmation/safety check, saving you from answering
`yes` manually.
2017-09-10 17:25:29 +05:30
2020-05-24 23:13:21 +05:30
### Discard `stdout`
2017-09-10 17:25:29 +05:30
Since the script would print a lot of information, it could be slowing down
your terminal, and it would generate more than 20G logs if you just redirect
it to a file. If we don't care about the output, we could just redirect it to
`/dev/null`:
2020-03-13 15:44:24 +05:30
```shell
2017-09-10 17:25:29 +05:30
echo 'yes' | bundle exec rake setup > /dev/null
```
2020-05-24 23:13:21 +05:30
Note that since you can't see the questions from `stdout`, you might just want
to `echo 'yes'` to keep it running. It would still print the errors on `stderr`
2017-09-10 17:25:29 +05:30
so no worries about missing errors.
2019-03-02 22:35:43 +05:30
### Extra Project seed options
There are a few environment flags you can pass to change how projects are seeded
- `SIZE`: defaults to `8`, max: `32`. Amount of projects to create.
2021-02-22 17:27:13 +05:30
- `LARGE_PROJECTS`: defaults to false. If set, clones 6 large projects to help with testing.
- `FORK`: defaults to false. If set to `true`, forks `torvalds/linux` five times. Can also be set to an existing project `full_path` to fork that instead.
2019-03-02 22:35:43 +05:30
2014-09-02 18:07:02 +05:30
## Run tests
2016-09-13 17:45:13 +05:30
In order to run the test you can use the following commands:
2019-07-07 11:18:12 +05:30
2020-05-24 23:13:21 +05:30
- `bin/rake spec` to run the RSpec suite
2020-01-01 13:55:28 +05:30
- `bin/rake spec:unit` to run only the unit tests
- `bin/rake spec:integration` to run only the integration tests
- `bin/rake spec:system` to run only the system tests
- `bin/rake karma` to run the Karma test suite
2014-09-02 18:07:02 +05:30
2020-05-24 23:13:21 +05:30
`bin/rake spec` takes significant time to pass.
Instead of running the full test suite locally, you can save a lot of time by running
a single test or directory related to your changes. After you submit a merge request,
2021-02-22 17:27:13 +05:30
CI runs full test suite for you. Green CI status in the merge request means
2017-08-17 22:00:37 +05:30
full test suite is passed.
2016-09-13 17:45:13 +05:30
2021-02-22 17:27:13 +05:30
You can't run `rspec .` since this tries to run all the `_spec.rb`
2016-09-13 17:45:13 +05:30
files it can find, also the ones in `/tmp`
2020-05-24 23:13:21 +05:30
You can pass RSpec command line options to the `spec:unit`,
`spec:integration`, and `spec:system` tasks. For example, `bin/rake "spec:unit[--tag ~geo --dry-run]"`.
2019-09-04 21:01:54 +05:30
2020-05-24 23:13:21 +05:30
For an RSpec test, to run a single test file you can run:
2016-09-13 17:45:13 +05:30
2020-05-24 23:13:21 +05:30
```shell
bin/rspec spec/controllers/commit_controller_spec.rb
```
2016-09-13 17:45:13 +05:30
To run several tests inside one directory:
2020-05-24 23:13:21 +05:30
- `bin/rspec spec/requests/api/` for the RSpec tests if you want to test API only
2016-09-13 17:45:13 +05:30
2020-05-24 23:13:21 +05:30
### Speed up tests, Rake tasks, and migrations
2014-09-02 18:07:02 +05:30
2020-06-23 00:09:42 +05:30
[Spring](https://github.com/rails/spring) is a Rails application pre-loader. It
2017-08-17 22:00:37 +05:30
speeds up development by keeping your application running in the background so
2020-04-22 19:07:51 +05:30
you don't need to boot it every time you run a test, Rake task or migration.
2014-09-02 18:07:02 +05:30
2021-02-22 17:27:13 +05:30
If you want to use it, you must export the `ENABLE_SPRING` environment
2017-08-17 22:00:37 +05:30
variable to `1`:
2014-09-02 18:07:02 +05:30
2020-03-13 15:44:24 +05:30
```shell
2017-08-17 22:00:37 +05:30
export ENABLE_SPRING=1
2014-09-02 18:07:02 +05:30
```
2015-09-25 12:07:36 +05:30
2018-03-27 19:54:05 +05:30
Alternatively you can use the following on each spec run,
2020-03-13 15:44:24 +05:30
```shell
2018-03-27 19:54:05 +05:30
bundle exec spring rspec some_spec.rb
```
2017-08-17 22:00:37 +05:30
## Compile Frontend Assets
You shouldn't ever need to compile frontend assets manually in development, but
if you ever need to test how the assets get compiled in a production
environment you can do so with the following command:
2020-03-13 15:44:24 +05:30
```shell
2017-08-17 22:00:37 +05:30
RAILS_ENV=production NODE_ENV=production bundle exec rake gitlab:assets:compile
```
2021-02-22 17:27:13 +05:30
This compiles and minifies all JavaScript and CSS assets and copy them along
2017-08-17 22:00:37 +05:30
with all other frontend assets (images, fonts, etc) into `/public/assets` where
they can be easily inspected.
2020-05-24 23:13:21 +05:30
## Emoji tasks
2018-03-17 18:26:18 +05:30
2020-05-24 23:13:21 +05:30
To update the Emoji aliases file (used for Emoji autocomplete), run the
2018-03-17 18:26:18 +05:30
following:
2020-03-13 15:44:24 +05:30
```shell
2018-03-17 18:26:18 +05:30
bundle exec rake gemojione:aliases
```
2020-05-24 23:13:21 +05:30
To update the Emoji digests file (used for Emoji autocomplete), run the
2016-08-24 12:49:21 +05:30
following:
2020-03-13 15:44:24 +05:30
```shell
2016-08-24 12:49:21 +05:30
bundle exec rake gemojione:digests
```
2021-02-22 17:27:13 +05:30
This updates the file `fixtures/emojis/digests.json` based on the currently
2016-08-24 12:49:21 +05:30
available Emoji.
2020-05-24 23:13:21 +05:30
To generate a sprite file containing all the Emoji, run:
2016-08-24 12:49:21 +05:30
2020-03-13 15:44:24 +05:30
```shell
2016-08-24 12:49:21 +05:30
bundle exec rake gemojione:sprite
```
2020-06-23 00:09:42 +05:30
If new emoji are added, the sprite sheet may change size. To compensate for
such changes, first generate the `emoji.png` sprite sheet with the above Rake
task, then check the dimensions of the new sprite sheet and update the
2016-08-24 12:49:21 +05:30
`SPRITESHEET_WIDTH` and `SPRITESHEET_HEIGHT` constants accordingly.
2017-09-10 17:25:29 +05:30
2020-05-24 23:13:21 +05:30
## Update project templates
2017-09-10 17:25:29 +05:30
Starting a project from a template needs this project to be exported. On a
2020-01-01 13:55:28 +05:30
up to date master branch run:
2017-09-10 17:25:29 +05:30
2020-03-13 15:44:24 +05:30
```shell
2020-01-01 13:55:28 +05:30
gdk start
2017-09-10 17:25:29 +05:30
bundle exec rake gitlab:update_project_templates
git checkout -b update-project-templates
git add vendor/project_templates
git commit
git push -u origin update-project-templates
```
Now create a merge request and merge that to master.
2018-11-08 19:23:39 +05:30
## Generate route lists
To see the full list of API routes, you can run:
```shell
bundle exec rake grape:path_helpers
```
2020-07-28 23:09:34 +05:30
The generated list includes a full list of API endpoints and functional
RESTful API verbs.
2018-11-08 19:23:39 +05:30
For the Rails controllers, run:
```shell
bundle exec rake routes
```
Since these take some time to create, it's often helpful to save the output to
a file for quick reference.
2019-10-12 21:52:04 +05:30
2019-12-04 20:38:33 +05:30
## Show obsolete `ignored_columns`
To see a list of all obsolete `ignored_columns` run:
2020-03-13 15:44:24 +05:30
```shell
2019-12-04 20:38:33 +05:30
bundle exec rake db:obsolete_ignored_columns
```
Feel free to remove their definitions from their `ignored_columns` definitions.
2019-12-21 20:55:43 +05:30
2020-05-24 23:13:21 +05:30
## Update GraphQL documentation and schema definitions
2019-12-21 20:55:43 +05:30
To generate GraphQL documentation based on the GitLab schema, run:
```shell
bundle exec rake gitlab:graphql:compile_docs
```
2020-04-22 19:07:51 +05:30
In its current state, the Rake task:
2019-12-21 20:55:43 +05:30
- Generates output for GraphQL objects.
- Places the output at `doc/api/graphql/reference/index.md`.
This uses some features from `graphql-docs` gem like its schema parser and helper methods.
The docs generator code comes from our side giving us more flexibility, like using Haml templates and generating Markdown files.
2021-02-22 17:27:13 +05:30
To edit the content, you may need to edit the following:
- The template. You can edit the template at `lib/gitlab/graphql/docs/templates/default.md.haml`.
The actual renderer is at `Gitlab::Graphql::Docs::Renderer`.
- The applicable `description` field in the code, which
[Updates machine-readable schema files](#update-machine-readable-schema-files),
which is then used by the `rake` task described earlier.
2019-12-21 20:55:43 +05:30
`@parsed_schema` is an instance variable that the `graphql-docs` gem expects to have available.
`Gitlab::Graphql::Docs::Helper` defines the `object` method we currently use. This is also where you
should implement any new methods for new types you'd like to display.
2019-12-26 22:10:19 +05:30
### Update machine-readable schema files
To generate GraphQL schema files based on the GitLab schema, run:
```shell
bundle exec rake gitlab:graphql:schema:dump
```
2020-04-22 19:07:51 +05:30
This uses GraphQL Ruby's built-in Rake tasks to generate files in both [IDL](https://www.prisma.io/blog/graphql-sdl-schema-definition-language-6755bcb9ce51) and JSON formats.