debian-mirror-gitlab/doc/development/application_limits.md

210 lines
8.2 KiB
Markdown
Raw Normal View History

2021-01-29 00:20:46 +05:30
---
2022-07-23 23:45:48 +05:30
stage: Systems
2021-03-11 19:13:27 +05:30
group: Distribution
2022-11-25 23:54:43 +05:30
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
2021-01-29 00:20:46 +05:30
---
2020-03-13 15:44:24 +05:30
# Application limits development
This document provides a development guide for contributors to add application
limits to GitLab.
## Documentation
First of all, you have to gather information and decide which are the different
2022-08-13 15:12:31 +05:30
limits that are set for the different GitLab tiers. Coordinate with others to [document](../administration/instance_limits.md)
2020-03-13 15:44:24 +05:30
and communicate those limits.
2022-08-27 11:52:29 +05:30
There is a guide about [introducing application limits](https://about.gitlab.com/handbook/product/product-processes/#introducing-application-limits).
2020-03-13 15:44:24 +05:30
2022-06-21 17:19:12 +05:30
## Implement plan limits
2020-03-13 15:44:24 +05:30
### Insert database plan limits
2020-11-24 15:15:51 +05:30
In the `plan_limits` table, create a new column and insert the limit values.
It's recommended to create two separate migration script files.
1. Add a new column to the `plan_limits` table with non-null default value that
represents desired limit, such as:
```ruby
add_column(:plan_limits, :project_hooks, :integer, default: 100, null: false)
```
Plan limits entries set to `0` mean that limits are not enabled. You should
use this setting only in special and documented circumstances.
1. (Optionally) Create the database migration that fine-tunes each level with a
desired limit using `create_or_update_plan_limit` migration helper, such as:
```ruby
2023-01-13 00:05:48 +05:30
class InsertProjectHooksPlanLimits < Gitlab::Database::Migration[2.0]
2020-11-24 15:15:51 +05:30
def up
create_or_update_plan_limit('project_hooks', 'default', 0)
create_or_update_plan_limit('project_hooks', 'free', 10)
create_or_update_plan_limit('project_hooks', 'bronze', 20)
create_or_update_plan_limit('project_hooks', 'silver', 30)
2021-09-30 23:02:18 +05:30
create_or_update_plan_limit('project_hooks', 'premium', 30)
create_or_update_plan_limit('project_hooks', 'premium_trial', 30)
2020-11-24 15:15:51 +05:30
create_or_update_plan_limit('project_hooks', 'gold', 100)
2021-09-30 23:02:18 +05:30
create_or_update_plan_limit('project_hooks', 'ultimate', 100)
create_or_update_plan_limit('project_hooks', 'ultimate_trial', 100)
2020-11-24 15:15:51 +05:30
end
def down
create_or_update_plan_limit('project_hooks', 'default', 0)
create_or_update_plan_limit('project_hooks', 'free', 0)
create_or_update_plan_limit('project_hooks', 'bronze', 0)
create_or_update_plan_limit('project_hooks', 'silver', 0)
2021-09-30 23:02:18 +05:30
create_or_update_plan_limit('project_hooks', 'premium', 0)
create_or_update_plan_limit('project_hooks', 'premium_trial', 0)
2020-11-24 15:15:51 +05:30
create_or_update_plan_limit('project_hooks', 'gold', 0)
2021-09-30 23:02:18 +05:30
create_or_update_plan_limit('project_hooks', 'ultimate', 0)
create_or_update_plan_limit('project_hooks', 'ultimate_trial', 0)
2020-11-24 15:15:51 +05:30
end
end
```
2021-02-22 17:27:13 +05:30
Some plans exist only on GitLab.com. This is a no-op for plans
2020-11-24 15:15:51 +05:30
that do not exist.
2020-03-13 15:44:24 +05:30
### Plan limits validation
#### Get current limit
Access to the current limit can be done through the project or the namespace,
2020-06-23 00:09:42 +05:30
such as:
2020-03-13 15:44:24 +05:30
```ruby
project.actual_limits.project_hooks
```
#### Check current limit
There is one method `PlanLimits#exceeded?` to check if the current limit is
being exceeded. You can use either an `ActiveRecord` object or an `Integer`.
2020-06-23 00:09:42 +05:30
Ensures that the count of the records does not exceed the defined limit, such as:
2020-03-13 15:44:24 +05:30
```ruby
project.actual_limits.exceeded?(:project_hooks, ProjectHook.where(project: project))
```
2020-06-23 00:09:42 +05:30
Ensures that the number does not exceed the defined limit, such as:
2020-03-13 15:44:24 +05:30
```ruby
project.actual_limits.exceeded?(:project_hooks, 10)
```
#### `Limitable` concern
2021-09-04 01:27:46 +05:30
The [`Limitable` concern](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/concerns/limitable.rb)
2020-03-13 15:44:24 +05:30
can be used to validate that a model does not exceed the limits. It ensures
that the count of the records for the current model does not exceed the defined
limit.
2020-07-28 23:09:34 +05:30
You must specify the limit scope of the object being validated
2020-04-08 14:13:33 +05:30
and the limit name if it's different from the pluralized model name.
2020-03-13 15:44:24 +05:30
```ruby
class ProjectHook
include Limitable
2020-04-08 14:13:33 +05:30
self.limit_name = 'project_hooks' # Optional as ProjectHook corresponds with project_hooks
self.limit_scope = :project
2020-03-13 15:44:24 +05:30
end
```
2020-04-08 14:13:33 +05:30
To test the model, you can include the shared examples.
```ruby
it_behaves_like 'includes Limitable concern' do
subject { build(:project_hook, project: create(:project)) }
end
```
2020-06-23 00:09:42 +05:30
### Testing instance-wide limits
Instance-wide features always use `default` Plan, as instance-wide features
do not have license assigned.
```ruby
class InstanceVariable
include Limitable
self.limit_name = 'instance_variables' # Optional as InstanceVariable corresponds with instance_variables
self.limit_scope = Limitable::GLOBAL_SCOPE
end
```
2020-04-08 14:13:33 +05:30
### Subscription Plans
2022-03-02 08:16:31 +05:30
> The `opensource` plan was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/346399) in GitLab 14.7.
2020-04-08 14:13:33 +05:30
Self-managed:
2021-04-17 20:07:23 +05:30
- `default`: Everyone.
2020-04-08 14:13:33 +05:30
GitLab.com:
2021-04-17 20:07:23 +05:30
- `default`: Any system-wide feature.
- `free`: Namespaces and projects with a Free subscription.
- `bronze`: Namespaces and projects with a Bronze subscription. This tier is no longer available for purchase.
- `silver`: Namespaces and projects with a Premium subscription.
2021-09-30 23:02:18 +05:30
- `premium`: Namespaces and projects with a Premium subscription.
- `premium_trial`: Namespaces and projects with a Premium Trial subscription.
2021-04-17 20:07:23 +05:30
- `gold`: Namespaces and projects with an Ultimate subscription.
2021-09-30 23:02:18 +05:30
- `ultimate`: Namespaces and projects with an Ultimate subscription.
- `ultimate_trial`: Namespaces and projects with an Ultimate Trial subscription.
2022-03-02 08:16:31 +05:30
- `opensource`: Namespaces and projects that are member of GitLab Open Source program.
2020-04-08 14:13:33 +05:30
2021-01-29 00:20:46 +05:30
The `test` environment doesn't have any plans.
2022-06-21 17:19:12 +05:30
## Implement rate limits using `Rack::Attack`
We use the [`Rack::Attack`](https://github.com/rack/rack-attack) middleware to throttle Rack requests.
This applies to Rails controllers, Grape endpoints, and any other Rack requests.
The process for adding a new throttle is loosely:
1. Add new columns to the `ApplicationSetting` model (`*_enabled`, `*_requests_per_period`, `*_period_in_seconds`).
1. Extend `Gitlab::RackAttack` and `Gitlab::RackAttack::Request` to configure the new rate limit,
and apply it to the desired requests.
1. Add the new settings to the Admin Area form in `app/views/admin/application_settings/_ip_limits.html.haml`.
1. Document the new settings in [User and IP rate limits](../user/admin_area/settings/user_and_ip_rate_limits.md) and [Application settings API](../api/settings.md).
1. Configure the rate limit for GitLab.com and document it in [GitLab.com-specific rate limits](../user/gitlab_com/index.md#gitlabcom-specific-rate-limits).
Refer to these past issues for implementation details:
- [Create a separate rate limit for the Files API](https://gitlab.com/gitlab-org/gitlab/-/issues/335075).
- [Create a separate rate limit for unauthenticated API traffic](https://gitlab.com/gitlab-org/gitlab/-/issues/335300).
## Implement rate limits using `Gitlab::ApplicationRateLimiter`
This module implements a custom rate limiter that can be used to throttle
certain actions. Unlike `Rack::Attack` and `Rack::Throttle`, which operate at
the middleware level, this can be used at the controller or API level.
See the `CheckRateLimit` concern for use in controllers. In other parts of the code
the `Gitlab::ApplicationRateLimiter` module can be called directly.
2022-07-23 23:45:48 +05:30
## Next rate limiting architecture
In May 2022 we've started working on the next iteration of our application
limits framework using a forward looking rate limiting architecture.
We are working on defining new requirements and designing the next
architecture, so if you need new functionalities to add new limits, instead of
building them right now, consider contributing to the
[Rate Limiting Architecture Working Group](https://about.gitlab.com/company/team/structure/working-groups/rate-limit-architecture/)
Examples of what features we might want to build into the next iteration of
rate limiting architecture:
1. Making it possible to define and override limits per namespace / per plan.
1. Automatically generating documentation about what limits are implemented and
what the defaults are.
2022-08-13 15:12:31 +05:30
1. Defining limits in a single place that can be found and explored.
2022-07-23 23:45:48 +05:30
1. Soft and hard limits, with support for notifying users when a limit is
approaching.