debian-mirror-gitlab/doc/development/redis.md

59 lines
2.5 KiB
Markdown
Raw Normal View History

2020-03-13 15:44:24 +05:30
# Redis guidelines
GitLab uses [Redis](https://redis.io) for three distinct purposes:
- Caching via `Rails.cache`.
- As a job processing queue with [Sidekiq](sidekiq_style_guide.md).
- To manage the shared application state.
Every application process is configured to use the same Redis servers, so they
can be used for inter-process communication in cases where [PostgreSQL](sql.md)
2020-04-08 14:13:33 +05:30
is less appropriate. For example, transient state or data that is written much
2020-03-13 15:44:24 +05:30
more often than it is read.
If [Geo](geo.md) is enabled, each Geo node gets its own, independent Redis
database.
## Key naming
Redis is a flat namespace with no hierarchy, which means we must pay attention
to key names to avoid collisions. Typically we use colon-separated elements to
2020-06-23 00:09:42 +05:30
provide a semblance of structure at application level. An example might be
2020-03-13 15:44:24 +05:30
`projects:1:somekey`.
Although we split our Redis usage into three separate purposes, and those may
map to separate Redis servers in a [Highly Available](../administration/high_availability/redis.md)
configuration, the default Omnibus and GDK setups share a single Redis server.
This means that keys should **always** be globally unique across the three
purposes.
It is usually better to use immutable identifiers - project ID rather than
full path, for instance - in Redis key names. If full path is used, the key will
stop being consulted if the project is renamed. If the contents of the key are
invalidated by a name change, it is better to include a hook that will expire
the entry, instead of relying on the key changing.
2020-07-28 23:09:34 +05:30
### Multi-key commands
2020-03-13 15:44:24 +05:30
We don't use [Redis Cluster](https://redis.io/topics/cluster-tutorial) at the
2020-06-23 00:09:42 +05:30
moment, but may wish to in the future: [#118820](https://gitlab.com/gitlab-org/gitlab/-/issues/118820).
2020-03-13 15:44:24 +05:30
This imposes an additional constraint on naming: where GitLab is performing
operations that require several keys to be held on the same Redis server - for
instance, diffing two sets held in Redis - the keys should ensure that by
2020-10-24 23:57:45 +05:30
enclosing the changeable parts in curly braces.
For example:
```plaintext
project:{1}:set_a
project:{1}:set_b
project:{2}:set_c
```
`set_a` and `set_b` are guaranteed to be held on the same Redis server, while `set_c` is not.
2020-07-28 23:09:34 +05:30
Currently, we validate this in the development and test environments
with the [`RedisClusterValidator`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/instrumentation/redis_cluster_validator.rb),
which is enabled for the `cache` and `shared_state`
[Redis instances](https://docs.gitlab.com/omnibus/settings/redis.html#running-with-multiple-redis-instances)..