debian-mirror-gitlab/doc/administration/repository_storage_paths.md

128 lines
5.1 KiB
Markdown
Raw Normal View History

2020-07-28 23:09:34 +05:30
---
stage: Create
group: Gitaly
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers
type: reference, howto
---
2017-08-17 22:00:37 +05:30
# Repository storage paths
2020-05-24 23:13:21 +05:30
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/4578) in GitLab 8.10.
2017-08-17 22:00:37 +05:30
2019-12-26 22:10:19 +05:30
GitLab allows you to define multiple repository storage paths (sometimes called
storage shards) to distribute the storage load between several mount points.
2017-08-17 22:00:37 +05:30
2018-11-20 20:47:30 +05:30
> **Notes:**
2017-08-17 22:00:37 +05:30
>
2018-11-20 20:47:30 +05:30
> - You must have at least one storage path called `default`.
> - The paths are defined in key-value pairs. The key is an arbitrary name you
> can pick to name the file path.
2020-06-23 00:09:42 +05:30
> - The target directories and any of its sub-paths must not be a symlink.
2019-02-15 15:39:39 +05:30
> - No target directory may be a sub-directory of another; no nesting.
Example: this is OK:
2020-04-08 14:13:33 +05:30
```plaintext
2019-02-15 15:39:39 +05:30
default:
path: /mnt/git-storage-1
storage2:
path: /mnt/git-storage-2
```
This is not OK because it nests storage paths:
2020-04-08 14:13:33 +05:30
```plaintext
2019-02-15 15:39:39 +05:30
default:
path: /mnt/git-storage-1
storage2:
path: /mnt/git-storage-1/git-storage-2 # <- NOT OK because of nesting
```
2017-08-17 22:00:37 +05:30
## Configure GitLab
2018-11-20 20:47:30 +05:30
> **Warning:**
2020-05-24 23:13:21 +05:30
> In order for [backups](../raketasks/backup_restore.md) to work correctly, the storage path must **not** be a
2018-11-20 20:47:30 +05:30
> mount point and the GitLab user should have correct permissions for the parent
> directory of the path. In Omnibus GitLab this is taken care of automatically,
> but for source installations you should be extra careful.
2017-08-17 22:00:37 +05:30
>
2018-11-20 20:47:30 +05:30
> The thing is that for compatibility reasons `gitlab.yml` has a different
> structure than Omnibus. In `gitlab.yml` you indicate the path for the
> repositories, for example `/home/git/repositories`, while in Omnibus you
> indicate `git_data_dirs`, which for the example above would be `/home/git`.
> Then, Omnibus will create a `repositories` directory under that path to use with
> `gitlab.yml`.
2017-08-17 22:00:37 +05:30
>
2018-11-20 20:47:30 +05:30
> This little detail matters because while restoring a backup, the current
2020-05-24 23:13:21 +05:30
> contents of `/home/git/repositories` [are moved to](https://gitlab.com/gitlab-org/gitlab/blob/033e5423a2594e08a7ebcd2379bd2331f4c39032/lib/backup/repository.rb#L54-56) `/home/git/repositories.old`,
2018-11-20 20:47:30 +05:30
> so if `/home/git/repositories` is the mount point, then `mv` would be moving
> things between mount points, and bad things could happen. Ideally,
> `/home/git` would be the mount point, so then things would be moving within the
> same mount point. This is guaranteed with Omnibus installations (because they
> don't specify the full repository path but the parent path), but not for source
> installations.
2017-08-17 22:00:37 +05:30
Now that you've read that big fat warning above, let's edit the configuration
files and add the full paths of the alternative repository storage paths. In
2020-06-23 00:09:42 +05:30
the example below, we add two more mount points that are named `nfs_1` and `nfs_2`
2017-08-17 22:00:37 +05:30
respectively.
2020-07-28 23:09:34 +05:30
NOTE: **Note:**
2020-10-24 23:57:45 +05:30
This example uses NFS. We do not recommend using EFS for storage as it may impact GitLab's performance. See the [relevant documentation](nfs.md#avoid-using-awss-elastic-file-system-efs) for more details.
2019-07-07 11:18:12 +05:30
2017-08-17 22:00:37 +05:30
**For installations from source**
1. Edit `gitlab.yml` and add the storage paths:
2019-09-30 21:07:59 +05:30
```yaml
repositories:
# Paths where repositories can be stored. Give the canonicalized absolute pathname.
# NOTE: REPOS PATHS MUST NOT CONTAIN ANY SYMLINK!!!
storages: # You must have at least a 'default' storage path.
default:
path: /home/git/repositories
2019-10-12 21:52:04 +05:30
nfs_1:
path: /mnt/nfs1/repositories
nfs_2:
path: /mnt/nfs2/repositories
2019-09-30 21:07:59 +05:30
```
2017-08-17 22:00:37 +05:30
2020-05-24 23:13:21 +05:30
1. [Restart GitLab](restart_gitlab.md#installations-from-source) for the changes to take effect.
2017-08-17 22:00:37 +05:30
2020-10-24 23:57:45 +05:30
NOTE: **Note:**
2020-05-24 23:13:21 +05:30
The [`gitlab_shell: repos_path` entry](https://gitlab.com/gitlab-org/gitlab-foss/-/blob/8-9-stable/config/gitlab.yml.example#L457) in `gitlab.yml` will be
2017-08-17 22:00:37 +05:30
deprecated and replaced by `repositories: storages` in the future, so if you
are upgrading from a version prior to 8.10, make sure to add the configuration
as described in the step above. After you make the changes and confirm they are
working, you can remove the `repos_path` line.
**For Omnibus installations**
1. Edit `/etc/gitlab/gitlab.rb` by appending the rest of the paths to the
default one:
2019-09-30 21:07:59 +05:30
```ruby
git_data_dirs({
"default" => { "path" => "/var/opt/gitlab/git-data" },
2019-10-12 21:52:04 +05:30
"nfs_1" => { "path" => "/mnt/nfs1/git-data" },
"nfs_2" => { "path" => "/mnt/nfs2/git-data" }
2019-09-30 21:07:59 +05:30
})
```
2017-08-17 22:00:37 +05:30
2019-09-30 21:07:59 +05:30
Note that Omnibus stores the repositories in a `repositories` subdirectory
of the `git-data` directory.
2017-08-17 22:00:37 +05:30
2020-04-22 19:07:51 +05:30
## Choose where new repositories will be stored
2017-08-17 22:00:37 +05:30
2020-04-22 19:07:51 +05:30
Once you set the multiple storage paths, you can choose where new repositories
will be stored under **Admin Area > Settings > Repository >
Repository storage > Storage nodes for new repositories**.
2017-08-17 22:00:37 +05:30
2020-07-28 23:09:34 +05:30
Each storage can be assigned a weight from 0-100. When a new project is created, these
weights are used to determine the storage location the repository will be created on.
![Choose repository storage path in Admin Area](img/repository_storages_admin_ui_v13_1.png)
2017-08-17 22:00:37 +05:30
2020-04-22 19:07:51 +05:30
Beginning with GitLab 8.13.4, multiple paths can be chosen. New repositories
will be randomly placed on one of the selected paths.