debian-mirror-gitlab/doc/raketasks/import.md

143 lines
5.1 KiB
Markdown
Raw Normal View History

2015-04-26 12:48:37 +05:30
# Import bare repositories into your GitLab instance
2014-09-02 18:07:02 +05:30
2015-04-26 12:48:37 +05:30
## Notes
2014-09-02 18:07:02 +05:30
2015-04-26 12:48:37 +05:30
- The owner of the project will be the first admin
2018-03-17 18:26:18 +05:30
- The groups will be created as needed, including subgroups
2015-04-26 12:48:37 +05:30
- The owner of the group will be the first admin
- Existing projects will be skipped
2018-12-13 13:39:08 +05:30
- Projects in hashed storage may be skipped (see [Importing bare repositories from hashed storage](#importing-bare-repositories-from-hashed-storage))
2018-03-17 18:26:18 +05:30
- The existing Git repos will be moved from disk (removed from the original path)
2014-09-02 18:07:02 +05:30
2015-04-26 12:48:37 +05:30
## How to use
2014-09-02 18:07:02 +05:30
2019-12-04 20:38:33 +05:30
### Create a new folder to import your Git repositories from
2014-09-02 18:07:02 +05:30
2019-12-04 20:38:33 +05:30
The new folder needs to have Git user ownership and read/write/execute access for Git user and its group:
2015-04-26 12:48:37 +05:30
```
2019-09-04 21:01:54 +05:30
sudo -u git mkdir -p /var/opt/gitlab/git-data/repository-import-<date>/new_group
2015-04-26 12:48:37 +05:30
```
2019-12-04 20:38:33 +05:30
### Copy your bare repositories inside this newly created folder
2015-04-26 12:48:37 +05:30
2019-12-04 20:38:33 +05:30
- Any `.git` repositories found on any of the subfolders will be imported as projects
2018-03-17 18:26:18 +05:30
- Groups will be created as needed, these could be nested folders. Example:
If we copy the repos to `/var/opt/gitlab/git-data/repository-import-<date>`, and repo A needs to be under the groups G1 and G2, it will
have to be created under those folders: `/var/opt/gitlab/git-data/repository-import-<date>/G1/G2/A.git`.
2015-04-26 12:48:37 +05:30
```
2018-03-17 18:26:18 +05:30
sudo cp -r /old/git/foo.git /var/opt/gitlab/git-data/repository-import-<date>/new_group/
2015-04-26 12:48:37 +05:30
# Do this once when you are done copying git repositories
2018-03-17 18:26:18 +05:30
sudo chown -R git:git /var/opt/gitlab/git-data/repository-import-<date>
2015-04-26 12:48:37 +05:30
```
2019-12-04 20:38:33 +05:30
`foo.git` needs to be owned by the `git` user and `git` users group.
2015-04-26 12:48:37 +05:30
2018-03-17 18:26:18 +05:30
If you are using an installation from source, replace `/var/opt/gitlab/` with `/home/git`.
2015-04-26 12:48:37 +05:30
2019-12-04 20:38:33 +05:30
### Run the command below depending on your type of installation
2015-04-26 12:48:37 +05:30
#### Omnibus Installation
2014-09-02 18:07:02 +05:30
2020-03-09 13:42:32 +05:30
```shell
2019-12-04 20:38:33 +05:30
sudo gitlab-rake gitlab:import:repos['/var/opt/gitlab/git-data/repository-import-<date>']
2015-04-26 12:48:37 +05:30
```
#### Installation from source
2014-09-02 18:07:02 +05:30
2015-04-26 12:48:37 +05:30
Before running this command you need to change the directory to where your GitLab installation is located:
2020-03-09 13:42:32 +05:30
```shell
2019-12-04 20:38:33 +05:30
cd /home/git/gitlab
sudo -u git -H bundle exec rake gitlab:import:repos['/var/opt/gitlab/git-data/repository-import-<date>'] RAILS_ENV=production
2014-09-02 18:07:02 +05:30
```
2015-04-26 12:48:37 +05:30
#### Example output
2014-09-02 18:07:02 +05:30
```
2018-03-17 18:26:18 +05:30
Processing /var/opt/gitlab/git-data/repository-import-1/a/b/c/blah.git
* Using namespace: a/b/c
* Created blah (a/b/c/blah)
* Skipping repo /var/opt/gitlab/git-data/repository-import-1/a/b/c/blah.wiki.git
Processing /var/opt/gitlab/git-data/repository-import-1/abcd.git
2014-09-02 18:07:02 +05:30
* Created abcd (abcd.git)
2018-03-17 18:26:18 +05:30
Processing /var/opt/gitlab/git-data/repository-import-1/group/xyz.git
* Using namespace: group (2)
2014-09-02 18:07:02 +05:30
* Created xyz (group/xyz.git)
2018-03-17 18:26:18 +05:30
* Skipping repo /var/opt/gitlab/git-data/repository-import-1/@shared/a/b/abcd.git
2014-09-02 18:07:02 +05:30
[...]
```
2018-12-13 13:39:08 +05:30
## Importing bare repositories from hashed storage
### Background
Projects in legacy storage have a directory structure that mirrors their full
project path in GitLab, including their namespace structure. This information is
leveraged by the bare repository importer to import projects into their proper
locations. Each project and its parent namespaces are meaningfully named.
However, the directory structure of projects in hashed storage do not contain
this information. This is beneficial for a variety of reasons, especially
improved performance and data integrity. See
[Repository Storage Types](../administration/repository_storage_types.md) for
more details.
### Which repositories are importable?
#### GitLab 10.3 or earlier
Importing bare repositories from hashed storage is unsupported.
#### GitLab 10.4 and later
To support importing bare repositories from hashed storage, GitLab 10.4 and
later stores the full project path with each repository, in a special section of
2019-12-04 20:38:33 +05:30
the Git repository's config file. This section is formatted as follows:
2018-12-13 13:39:08 +05:30
```
[gitlab]
2019-12-21 20:55:43 +05:30
fullpath = gitlab-org/gitlab
2018-12-13 13:39:08 +05:30
```
However, existing repositories were not migrated to include this path.
Bare repositories are importable if the following events occurred to the
repository in GitLab 10.4 and later:
- Created
- Migrated to hashed storage
- Renamed
- Transferred to another namespace
- Ancestor renamed
- Ancestor transferred to another namespace
Bare repositories are **not** importable by GitLab 10.4 and later when all the following are true about the repository:
- It was created in GitLab 10.3 or earlier.
- It was not renamed, transferred, or migrated to hashed storage in GitLab 10.4 and later.
- Its ancestor namespaces were not renamed or transferred in GitLab 10.4 and later.
There is an [open issue to add a migration to make all bare repositories
2019-12-04 20:38:33 +05:30
importable](https://gitlab.com/gitlab-org/gitlab-foss/issues/41776).
2018-12-13 13:39:08 +05:30
Until then, you may wish to manually migrate repositories yourself. You can use
[Rails console](https://docs.gitlab.com/omnibus/maintenance/#starting-a-rails-console-session)
to do so. In a Rails console session, run the following to migrate a project:
```
2019-12-21 20:55:43 +05:30
project = Project.find_by_full_path('gitlab-org/gitlab')
2018-12-13 13:39:08 +05:30
project.write_repository_config
```
In a Rails console session, run the following to migrate all of a namespace's
projects (this may take a while if there are 1000s of projects in a namespace):
```
namespace = Namespace.find_by_full_path('gitlab-org')
namespace.send(:write_projects_repository_config)
```