debian-mirror-gitlab/doc/raketasks/import.md

177 lines
7.1 KiB
Markdown
Raw Normal View History

2021-01-29 00:20:46 +05:30
---
2022-07-23 23:45:48 +05:30
stage: Systems
2021-03-11 19:13:27 +05:30
group: Distribution
2022-11-25 23:54:43 +05:30
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
2021-01-29 00:20:46 +05:30
---
2023-03-17 16:20:25 +05:30
# Import bare repositories (deprecated) **(FREE SELF)**
WARNING:
The Rake task for importing bare repositories was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/108507)
in GitLab 15.8 and will be removed in GitLab 16.0.
Alternatives to using the `gitlab:import:repos` Rake task include:
- Migrating projects using either [an export file](../user/project/settings/import_export.md) or [direct transfer](../user/group/import/index.md#migrate-groups-by-direct-transfer-recommended) migrate repositories as well.
- Importing a [repository by URL](../user/project/import/repo_by_url.md).
- Importing a [repositories from a non-GitLab source](../user/project/import/index.md).
2014-09-02 18:07:02 +05:30
2020-05-24 23:13:21 +05:30
Rake tasks are available to import bare repositories into a GitLab instance.
2023-03-17 16:20:25 +05:30
When migrating from an existing GitLab instance, and to preserve ownership by users and their namespaces,
migrate projects using either:
- [Direct transfer](../user/group/import/index.md#migrate-groups-by-direct-transfer-recommended).
- [An export file](../user/project/settings/import_export.md).
2014-09-02 18:07:02 +05:30
2023-04-23 21:23:45 +05:30
When you import a repository:
2014-09-02 18:07:02 +05:30
2021-02-22 17:27:13 +05:30
- The owner of the project is the first administrator.
- The groups are created as needed, including subgroups.
- The owner of the group is the first administrator.
- Existing projects are skipped.
2020-05-24 23:13:21 +05:30
- Projects in hashed storage may be skipped. For more information, see
[Importing bare repositories from hashed storage](#importing-bare-repositories-from-hashed-storage).
2022-11-25 23:54:43 +05:30
- The existing Git repositories are moved from disk (removed from the original path).
- You must manually [push Git LFS objects](#push-git-lfs-objects).
2014-09-02 18:07:02 +05:30
2020-05-24 23:13:21 +05:30
To import bare repositories into a GitLab instance:
2014-09-02 18:07:02 +05:30
2020-11-24 15:15:51 +05:30
1. Create a new folder to import your Git repositories from.
You can also import projects into a (sub)group's namespace,
instead of the administrator's namespace. To do so, create subfolders and
give ownership and read/write/execute permissions of those subfolders to the
`git` user and its group:
2015-04-26 12:48:37 +05:30
2020-05-24 23:13:21 +05:30
```shell
2020-11-24 15:15:51 +05:30
sudo -u git mkdir -p /var/opt/gitlab/git-data/repository-import-$(date "+%Y-%m-%d")/<optional_groupname>/<optional_subgroup>
2020-05-24 23:13:21 +05:30
```
2015-04-26 12:48:37 +05:30
2020-05-24 23:13:21 +05:30
1. Copy your bare repositories inside this newly created folder. Note:
2018-03-17 18:26:18 +05:30
2021-02-22 17:27:13 +05:30
- Any `.git` repositories found on any of the subfolders are imported as projects.
- Groups are created as needed, these could be nested folders.
2018-03-17 18:26:18 +05:30
2020-11-24 15:15:51 +05:30
For example, if we copy the repositories to `/var/opt/gitlab/git-data/repository-import-2020-08-22`,
2022-08-13 15:12:31 +05:30
and repository `A` must be under the groups `G1` and `G2`, it must be created under those folders:
2020-11-24 15:15:51 +05:30
`/var/opt/gitlab/git-data/repository-import-2020-08-22/G1/G2/A.git`.
2015-04-26 12:48:37 +05:30
2020-05-24 23:13:21 +05:30
```shell
2020-11-24 15:15:51 +05:30
sudo cp -r /old/git/foo.git /var/opt/gitlab/git-data/repository-import-$(date "+%Y-%m-%d")/<optional_groupname>/<optional_subgroup>
2015-04-26 12:48:37 +05:30
2020-05-24 23:13:21 +05:30
# Do this once when you are done copying git repositories
2020-11-24 15:15:51 +05:30
sudo chown -R git:git /var/opt/gitlab/git-data/repository-import-$(date "+%Y-%m-%d")
2020-05-24 23:13:21 +05:30
```
2015-04-26 12:48:37 +05:30
2022-08-13 15:12:31 +05:30
`foo.git` must be owned by the `git` user and `git` users group.
2015-04-26 12:48:37 +05:30
2020-05-24 23:13:21 +05:30
If you are using an installation from source, replace `/var/opt/gitlab/` with `/home/git`.
2015-04-26 12:48:37 +05:30
2020-05-24 23:13:21 +05:30
1. Run the following command depending on your type of installation:
2014-09-02 18:07:02 +05:30
2020-05-24 23:13:21 +05:30
- Omnibus Installation
2015-04-26 12:48:37 +05:30
2020-05-24 23:13:21 +05:30
```shell
2021-12-11 22:18:48 +05:30
sudo gitlab-rake gitlab:import:repos["/var/opt/gitlab/git-data/repository-import-$(date "+%Y-%m-%d")"]
2020-05-24 23:13:21 +05:30
```
2014-09-02 18:07:02 +05:30
2022-08-13 15:12:31 +05:30
- Installation from source. Before running this command you must change to the directory where
2020-05-24 23:13:21 +05:30
your GitLab installation is located:
2015-04-26 12:48:37 +05:30
2020-05-24 23:13:21 +05:30
```shell
cd /home/git/gitlab
2021-12-11 22:18:48 +05:30
sudo -u git -H bundle exec rake gitlab:import:repos["/var/opt/gitlab/git-data/repository-import-$(date "+%Y-%m-%d")"] RAILS_ENV=production
2020-05-24 23:13:21 +05:30
```
2014-09-02 18:07:02 +05:30
2020-05-24 23:13:21 +05:30
## Example output
2014-09-02 18:07:02 +05:30
2020-04-08 14:13:33 +05:30
```plaintext
2018-03-17 18:26:18 +05:30
Processing /var/opt/gitlab/git-data/repository-import-1/a/b/c/blah.git
* Using namespace: a/b/c
* Created blah (a/b/c/blah)
* Skipping repo /var/opt/gitlab/git-data/repository-import-1/a/b/c/blah.wiki.git
Processing /var/opt/gitlab/git-data/repository-import-1/abcd.git
2014-09-02 18:07:02 +05:30
* Created abcd (abcd.git)
2018-03-17 18:26:18 +05:30
Processing /var/opt/gitlab/git-data/repository-import-1/group/xyz.git
* Using namespace: group (2)
2014-09-02 18:07:02 +05:30
* Created xyz (group/xyz.git)
2018-03-17 18:26:18 +05:30
* Skipping repo /var/opt/gitlab/git-data/repository-import-1/@shared/a/b/abcd.git
2014-09-02 18:07:02 +05:30
[...]
```
2018-12-13 13:39:08 +05:30
## Importing bare repositories from hashed storage
Projects in legacy storage have a directory structure that mirrors their full
project path in GitLab, including their namespace structure. This information is
leveraged by the bare repository importer to import projects into their proper
locations. Each project and its parent namespaces are meaningfully named.
However, the directory structure of projects in hashed storage do not contain
this information. This is beneficial for a variety of reasons, especially
improved performance and data integrity. See
[Repository Storage Types](../administration/repository_storage_types.md) for
more details.
2020-05-24 23:13:21 +05:30
The repositories that are importable depends on the version of GitLab.
2018-12-13 13:39:08 +05:30
2020-05-24 23:13:21 +05:30
### GitLab 10.3 or earlier
2018-12-13 13:39:08 +05:30
Importing bare repositories from hashed storage is unsupported.
2020-05-24 23:13:21 +05:30
### GitLab 10.4 and later
2018-12-13 13:39:08 +05:30
To support importing bare repositories from hashed storage, GitLab 10.4 and
later stores the full project path with each repository, in a special section of
2020-05-24 23:13:21 +05:30
the Git repository's configuration file. This section is formatted as follows:
2018-12-13 13:39:08 +05:30
2020-04-08 14:13:33 +05:30
```ini
2018-12-13 13:39:08 +05:30
[gitlab]
2019-12-21 20:55:43 +05:30
fullpath = gitlab-org/gitlab
2018-12-13 13:39:08 +05:30
```
However, existing repositories were not migrated to include this path.
Bare repositories are importable if the following events occurred to the
repository in GitLab 10.4 and later:
- Created
- Migrated to hashed storage
- Renamed
- Transferred to another namespace
- Ancestor renamed
- Ancestor transferred to another namespace
2020-06-23 00:09:42 +05:30
Bare repositories are **not** importable by GitLab 10.4 to GitLab 11.6, if all the following are true about the repository:
2018-12-13 13:39:08 +05:30
- It was created in GitLab 10.3 or earlier.
2020-06-23 00:09:42 +05:30
- It was not renamed, transferred, or migrated to [hashed storage](../administration/repository_storage_types.md#hashed-storage) in GitLab 10.4 to GitLab 11.6.
- Its ancestor namespaces were not renamed or transferred in GitLab 10.4 to GitLab 11.6.
2018-12-13 13:39:08 +05:30
2021-01-03 14:25:43 +05:30
[In GitLab 11.6](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/41776) and later, all
2020-06-23 00:09:42 +05:30
bare repositories are importable.
2018-12-13 13:39:08 +05:30
2020-06-23 00:09:42 +05:30
To manually migrate repositories yourself (for GitLab 10.4 to GitLab 11.6), you can use the
2021-01-03 14:25:43 +05:30
[Rails console](../administration/operations/rails_console.md#starting-a-rails-console-session)
2018-12-13 13:39:08 +05:30
to do so. In a Rails console session, run the following to migrate a project:
2020-04-08 14:13:33 +05:30
```ruby
2019-12-21 20:55:43 +05:30
project = Project.find_by_full_path('gitlab-org/gitlab')
2021-10-27 15:23:28 +05:30
project.set_full_path
2018-12-13 13:39:08 +05:30
```
In a Rails console session, run the following to migrate all of a namespace's
2023-04-23 21:23:45 +05:30
projects (this may take a while if there are thousands of projects in a namespace):
2018-12-13 13:39:08 +05:30
2020-04-08 14:13:33 +05:30
```ruby
2018-12-13 13:39:08 +05:30
namespace = Namespace.find_by_full_path('gitlab-org')
namespace.send(:write_projects_repository_config)
```
2022-11-25 23:54:43 +05:30
## Push Git LFS objects
The import task doesn't import Git LFS objects. You must manually push the LFS objects to the newly
created GitLab repository using the following command:
```shell
git lfs push --all
```