2020-06-23 00:09:42 +05:30
---
stage: Package
2023-01-13 00:05:48 +05:30
group: Package Registry
2022-11-25 23:54:43 +05:30
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
2020-06-23 00:09:42 +05:30
---
2021-03-11 19:13:27 +05:30
# PyPI packages in the Package Registry **(FREE)**
2020-04-22 19:07:51 +05:30
2021-12-11 22:18:48 +05:30
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208747) in GitLab 12.10.
> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) from GitLab Premium to GitLab Free in 13.3.
2020-04-22 19:07:51 +05:30
2021-04-29 21:17:54 +05:30
Publish PyPI packages in your project's Package Registry. Then install the
2021-01-29 00:20:46 +05:30
packages whenever you need to use them as a dependency.
2020-04-22 19:07:51 +05:30
2021-01-29 00:20:46 +05:30
The Package Registry works with:
2020-04-22 19:07:51 +05:30
- [pip ](https://pypi.org/project/pip/ )
- [twine ](https://pypi.org/project/twine/ )
2021-04-29 21:17:54 +05:30
For documentation of the specific API endpoints that the `pip` and `twine`
clients use, see the [PyPI API documentation ](../../../api/packages/pypi.md ).
2023-01-13 00:05:48 +05:30
Learn how to [build a PyPI package ](../workflows/build_packages.md#pypi ).
2020-04-22 19:07:51 +05:30
2021-01-29 00:20:46 +05:30
## Authenticate with the Package Registry
2020-04-22 19:07:51 +05:30
2021-01-29 00:20:46 +05:30
Before you can publish to the Package Registry, you must authenticate.
2020-05-24 23:13:21 +05:30
2021-01-29 00:20:46 +05:30
To do this, you can use:
2020-04-22 19:07:51 +05:30
2021-01-29 00:20:46 +05:30
- A [personal access token ](../../../user/profile/personal_access_tokens.md )
with the scope set to `api` .
- A [deploy token ](../../project/deploy_tokens/index.md ) with the scope set to
`read_package_registry` , `write_package_registry` , or both.
- A [CI job token ](#authenticate-with-a-ci-job-token ).
2020-04-22 19:07:51 +05:30
2021-01-29 00:20:46 +05:30
### Authenticate with a personal access token
To authenticate with a personal access token, edit the `~/.pypirc` file and add:
2020-04-22 19:07:51 +05:30
```ini
2020-05-24 23:13:21 +05:30
[distutils]
index-servers =
gitlab
2020-04-22 19:07:51 +05:30
[gitlab]
2021-01-29 00:20:46 +05:30
repository = https://gitlab.example.com/api/v4/projects/< project_id > /packages/pypi
2021-04-17 20:07:23 +05:30
username = < your_personal_access_token_name >
password = < your_personal_access_token >
2020-04-22 19:07:51 +05:30
```
2021-10-27 15:23:28 +05:30
The `<project_id>` is either the project's
[URL-encoded ](../../../api/index.md#namespaced-path-encoding )
path (for example, `group%2Fproject` ), or the project's ID (for example `42` ).
2020-05-24 23:13:21 +05:30
2021-01-29 00:20:46 +05:30
### Authenticate with a deploy token
2020-05-24 23:13:21 +05:30
2021-01-29 00:20:46 +05:30
To authenticate with a deploy token, edit your `~/.pypirc` file and add:
2020-05-24 23:13:21 +05:30
```ini
[distutils]
index-servers =
gitlab
[gitlab]
2021-01-29 00:20:46 +05:30
repository = https://gitlab.example.com/api/v4/projects/< project_id > /packages/pypi
2020-05-24 23:13:21 +05:30
username = < deploy token username >
password = < deploy token >
```
2021-10-27 15:23:28 +05:30
The `<project_id>` is either the project's
[URL-encoded ](../../../api/index.md#namespaced-path-encoding )
path (for example, `group%2Fproject` ), or the project's ID (for example `42` ).
2020-04-22 19:07:51 +05:30
2021-01-29 00:20:46 +05:30
### Authenticate with a CI job token
2020-04-22 19:07:51 +05:30
2021-01-29 00:20:46 +05:30
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202012) in GitLab 13.4.
2021-09-30 23:02:18 +05:30
To work with PyPI commands within [GitLab CI/CD ](../../../ci/index.md ), you
2021-01-29 00:20:46 +05:30
can use `CI_JOB_TOKEN` instead of a personal access token or deploy token.
For example:
```yaml
image: python:latest
run:
script:
2022-08-13 15:12:31 +05:30
- pip install build twine
- python -m build
2021-09-04 01:27:46 +05:30
- TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token python -m twine upload --repository-url ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/pypi dist/*
2021-01-29 00:20:46 +05:30
```
You can also use `CI_JOB_TOKEN` in a `~/.pypirc` file that you check in to
GitLab:
```ini
[distutils]
index-servers =
gitlab
[gitlab]
repository = https://gitlab.example.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/pypi
username = gitlab-ci-token
password = ${env.CI_JOB_TOKEN}
```
2021-09-04 01:27:46 +05:30
### Authenticate to access packages within a group
Follow the instructions above for the token type, but use the group URL in place of the project URL:
```shell
https://gitlab.example.com/api/v4/groups/< group_id > /-/packages/pypi
```
2021-01-29 00:20:46 +05:30
## Publish a PyPI package
2021-03-08 18:12:59 +05:30
Prerequisites:
2021-01-29 00:20:46 +05:30
2021-03-08 18:12:59 +05:30
- You must [authenticate with the Package Registry ](#authenticate-with-the-package-registry ).
- Your [version string must be valid ](#ensure-your-version-string-is-valid ).
- The maximum allowed package size is 5 GB.
2021-01-29 00:20:46 +05:30
- You can't upload the same version of a package multiple times. If you try,
2021-03-08 18:12:59 +05:30
you receive the error `400 Bad Request` .
2022-05-07 20:08:51 +05:30
- PyPI packages are published using your projectID.
- If your project is in a group, PyPI packages published to your project registry are also available
at the group-level registry (see [Install from the group level ](#install-from-the-group-level )).
2021-03-08 18:12:59 +05:30
You can then [publish a package by using twine ](#publish-a-pypi-package-by-using-twine ).
2020-11-24 15:15:51 +05:30
### Ensure your version string is valid
2021-02-22 17:27:13 +05:30
If your version string (for example, `0.0.1` ) isn't valid, it gets rejected.
2021-01-29 00:20:46 +05:30
GitLab uses the following regex to validate the version string.
2020-11-24 15:15:51 +05:30
```ruby
\A(?:
v?
(?:([0-9]+)!)? (?# epoch)
([0-9]+(?:\.[0-9]+)*) (?# release segment)
([-_\.]?((a|b|c|rc|alpha|beta|pre|preview))[-_\.]?([0-9]+)?)? (?# pre-release)
((?:-([0-9]+))|(?:[-_\.]?(post|rev|r)[-_\.]?([0-9]+)?))? (?# post release)
([-_\.]?(dev)[-_\.]?([0-9]+)?)? (?# dev release)
(?:\+([a-z0-9]+(?:[-_\.][a-z0-9]+)*))? (?# local version)
)\z}xi
```
2021-01-29 00:20:46 +05:30
You can experiment with the regex and try your version strings by using this
[regular expression editor ](https://rubular.com/r/FKM6d07ouoDaFV ).
2020-11-24 15:15:51 +05:30
2021-01-29 00:20:46 +05:30
For more details about the regex, review this [documentation ](https://www.python.org/dev/peps/pep-0440/#appendix-b-parsing-version-strings-with-regular-expressions ).
2020-04-22 19:07:51 +05:30
2021-01-29 00:20:46 +05:30
### Publish a PyPI package by using twine
2020-04-22 19:07:51 +05:30
2021-01-29 00:20:46 +05:30
To publish a PyPI package, run a command like:
2020-05-24 23:13:21 +05:30
```shell
python3 -m twine upload --repository gitlab dist/*
```
2021-01-29 00:20:46 +05:30
This message indicates that the package was published successfully:
2020-05-24 23:13:21 +05:30
```plaintext
2021-01-29 00:20:46 +05:30
Uploading distributions to https://gitlab.example.com/api/v4/projects/< your_project_id > /packages/pypi
2020-05-24 23:13:21 +05:30
Uploading mypypipackage-0.0.1-py3-none-any.whl
100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.58k/4.58k [00:00< 00:00 , 10 . 9kB / s ]
Uploading mypypipackage-0.0.1.tar.gz
100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.24k/4.24k [00:00< 00:00 , 11 . 0kB / s ]
```
2022-10-11 01:57:18 +05:30
To view the published package, go to your project's **Packages and registries**
2021-01-29 00:20:46 +05:30
page.
2020-05-24 23:13:21 +05:30
2021-01-29 00:20:46 +05:30
If you didn't use a `.pypirc` file to define your repository source, you can
publish to the repository with the authentication inline:
2020-11-24 15:15:51 +05:30
```shell
2021-01-29 00:20:46 +05:30
TWINE_PASSWORD=< personal_access_token or deploy_token > TWINE_USERNAME=< username or deploy_token_username > python3 -m twine upload --repository-url https://gitlab.example.com/api/v4/projects/< project_id > /packages/pypi dist/*
2020-11-24 15:15:51 +05:30
```
2021-01-29 00:20:46 +05:30
If you didn't follow the steps on this page, ensure your package was properly
built, and that you [created a PyPI package with `setuptools` ](https://packaging.python.org/tutorials/packaging-projects/ ).
2020-05-24 23:13:21 +05:30
2021-01-29 00:20:46 +05:30
You can then upload your package by using the following command:
2020-04-22 19:07:51 +05:30
```shell
python -m twine upload --repository < source_name > dist/< package_file >
```
- `<package_file>` is your package filename, ending in `.tar.gz` or `.whl` .
2021-01-29 00:20:46 +05:30
- `<source_name>` is the [source name used during setup ](#authenticate-with-the-package-registry ).
2020-04-22 19:07:51 +05:30
2021-03-08 18:12:59 +05:30
### Publishing packages with the same name or version
You cannot publish a package if a package of the same name and version already exists.
2022-07-16 23:28:13 +05:30
You must [delete the existing package ](../../packages/package_registry/reduce_package_registry_storage.md#delete-a-package ) first.
If you attempt to publish the same package
2021-09-30 23:02:18 +05:30
more than once, a `400 Bad Request` error occurs.
2021-03-08 18:12:59 +05:30
2021-01-29 00:20:46 +05:30
## Install a PyPI package
2020-04-22 19:07:51 +05:30
2021-10-27 15:23:28 +05:30
In [GitLab 14.2 and later ](https://gitlab.com/gitlab-org/gitlab/-/issues/233413 ),
when a PyPI package is not found in the Package Registry, the request is forwarded to [pypi.org ](https://pypi.org/ ).
Administrators can disable this behavior in the [Continuous Integration settings ](../../admin_area/settings/continuous_integration.md ).
2022-09-01 20:07:04 +05:30
WARNING:
When you use the `--index-url` option, do not specify the port if it is a default
port, such as `80` for a URL starting with `http` or `443` for a URL starting
with `https` .
2021-09-04 01:27:46 +05:30
### Install from the project level
2021-01-29 00:20:46 +05:30
To install the latest version of a package, use the following command:
2020-04-22 19:07:51 +05:30
```shell
2021-04-17 20:07:23 +05:30
pip install --index-url https://< personal_access_token_name > :< personal_access_token > @gitlab.example.com/api/v4/projects/< project_id > /packages/pypi/simple --no-deps < package_name >
2020-04-22 19:07:51 +05:30
```
- `<package_name>` is the package name.
2021-04-17 20:07:23 +05:30
- `<personal_access_token_name>` is a personal access token name with the `read_api` scope.
2020-05-24 23:13:21 +05:30
- `<personal_access_token>` is a personal access token with the `read_api` scope.
2021-10-27 15:23:28 +05:30
- `<project_id>` is either the project's [URL-encoded ](../../../api/index.md#namespaced-path-encoding )
path (for example, `group%2Fproject` ), or the project's ID (for example `42` ).
2020-05-24 23:13:21 +05:30
2021-04-17 20:07:23 +05:30
In these commands, you can use `--extra-index-url` instead of `--index-url` . However, using
`--extra-index-url` makes you vulnerable to dependency confusion attacks because it checks the PyPi
repository for the package before it checks the custom repository. `--extra-index-url` adds the
provided URL as an additional registry which the client checks if the package is present.
`--index-url` tells the client to check for the package on the provided URL only.
2021-01-29 00:20:46 +05:30
If you were following the guide and want to install the
`MyPyPiPackage` package, you can run:
2020-05-24 23:13:21 +05:30
```shell
2021-04-17 20:07:23 +05:30
pip install mypypipackage --no-deps --index-url https://< personal_access_token_name > :< personal_access_token > @gitlab.example.com/api/v4/projects/< your_project_id > /packages/pypi/simple
2020-05-24 23:13:21 +05:30
```
2021-01-29 00:20:46 +05:30
This message indicates that the package was installed successfully:
2020-05-24 23:13:21 +05:30
```plaintext
2021-04-17 20:07:23 +05:30
Looking in indexes: https://< personal_access_token_name > :****@gitlab.example.com/api/v4/projects/< your_project_id > /packages/pypi/simple
2020-05-24 23:13:21 +05:30
Collecting mypypipackage
2021-01-29 00:20:46 +05:30
Downloading https://gitlab.example.com/api/v4/projects/< your_project_id > /packages/pypi/files/d53334205552a355fee8ca35a164512ef7334f33d309e60240d57073ee4386e6/mypypipackage-0.0.1-py3-none-any.whl (1.6 kB)
2020-05-24 23:13:21 +05:30
Installing collected packages: mypypipackage
Successfully installed mypypipackage-0.0.1
```
2020-11-24 15:15:51 +05:30
2021-09-04 01:27:46 +05:30
### Install from the group level
To install the latest version of a package from a group, use the following command:
```shell
pip install --index-url https://< personal_access_token_name > :< personal_access_token > @gitlab.example.com/api/v4/groups/< group_id > /-/packages/pypi/simple --no-deps < package_name >
```
In this command:
- `<package_name>` is the package name.
- `<personal_access_token_name>` is a personal access token name with the `read_api` scope.
- `<personal_access_token>` is a personal access token with the `read_api` scope.
- `<group_id>` is the group ID.
In these commands, you can use `--extra-index-url` instead of `--index-url` . However, using
`--extra-index-url` makes you vulnerable to dependency confusion attacks because it checks the PyPi
repository for the package before it checks the custom repository. `--extra-index-url` adds the
provided URL as an additional registry which the client checks if the package is present.
`--index-url` tells the client to check for the package at the provided URL only.
If you're following the guide and want to install the `MyPyPiPackage` package, you can run:
```shell
pip install mypypipackage --no-deps --index-url https://< personal_access_token_name > :< personal_access_token > @gitlab.example.com/api/v4/groups/< your_group_id > /-/packages/pypi/simple
```
2021-01-29 00:20:46 +05:30
### Package names
2020-11-24 15:15:51 +05:30
2021-01-29 00:20:46 +05:30
GitLab looks for packages that use
[Python normalized names (PEP-503) ](https://www.python.org/dev/peps/pep-0503/#normalized-names ).
The characters `-` , `_` , and `.` are all treated the same, and repeated
characters are removed.
2020-11-24 15:15:51 +05:30
2021-01-29 00:20:46 +05:30
A `pip install` request for `my.package` looks for packages that match any of
the three characters, such as `my-package` , `my_package` , and `my....package` .
2021-04-29 21:17:54 +05:30
2023-03-04 22:38:38 +05:30
## Using `requirements.txt`
If you want pip to access your private registry, add the `--extra-index-url` parameter along with the URL for your registry to your `requirements.txt` file.
```plaintext
--extra-index-url https://gitlab.example.com/api/v4/projects/< project_id > /packages/pypi/simple
package-name==1.0.0
```
If this is a private registry, you can authenticate in a couple of ways. For example:
- Using your `requirements.txt` file:
```plaintext
--extra-index-url https://__token__:< your_personal_token > @gitlab.example.com/api/v4/projects/< project_id > /packages/pypi/simple
package-name==1.0.0
```
- Using a `~/.netrc` file:
```plaintext
machine gitlab.example.com
login __token__
password < your_personal_token >
```
2022-04-04 11:22:00 +05:30
## Troubleshooting
2022-11-25 23:54:43 +05:30
To improve performance, the pip command caches files related to a package. Note that pip doesn't remove data by
2022-04-04 11:22:00 +05:30
itself. The cache grows as new packages are installed. If you encounter issues, clear the cache with
this command:
```shell
pip cache purge
```
2023-03-04 22:38:38 +05:30
### Multiple `index-url` or `extra-index-url` parameters
You can define multiple `index-url` and `extra-index-url` parameters.
If you use the same domain name (such as `gitlab.example.com` ) multiple times with different authentication
tokens, `pip` may not be able to find your packages. This problem is due to how `pip`
[registers and stores your tokens ](https://github.com/pypa/pip/pull/10904#issuecomment-1126690115 ) during commands executions.
To workaround this issue, you can use a [group deploy token ](../../project/deploy_tokens/index.md ) with the
scope `read_package_registry` from a common parent group for all projects or groups targeted by the
`index-url` and `extra-index-url` values.
2021-04-29 21:17:54 +05:30
## Supported CLI commands
The GitLab PyPI repository supports the following CLI commands:
- `twine upload` : Upload a package to the registry.
- `pip install` : Install a PyPI package from the registry.