2022-07-17 14:43:12 +02:00

15 KiB

Raw Blame History

stage	group	info
Package	Package	To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments

PyPI packages in the Package Registry (FREE)

Introduced in GitLab 12.10.

Moved from GitLab Premium to GitLab Free in 13.3.

Publish PyPI packages in your project's Package Registry. Then install the packages whenever you need to use them as a dependency.

The Package Registry works with:

pip
twine

For documentation of the specific API endpoints that the pip and twine clients use, see the PyPI API documentation.

Build a PyPI package

This section explains how to create a PyPI package.

If you already use PyPI and know how to build your own packages, go to the next section.

Install pip and twine

Install a recent version of pip and twine.

Create a project

Create a test project.

Open your terminal.
Create a directory called MyPyPiPackage, and then go to that directory:
```
mkdir MyPyPiPackage && cd MyPyPiPackage
```

Create another directory and go to it:

mkdir mypypipackage && cd mypypipackage

Create the required files in this directory:
```
touch __init__.py
touch greet.py
```

Open the greet.py file, and then add:

def SayHello():
    print("Hello from MyPyPiPackage")
    return

Open the __init__.py file, and then add:
```
from .greet import SayHello
```
To test the code, in your MyPyPiPackage directory, start the Python prompt.
```
python
```

Run this command:

>>> from mypypipackage import SayHello
>>> SayHello()

A message indicates that the project was set up successfully:

Python 3.8.2 (v3.8.2:7b3ab5921f, Feb 24 2020, 17:52:18)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from mypypipackage import SayHello
>>> SayHello()
Hello from MyPyPiPackage

Create a package

After you create a project, you can create a package.

In your terminal, go to the MyPyPiPackage directory.
Create a setup.py file:
```
touch setup.py
```
This file contains all the information about the package. For more information about this file, see creating setup.py. Because GitLab identifies packages based on Python normalized names (PEP-503), ensure your package name meets these requirements. See the installation section for details.

Open the setup.py file, and then add basic information:

import setuptools

setuptools.setup(
    name="mypypipackage",
    version="0.0.1",
    author="Example Author",
    author_email="author@example.com",
    description="A small example package",
    packages=setuptools.find_packages(),
    classifiers=[
        "Programming Language :: Python :: 3",
        "License :: OSI Approved :: MIT License",
        "Operating System :: OS Independent",
    ],
    python_requires='>=3.6',
)

Save the file.
Execute the setup:
```
python3 setup.py sdist bdist_wheel
```

The output should be visible in a newly-created dist folder:

ls dist

The output should appear similar to the following:

mypypipackage-0.0.1-py3-none-any.whl mypypipackage-0.0.1.tar.gz

The package is now ready to be published to the Package Registry.

Authenticate with the Package Registry

Before you can publish to the Package Registry, you must authenticate.

To do this, you can use:

A personal access token with the scope set to api.
A deploy token with the scope set to read_package_registry, write_package_registry, or both.
A CI job token.

Authenticate with a personal access token

To authenticate with a personal access token, edit the ~/.pypirc file and add:

[distutils]
index-servers =
    gitlab

[gitlab]
repository = https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi
username = <your_personal_access_token_name>
password = <your_personal_access_token>

The <project_id> is either the project's URL-encoded path (for example, group%2Fproject), or the project's ID (for example 42).

Authenticate with a deploy token

To authenticate with a deploy token, edit your ~/.pypirc file and add:

[distutils]
index-servers =
    gitlab

[gitlab]
repository = https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi
username = <deploy token username>
password = <deploy token>

The <project_id> is either the project's URL-encoded path (for example, group%2Fproject), or the project's ID (for example 42).

Authenticate with a CI job token

Introduced in GitLab 13.4.

To work with PyPI commands within GitLab CI/CD, you can use CI_JOB_TOKEN instead of a personal access token or deploy token.

For example:

image: python:latest

run:
  script:
    - pip install twine
    - python setup.py sdist bdist_wheel
    - TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token python -m twine upload --repository-url ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/pypi dist/*

You can also use CI_JOB_TOKEN in a ~/.pypirc file that you check in to GitLab:

[distutils]
index-servers =
    gitlab

[gitlab]
repository = https://gitlab.example.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/pypi
username = gitlab-ci-token
password = ${env.CI_JOB_TOKEN}

Authenticate to access packages within a group

Follow the instructions above for the token type, but use the group URL in place of the project URL:

https://gitlab.example.com/api/v4/groups/<group_id>/-/packages/pypi

Publish a PyPI package

Prerequisites:

You must authenticate with the Package Registry.
Your version string must be valid.
The maximum allowed package size is 5 GB.
You can't upload the same version of a package multiple times. If you try, you receive the error 400 Bad Request.
PyPI packages are published using your projectID.
If your project is in a group, PyPI packages published to your project registry are also available at the group-level registry (see Install from the group level).

You can then publish a package by using twine.

Ensure your version string is valid

If your version string (for example, 0.0.1) isn't valid, it gets rejected. GitLab uses the following regex to validate the version string.

\A(?:
    v?
    (?:([0-9]+)!)?                                                 (?# epoch)
    ([0-9]+(?:\.[0-9]+)*)                                          (?# release segment)
    ([-_\.]?((a|b|c|rc|alpha|beta|pre|preview))[-_\.]?([0-9]+)?)?  (?# pre-release)
    ((?:-([0-9]+))|(?:[-_\.]?(post|rev|r)[-_\.]?([0-9]+)?))?       (?# post release)
    ([-_\.]?(dev)[-_\.]?([0-9]+)?)?                                (?# dev release)
    (?:\+([a-z0-9]+(?:[-_\.][a-z0-9]+)*))?                         (?# local version)
)\z}xi

You can experiment with the regex and try your version strings by using this regular expression editor.

For more details about the regex, review this documentation.

Publish a PyPI package by using twine

To publish a PyPI package, run a command like:

python3 -m twine upload --repository gitlab dist/*

This message indicates that the package was published successfully:

Uploading distributions to https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi
Uploading mypypipackage-0.0.1-py3-none-any.whl
100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.58k/4.58k [00:00<00:00, 10.9kB/s]
Uploading mypypipackage-0.0.1.tar.gz
100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.24k/4.24k [00:00<00:00, 11.0kB/s]

To view the published package, go to your project's Packages & Registries page.

If you didn't use a .pypirc file to define your repository source, you can publish to the repository with the authentication inline:

TWINE_PASSWORD=<personal_access_token or deploy_token> TWINE_USERNAME=<username or deploy_token_username> python3 -m twine upload --repository-url https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi dist/*

If you didn't follow the steps on this page, ensure your package was properly built, and that you created a PyPI package with setuptools.

You can then upload your package by using the following command:

python -m twine upload --repository <source_name> dist/<package_file>

<package_file> is your package filename, ending in .tar.gz or .whl.
<source_name> is the source name used during setup.

Publishing packages with the same name or version

You cannot publish a package if a package of the same name and version already exists. You must delete the existing package first. If you attempt to publish the same package more than once, a 400 Bad Request error occurs.

Install a PyPI package

In GitLab 14.2 and later, when a PyPI package is not found in the Package Registry, the request is forwarded to pypi.org.

Administrators can disable this behavior in the Continuous Integration settings.

Install from the project level

To install the latest version of a package, use the following command:

pip install --index-url https://<personal_access_token_name>:<personal_access_token>@gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/simple --no-deps <package_name>

<package_name> is the package name.
<personal_access_token_name> is a personal access token name with the read_api scope.
<personal_access_token> is a personal access token with the read_api scope.
<project_id> is either the project's URL-encoded path (for example, group%2Fproject), or the project's ID (for example 42).

In these commands, you can use --extra-index-url instead of --index-url. However, using --extra-index-url makes you vulnerable to dependency confusion attacks because it checks the PyPi repository for the package before it checks the custom repository. --extra-index-url adds the provided URL as an additional registry which the client checks if the package is present. --index-url tells the client to check for the package on the provided URL only.

If you were following the guide and want to install the MyPyPiPackage package, you can run:

pip install mypypipackage --no-deps --index-url https://<personal_access_token_name>:<personal_access_token>@gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi/simple

This message indicates that the package was installed successfully:

Looking in indexes: https://<personal_access_token_name>:****@gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi/simple
Collecting mypypipackage
  Downloading https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi/files/d53334205552a355fee8ca35a164512ef7334f33d309e60240d57073ee4386e6/mypypipackage-0.0.1-py3-none-any.whl (1.6 kB)
Installing collected packages: mypypipackage
Successfully installed mypypipackage-0.0.1

Install from the group level

To install the latest version of a package from a group, use the following command:

pip install --index-url https://<personal_access_token_name>:<personal_access_token>@gitlab.example.com/api/v4/groups/<group_id>/-/packages/pypi/simple --no-deps <package_name>

In this command:

<package_name> is the package name.
<personal_access_token_name> is a personal access token name with the read_api scope.
<personal_access_token> is a personal access token with the read_api scope.
<group_id> is the group ID.

If you're following the guide and want to install the MyPyPiPackage package, you can run:

pip install mypypipackage --no-deps --index-url https://<personal_access_token_name>:<personal_access_token>@gitlab.example.com/api/v4/groups/<your_group_id>/-/packages/pypi/simple

Package names

GitLab looks for packages that use Python normalized names (PEP-503). The characters -, _, and . are all treated the same, and repeated characters are removed.

A pip install request for my.package looks for packages that match any of the three characters, such as my-package, my_package, and my....package.

Troubleshooting

To improve performance, PyPI caches files related to a package. Note that PyPI doesn't remove data by itself. The cache grows as new packages are installed. If you encounter issues, clear the cache with this command:

pip cache purge

Supported CLI commands

The GitLab PyPI repository supports the following CLI commands:

twine upload: Upload a package to the registry.
pip install: Install a PyPI package from the registry.

15 KiB Raw Blame History