4

I created a private package for my employer. Since I’m forbidden to upload it to PyPI (it’s proprietary), I uploaded it to the packages index for my project on our private Gitlab hub. I can install it manually with:

$ pip install my-package --extra-index-url https://__token__:[email protected]/api/v4/projects/123456/packages/pypi/simple

Now I also want setuptools to be able to find it when listed in the install_requires argument to setup(). I tried:

setup( install_requires=[ f"my-package @ https://__token__:{API_TOKEN}@gitlab.company-domain.com/api/v4/projects/123456/packages/pypi/simple", ... ], ...

pip install -e . results in

ERROR: HTTP error 404 while getting https://__token__:****@gitlab.company-domain.com/api/v4/projects/123456/packages/pypi/simple

This is different than

my-package @ git+https://user:[email protected]/..../my-package.git

That works, but I want to be able to download it as a pre-built wheel.

I’m not sure whether this is a setuptools issue or a Gitlab issue. The 404 response tells me that it might be a gitlab issue, yet the same URI works perfectly when used with the pip install CLI command.

This question is similar to Include python packages from gitlab's package registry and other external indexes directly into setup.py dependencies, but I don't think that one got sufficient response. I posted the same question to discuss.python.org, but that discussion is old and I think I might get a quicker response here.

I also found this response to a similar question, which wasn't encouraging. It recommends Poetry or Pipenv. I've tried both, and found each to be excruciatingly slow when resolving dependencies, so I fell back on setuptools.

Improve this question

3 Answers 3

Reset to default
2

Only include the package name in install_requires. Then, configure your (extra) index URL in your pip configuration (either environment variables or pip.conf/.pypirc or CLI argument). Then using pip install as normal will work.

For example:

In setup.py:

 # ... install_requires=[ 'my-package-name', # ... ], # ...

Then the install command (assuming the environment variable API_TOKEN exists):

GITLAB_INDEX="https://__token__:${API_TOKEN}@gitlab.company-domain.com/api/v4/projects/123456/packages/pypi/simple" pip install --extra-index-url "${GITLAB_INDEX}" -e .
Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

how to add it in .pypirc?
1

My solution is do not use the "GitLab package registry" approach, but use simple the "repository level" approach:

dependencies = [ "my-package @ git+https://deploy-token-name:[email protected]/path-to-package-project/my-package.git", ]

You might generate deploy-token in repository setting.

This solution allows to bypass any deals with extra-index-url. And anyone in my team can install my package on any platform without an extra configuration of pip.

Improve this answer

Comments

-2

I found out that you can just maintain your Python wheels in a directly like so: /path/to/simple/my-module. I don’t even bother to use twine to post new versions of my module. Instead, I use a simple bash script that makes sure that new wheel files are given read-only permissions so I can’t accidentally overwrite them without bumping the semver version number. With Docker and nginx, I set up a simple “autoindex” HTTP file server to the “simple” top-level-directory so I can access the wheels from other hosts if I want to. Finally, in my pip.conf, I have extra-index-url=https://my-server:port/simple and trusted-host=my-server.

I tried hosting my custom-built Python wheels in Gitlab and OneArtifactory. My company maintains instances of both. But I found that the access requirements and red-tape were too cumbersome, so I switched to my roll-your-own method, which I regret not having taken up much sooner.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.