In my corporate environment we are forced to proxy PyPI through our internal artifactory service, e.g.:
https://[COMPANY URL]/artifactory/api/pypi/pypi-release/simple I therefore set up my pyproject.toml with this as the primary source:
[[tool.poetry.source]] name = "artifactory_release" url = "https://[COMPANY URL]/artifactory/api/pypi/pypi-release/simple" priority = "primary" The problem is that my company actually hosts multiple levels of artifactory for different types of internal builds. Most projects would get built & hosted in a different entirely firewalled internal repository, which does not have access to the wider PyPI:
https://[COMPANY URL]/artifactory/api/pypi/python-internal-unstable/simple Normally I would list this as a secondary/supplemental source, like so:
[[tool.poetry.source]] name = "artifactory_unstable" url = "https://[COMPANY URL]/artifactory/api/pypi/python-internal-unstable/simple" priority = "supplemental" I would then expect to be able to pull packages specifically from this secondary repository by specifying the dependencies in my pyproject.toml file like so:
[tool.poetry.dependencies] python = "^3.10 <3.12" numpy = "1.23.5" pandas = "1.4.3" [MY PACKAGE] = {version = "[VERSION NO]", source = "artifactory_unstable"} My problem is that when I try to pull some internally built packages from this secondary source, it looks like Poetry tries to resolve their dependencies (i.e. transitive dependencies) through that same secondary source, which does not have access to the wider PyPI. This therefore results in an error:
(base) my-computer: curr_dir$ poetry lock Updating dependencies Resolving dependencies... (10.6s) ValueError Package('[MY PACKAGE]', '[VERSION NO]') is not in list at ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/repositories/legacy_repository.py:66 in package 62│ Note that this will be cached so the subsequent operations 63│ should be much faster. 64│ """ 65│ try: → 66│ index = self._packages.index(Package(name, version)) 67│ 68│ return self._packages[index] 69│ except ValueError: 70│ package = super().package(name, version, extras) The following error occurred when trying to handle this error: AssertionError at ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/inspection/lazy_wheel.py:526 in _fetch_content_length 522│ # If we *could* download some file contents, then write them to the end of 523│ # the file and set up our bisect boundaries by hand. 524│ with self._stay(): 525│ response_length = int(tail.headers["Content-Length"]) → 526│ assert response_length == min(initial_chunk_size, ret_length) 527│ self.seek(-response_length, io.SEEK_END) 528│ # Default initial chunk size is currently 1MB, but streaming content 529│ # here allows it to be set arbitrarily large. 530│ for chunk in tail.iter_content(CONTENT_CHUNK_SIZE): When I use normal pip rather than Poetry, I can force the correct behaviour by specifying the primary PyPI index URL as an extra index URL for this specific package. This allows pip to pull my package from the internal firewalled repository, but to resolve its dependencies through the public PyPI proxy:
(base) my-computer: curr_dir$ python -m pip install --index-url https://[COMPANY URL]/artifactory/api/pypi/python-internal-unstable/simple/ --extra-index-url https://[COMPANY URL]/artifactory/api/pypi/pypi-release/simple [MY PACKAGE] Looking in indexes: https://[COMPANY URL]/artifactory/api/pypi/python-internal-unstable/simple/, https://[COMPANY URL]/artifactory/api/pypi/pypi-release/simple Collecting [MY PACKAGE] Using cached https://[COMPANY URL]/artifactory/api/pypi/python-internal-unstable/[DIR PATH]/[MY PACKAGE]-[VERSION NO]-py3-none-any.whl (26 kB) Requirement already satisfied: azure-monitor-opentelemetry<2.0.0,>=1.6.5 in /Users/[ME]/Library/Caches/pypoetry/virtualenvs/[POETRY ENV]/lib/python3.11/site-packages (from [MY PACKAGE]) (1.6.10) Collecting opencensus-ext-azure<2.0.0,>=1.1.4 (from [MY PACKAGE]) ... ... ... Downloading https://COMPANY URL/artifactory/api/pypi/pypi-release/packages/packages/.../.../pyasn1-0.6.1-py3-none-any.whl (83 kB) Installing collected packages: opencensus-context, typing-inspection, python-dotenv, pydantic-core, ..., opencensus-ext-azure, [MY PACKAGE] Successfully installed annotated-types-0.7.0 cachetools-5.5.2 ... opencensus-ext-logging-0.1.1 [MY PACKAGE]-[VERSION NO] ... rsa-4.9.1 typing-inspection-0.4.1 Is this possible to do with Poetry?
