Fix upload metadata #1576
Fix upload metadata #1576
Conversation
| We may want to pull in the unit tests from |
pganssle force-pushed the fix_upload_metadata branch from 8831858 to db8cdc6 Compare pganssle force-pushed the fix_upload_metadata branch from 6160ab7 to ae881ef Compare pganssle force-pushed the fix_upload_metadata branch from ae881ef to caa75d3 Compare pganssle force-pushed the fix_upload_metadata branch from caa75d3 to 1bff011 Compare | @jaraco @benoit-pierre This is mostly ready for review. I think I may need a few more tests, I didn't realize that we had the diff coverage required configured to be 81% or something and not 100%. That feels like a mistake, so I'll probably add a few more tests to get the diff coverage to 100% (or at least closer to it), but if y'all can give some feedback on what I have so far I'd appreciate it. The biggest change (which is why I don't feel comfortable just unilaterally merging this) is that we're now monkey-patching more of distutils. We can avoid that in this issue, but I think probably it will be hard to avoid if we also want to solve things like #1578. |
pganssle force-pushed the fix_upload_metadata branch from 1bff011 to c39491d Compare pganssle force-pushed the fix_upload_metadata branch from 7e86b7c to c6d5d8e Compare | OK, so lesson learned about not having 100% diff coverage in a situation like this. Almost everything that I originally thought "maybe I should throw that under a I think I have 100% coverage on the diff now, though ( |
This is the baseline, unchanged from the version in distutils.dist, to be modified before patching.
This turns get_metadata_version into a method on DistributionMetadata, populated either by inferrence (in the case of package metadata specified in `setup`) or from the data in a specified PKG-INFO file. To populate metadata_version from PKG-INFO, we need to monkey patch read_pkg_file in addition to write_pkg_file.
Previously this value was hard-coded to '1.0', which was inaccurate for many packages. Fixes pypa#1381
This creates a wrapper function for writing fields in the PKG-INFO file, both to simplify the syntax and to add a point where we can inject an encoding function in order to support Python 2.7 compatibility.
Rather than writing to a file in a temporary directory, we can write to and read from an in-memory buffer, now that the encoding functionality in write_pkg_file is fixed.
This is a fixture to create an upload command with a patched version of urlopen so that no HTTP queries are sent.
`test_upload_metadata` was written before the fixture, so this updates the test to use the fixture.
This fixes an issue where `distutils.spawn.spawn` was not available in the ported upload_file, which is only used when signing the data. This also adds a test that the gpg signature command is invoked and included in the uploaded data.
This fixes uploads when bdist_rpm or bdist_dumb are the command, both of which insert a comment about what platform they are built for.
The `upload.show_response` feature was not added until Python 3. Rather than backport it, it is now enabled only if supported. This also adds a "smoke test" for the feature.
pganssle force-pushed the fix_upload_metadata branch from c6d5d8e to fe2c9e4 Compare 
| I'd like to get this one in for the next release, but I'm OK with pushing it off to the release after that. I'd like to cut a release soon, so @jaraco @benoit-pierre if you have no objections to merging this let me know preferably in the next few days. |
jaraco left a comment
jaraco left a comment There was a problem hiding this comment.
I'm okay with this. The approach seems sound. I'm a little uneasy about improving support for a deprecated command, but since you've put in the effort, I see no reason not to roll it out.
| @jaraco To clarify, the reason why I wanted to improve the deprecated command is because it's currently polluting PyPI with bad metadata, and I think that when |
This comment is not used anywhere and `platform.dist()` is deprecated. See CPython PR #10414: python/cpython#10414 and bpo-35186: https://bugs.python.org/issue35186
| I've also ported python/cpython#10414 in to this branch, because |
3 participants
Summary of changes
This PR adds
get_metadata_versiontodistutils.dist.DistributionMetadataand also monkey patchesdistutils.dist.DistributionMetadata.read_pkg_fileto populate themetadata_versionattribute. We also have overriddendistutils.command.upload.upload_fileso that the metadata version is retrieved fromget_metadata_versionrather than hard-coded as 1.0.I felt that it would be preferable for
read_pkg_fileto actually store the specified metadata version rather than to infer it from the fields present (as is done when writing the metadata file), however, I think the bug I'm concerned about would still be fixed if we always hadget_metadata_versioninfer the version from the keys present.Closes #1381
Pull Request Checklist