|
Size: 4216
Comment: more pip dependencies
|
Size: 6056
Comment: TOC, explanations
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 9: | Line 9: |
| <<TableOfContents(3)>> | |
| Line 13: | Line 14: |
| We're partway through a next-generation rewrite of the dependency resolver within pip, Python's package download and installation tool. The project ran into massive technical debt, but the refactoring is nearly finished and prototype functionality is in alpha now. ([[https://docs.google.com/document/d/1x_VrNtXCup75qA3glDd2fQOB2TakldwjKZ6pXaAjAfg/edit|In-depth explanation by Sebastian Awwad of the problem & our approach]], [[https://gist.github.com/pradyunsg/5cf4a35b81f08b6432f280aba6f511eb|lead developer Pradyun Gedam's initial plan]] and [[https://pradyunsg.me/gsoc-2017/|2017 status updates]], [[https://github.com/pypa/pip/issues/988|GitHub issue #988 tracking progress]] and [[https://github.com/pypa/pip/issues/6536|issue #6536 for planning rollout]].) | |
| Line 14: | Line 16: |
| See [[https://github.com/pypa/pip/issues/988|pip issue #988]] and [[https://github.com/pypa/pip/issues/6536|#6536]]. This includes help with user experience, communications/publicity, and testing (including developing robust testing/CI infrastructure) as well as, potentially, core feature development and review. | Funding would support user experience, communications/publicity, and testing work (including developing robust testing/CI infrastructure) as well as core feature development and review. |
| Line 16: | Line 18: |
| We need to finish this because so many other improvements are blocked on it: | We need to finish the resolver because so many other improvements are blocked on it: |
| Line 23: | Line 25: |
| * [[https://github.com/pypa/packaging-problems/issues/215|recording requested and installed extras]] | |
| Line 24: | Line 27: |
| * [[https://mail.python.org/archives/list/distutils-sig@python.org/thread/2QECNWSHNEW7UBB24M2K5BISYJY7GMZF/#2QECNWSHNEW7UBB24M2K5BISYJY7GMZF|minimizing duplication of work between pip and pipenv]] * [[https://discuss.python.org/t/namespace-support-in-pypi/1609/35|package namespace support]] * [[https://discuss.python.org/t/if-python-started-moving-more-code-out-of-the-stdlib-and-into-pypi-packages-what-technical-mechanisms-could-packaging-use-to-ease-that-transition/1738/24|moving more code out of Python's standard library so we can release improvements faster]] |
|
| Line 35: | Line 41: |
| * [[https://github.com/ofek/hatch/issues/47|hatch, another packaging tool]] | |
| Line 39: | Line 46: |
| * [[https://github.com/pypa/pip/issues/4582|numpy and scipy]] | |
| Line 51: | Line 59: |
| We need funding to ensure core packaging tools work well with each other; currently they aren't seamlessly interoperable. See [[https://github.com/pypa/integration-test|the integration-test project]]. | We need funding to ensure core packaging tools work well with each other; currently they aren't seamlessly interoperable. See [[https://github.com/pypa/integration-test|the integration-test project]]. This will help us get faster at testing and rolling out bugfixes and features for '''all''' [[https://packaging.python.org/key_projects/|Python packaging and distribution tools]]: well-known projects like pip, virtualenv, and wheel, but also all the downstream projects that depend on them. |
| Line 55: | Line 63: |
| See [[PackagingWG/2019-03-22-Warehouse|notes from this meeting]]. To scale up our anti-abuse moderation, we need to be able to, for instance, mark a release as deprecated or a project as unsupported. We need support to do the architectural design to implement this. | To scale up our anti-abuse moderation and help package maintainers with security response, we need to be able to, for instance, mark a release as deprecated or a project as unsupported. We need support to do the architectural design to implement this. (See [[PackagingWG/2019-03-22-Warehouse|notes from this meeting]].) |
| Line 59: | Line 67: |
| It's difficult to roll out new features gradually to our test site or to selected test users; a feature flag system would help us deploy more confidently. See [[https://github.com/pypa/warehouse/issues/5869|this GitHub issue]]. | It's difficult to roll out new features gradually to PyPI's test site or to selected test users; a feature flag system would help us deploy more confidently. See [[https://github.com/pypa/warehouse/issues/5869|this GitHub issue]]. |
| Line 61: | Line 69: |
| == virtualenv rewrite == | === virtualenv rewrite === |
Packaging improvements that could be funded
This page lists specific things that
- the Python packaging community wants
- are fairly well-scoped
- would happen much faster if the Packaging Working Group got funding to achieve them
Contents
Projects
finish dependency resolver for pip
We're partway through a next-generation rewrite of the dependency resolver within pip, Python's package download and installation tool. The project ran into massive technical debt, but the refactoring is nearly finished and prototype functionality is in alpha now. (In-depth explanation by Sebastian Awwad of the problem & our approach, lead developer Pradyun Gedam's initial plan and 2017 status updates, GitHub issue #988 tracking progress and issue #6536 for planning rollout.)
Funding would support user experience, communications/publicity, and testing work (including developing robust testing/CI infrastructure) as well as core feature development and review.
We need to finish the resolver because so many other improvements are blocked on it:
warning when trying to download or build wheels from incompatible set of packages/requirements
making PyPI and pip enforce metadata compliance more strictly
warning the user when uninstalling a package that other packages depend on
option to show what versions of packages are currently available
moving more code out of Python's standard library so we can release improvements faster
and it would fix so many dependency issues for our users:
And in our larger ecology, this causes installation problems for:
Robust interoperability testing
We need funding to ensure core packaging tools work well with each other; currently they aren't seamlessly interoperable. See the integration-test project. This will help us get faster at testing and rolling out bugfixes and features for all Python packaging and distribution tools: well-known projects like pip, virtualenv, and wheel, but also all the downstream projects that depend on them.
System for generic statuses/flags on projects on PyPI
To scale up our anti-abuse moderation and help package maintainers with security response, we need to be able to, for instance, mark a release as deprecated or a project as unsupported. We need support to do the architectural design to implement this. (See notes from this meeting.)
Feature flag system on PyPI
It's difficult to roll out new features gradually to PyPI's test site or to selected test users; a feature flag system would help us deploy more confidently. See this GitHub issue.