Differences between revisions 11 and 39 (spanning 28 versions)
Revision 11 as of 2019-06-21 21:26:42
Size: 5681
Comment: pip explanation
Revision 39 as of 2020-06-05 22:32:24
Size: 808
Comment: moved to GitHub
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:
This page lists specific things that The [[https://github.com/psf/fundable-packaging-improvements|Fundable Packaging Improvements repository]] lists specific things that
Line 7: Line 7:
 3. would happen much faster if the Packaging Working Group got funding to achieve them  3. would happen much faster if the [[PackagingWG|Packaging Working Group]] got funding to achieve them (through [[http://donate.pypi.org/|donations]] or grants/directed gifts)
Line 9: Line 9:
That list [[https://wiki.python.org/psf/Fundable%20Packaging%20Improvements?action=info|was previously on this wiki page]] and has now been moved [[https://github.com/psf/fundable-packaging-improvements|to GitHub]].
Line 10: Line 11:
== Projects ==

=== finish dependency resolver for pip ===
We're partway through a next-generation rewrite of the dependency resolver within pip, Python's package download and installation tool. The project ran into massive technical debt, but the refactoring is nearly finished and prototype functionality is in alpha now. ([[https://docs.google.com/document/d/1x_VrNtXCup75qA3glDd2fQOB2TakldwjKZ6pXaAjAfg/edit|In-depth explanation by Sebastian Awwad of the problem & our approach]], [[https://gist.github.com/pradyunsg/5cf4a35b81f08b6432f280aba6f511eb|lead developer Pradyun Gedam's initial plan]] and [[https://pradyunsg.me/gsoc-2017/|2017 status updates]], [[https://github.com/pypa/pip/issues/988|GitHub issue #988 tracking progress]] and [[https://github.com/pypa/pip/issues/6536|issue #6536 for planning rollout]].)

Funding would support user experience, communications/publicity, and testing work (including developing robust testing/CI infrastructure) as well as core feature development and review.

We need to finish the resolver because so many other improvements are blocked on it:
 * [[https://github.com/pypa/pip/issues/4551|adding an "upgrade-all" command to pip]]
 * [[https://github.com/pypa/pip/issues/5497|warning when trying to download or build wheels from incompatible set of packages/requirements]]
 * [[https://github.com/pypa/pip/issues/4745|adding a no-implicit-upgrades strategy]]
 * [[https://github.com/pypa/packaging-problems/issues/264|making PyPI and pip enforce metadata compliance more strictly]]
 * [[https://github.com/pypa/pip/issues/4681|warning the user when uninstalling a package that other packages depend on]]
 * [[https://github.com/pypa/pip/issues/6495|properly respecting constraints]]
 * [[https://github.com/pypa/packaging-problems/issues/215|recording requested and installed extras]]
 * [[https://github.com/pypa/pip/issues/53|option to show what versions of packages are currently available]]
 * [[https://mail.python.org/archives/list/distutils-sig@python.org/thread/2QECNWSHNEW7UBB24M2K5BISYJY7GMZF/#2QECNWSHNEW7UBB24M2K5BISYJY7GMZF|minimizing duplication of work between pip and pipenv]]
 * [[https://discuss.python.org/t/namespace-support-in-pypi/1609/35|package namespace support]]
 * [[https://discuss.python.org/t/if-python-started-moving-more-code-out-of-the-stdlib-and-into-pypi-packages-what-technical-mechanisms-could-packaging-use-to-ease-that-transition/1738/24|moving more code out of Python's standard library so we can release improvements faster]]

and it would fix so many dependency issues for our users:
 * [[https://github.com/pypa/pip/issues/4907|Django installation conflict]]
 * [[https://github.com/pradyunsg/zazo/issues/2|cherrypy/six/cheroot installation conflict]]
 * [[https://github.com/pypa/pip/issues/5043|Spyder downgrade requirement]]
 * [[https://github.com/pradyunsg/zazo/issues/4|boto3/bravado dependency failure]]
 * [[https://github.com/pypa/pip/issues/5313|Ansible/PyOpenSSL/cryptography failure]]
 * [[https://github.com/pypa/pip/issues/4957|extras installation failure]]
 * [[https://github.com/pypa/pip/issues/4391|extras upgrade failure]]
 * [[https://github.com/pypa/pip/issues/6494|breaking installed packages]]
 * [[https://github.com/pradyunsg/zazo/issues/14|elasticsearch/requests failure]]
 * [[https://github.com/ofek/hatch/issues/47|hatch, another packaging tool]]

And in our larger ecology, this causes installation problems for:
 * [[https://github.com/conda/conda/issues/8657|conda's compatibility with pip]]
 * [[https://github.com/servo/servo/issues/10611|the Servo browser engine]]
 * [[https://github.com/pypa/pip/issues/4582|numpy and scipy]]
 * [[https://github.com/juju/python-libjuju/issues/45|Canonical's DevOps tool Juju]]
 * [[https://github.com/antocuni/capnpy/issues/16|a Cap'n Proto implementation]]
 * [[https://github.com/DataBiosphere/toil/issues/2230|toil, awscli, and boto3]]
 * [[https://github.com/mozilla/bedrock/issues/5967|the Mozilla website & icalendar]]
 * [[https://github.com/certbot/certbot/issues/5195|certbot, in the past and possibly the future]]
 * [[https://github.com/TurboGears/tg2devtools/issues/13|TurboGears]]
 * [[https://github.com/pycontribs/jira/pull/744|a JIRA API client library]]
 * [[https://github.com/crossbario/autobahn-testsuite/issues/55|a WebSocket protocol test suite]]
 * [[https://github.com/gerkey/ros1_external_use/issues/7|Robot Operating System tooling]]

=== Robust interoperability testing ===

We need funding to ensure core packaging tools work well with each other; currently they aren't seamlessly interoperable. See [[https://github.com/pypa/integration-test|the integration-test project]].

=== System for generic statuses/flags on projects on PyPI ===

See [[PackagingWG/2019-03-22-Warehouse|notes from this meeting]]. To scale up our anti-abuse moderation, we need to be able to, for instance, mark a release as deprecated or a project as unsupported. We need support to do the architectural design to implement this.

=== Feature flag system on PyPI ===

It's difficult to roll out new features gradually to our test site or to selected test users; a feature flag system would help us deploy more confidently. See [[https://github.com/pypa/warehouse/issues/5869|this GitHub issue]].

== virtualenv rewrite ==

See [[https://github.com/pypa/virtualenv/pull/697|virtualenv issue #697]].
Please contact the Packaging WG to ask us to estimate how much one of these improvements would cost; we'll get back to you within a few business days.

Packaging improvements that could be funded

The Fundable Packaging Improvements repository lists specific things that

  1. the Python packaging community wants
  2. are fairly well-scoped
  3. would happen much faster if the Packaging Working Group got funding to achieve them (through donations or grants/directed gifts)

That list was previously on this wiki page and has now been moved to GitHub.

Please contact the Packaging WG to ask us to estimate how much one of these improvements would cost; we'll get back to you within a few business days.

Fundable Packaging Improvements (last edited 2020-06-05 22:32:24 by SumanaHarihareswara)

Unable to view page? See the FrontPage for instructions.