Differences between revisions 12 and 39 (spanning 27 versions)
Revision 12 as of 2019-06-22 18:13:31
Size: 6056
Comment: TOC, explanations
Revision 39 as of 2020-06-05 22:32:24
Size: 808
Comment: moved to GitHub
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:
This page lists specific things that The [[https://github.com/psf/fundable-packaging-improvements|Fundable Packaging Improvements repository]] lists specific things that
Line 7: Line 7:
 3. would happen much faster if the Packaging Working Group got funding to achieve them  3. would happen much faster if the [[PackagingWG|Packaging Working Group]] got funding to achieve them (through [[http://donate.pypi.org/|donations]] or grants/directed gifts)
Line 9: Line 9:
<<TableOfContents(3)>> That list [[https://wiki.python.org/psf/Fundable%20Packaging%20Improvements?action=info|was previously on this wiki page]] and has now been moved [[https://github.com/psf/fundable-packaging-improvements|to GitHub]].
Line 11: Line 11:
== Projects ==

=== finish dependency resolver for pip ===
We're partway through a next-generation rewrite of the dependency resolver within pip, Python's package download and installation tool. The project ran into massive technical debt, but the refactoring is nearly finished and prototype functionality is in alpha now. ([[https://docs.google.com/document/d/1x_VrNtXCup75qA3glDd2fQOB2TakldwjKZ6pXaAjAfg/edit|In-depth explanation by Sebastian Awwad of the problem & our approach]], [[https://gist.github.com/pradyunsg/5cf4a35b81f08b6432f280aba6f511eb|lead developer Pradyun Gedam's initial plan]] and [[https://pradyunsg.me/gsoc-2017/|2017 status updates]], [[https://github.com/pypa/pip/issues/988|GitHub issue #988 tracking progress]] and [[https://github.com/pypa/pip/issues/6536|issue #6536 for planning rollout]].)

Funding would support user experience, communications/publicity, and testing work (including developing robust testing/CI infrastructure) as well as core feature development and review.

We need to finish the resolver because so many other improvements are blocked on it:
 * [[https://github.com/pypa/pip/issues/4551|adding an "upgrade-all" command to pip]]
 * [[https://github.com/pypa/pip/issues/5497|warning when trying to download or build wheels from incompatible set of packages/requirements]]
 * [[https://github.com/pypa/pip/issues/4745|adding a no-implicit-upgrades strategy]]
 * [[https://github.com/pypa/packaging-problems/issues/264|making PyPI and pip enforce metadata compliance more strictly]]
 * [[https://github.com/pypa/pip/issues/4681|warning the user when uninstalling a package that other packages depend on]]
 * [[https://github.com/pypa/pip/issues/6495|properly respecting constraints]]
 * [[https://github.com/pypa/packaging-problems/issues/215|recording requested and installed extras]]
 * [[https://github.com/pypa/pip/issues/53|option to show what versions of packages are currently available]]
 * [[https://mail.python.org/archives/list/distutils-sig@python.org/thread/2QECNWSHNEW7UBB24M2K5BISYJY7GMZF/#2QECNWSHNEW7UBB24M2K5BISYJY7GMZF|minimizing duplication of work between pip and pipenv]]
 * [[https://discuss.python.org/t/namespace-support-in-pypi/1609/35|package namespace support]]
 * [[https://discuss.python.org/t/if-python-started-moving-more-code-out-of-the-stdlib-and-into-pypi-packages-what-technical-mechanisms-could-packaging-use-to-ease-that-transition/1738/24|moving more code out of Python's standard library so we can release improvements faster]]

and it would fix so many dependency issues for our users:
 * [[https://github.com/pypa/pip/issues/4907|Django installation conflict]]
 * [[https://github.com/pradyunsg/zazo/issues/2|cherrypy/six/cheroot installation conflict]]
 * [[https://github.com/pypa/pip/issues/5043|Spyder downgrade requirement]]
 * [[https://github.com/pradyunsg/zazo/issues/4|boto3/bravado dependency failure]]
 * [[https://github.com/pypa/pip/issues/5313|Ansible/PyOpenSSL/cryptography failure]]
 * [[https://github.com/pypa/pip/issues/4957|extras installation failure]]
 * [[https://github.com/pypa/pip/issues/4391|extras upgrade failure]]
 * [[https://github.com/pypa/pip/issues/6494|breaking installed packages]]
 * [[https://github.com/pradyunsg/zazo/issues/14|elasticsearch/requests failure]]
 * [[https://github.com/ofek/hatch/issues/47|hatch, another packaging tool]]

And in our larger ecology, this causes installation problems for:
 * [[https://github.com/conda/conda/issues/8657|conda's compatibility with pip]]
 * [[https://github.com/servo/servo/issues/10611|the Servo browser engine]]
 * [[https://github.com/pypa/pip/issues/4582|numpy and scipy]]
 * [[https://github.com/juju/python-libjuju/issues/45|Canonical's DevOps tool Juju]]
 * [[https://github.com/antocuni/capnpy/issues/16|a Cap'n Proto implementation]]
 * [[https://github.com/DataBiosphere/toil/issues/2230|toil, awscli, and boto3]]
 * [[https://github.com/mozilla/bedrock/issues/5967|the Mozilla website & icalendar]]
 * [[https://github.com/certbot/certbot/issues/5195|certbot, in the past and possibly the future]]
 * [[https://github.com/TurboGears/tg2devtools/issues/13|TurboGears]]
 * [[https://github.com/pycontribs/jira/pull/744|a JIRA API client library]]
 * [[https://github.com/crossbario/autobahn-testsuite/issues/55|a WebSocket protocol test suite]]
 * [[https://github.com/gerkey/ros1_external_use/issues/7|Robot Operating System tooling]]

=== Robust interoperability testing ===

We need funding to ensure core packaging tools work well with each other; currently they aren't seamlessly interoperable. See [[https://github.com/pypa/integration-test|the integration-test project]]. This will help us get faster at testing and rolling out bugfixes and features for '''all''' [[https://packaging.python.org/key_projects/|Python packaging and distribution tools]]: well-known projects like pip, virtualenv, and wheel, but also all the downstream projects that depend on them.

=== System for generic statuses/flags on projects on PyPI ===

To scale up our anti-abuse moderation and help package maintainers with security response, we need to be able to, for instance, mark a release as deprecated or a project as unsupported. We need support to do the architectural design to implement this. (See [[PackagingWG/2019-03-22-Warehouse|notes from this meeting]].)

=== Feature flag system on PyPI ===

It's difficult to roll out new features gradually to PyPI's test site or to selected test users; a feature flag system would help us deploy more confidently. See [[https://github.com/pypa/warehouse/issues/5869|this GitHub issue]].

=== virtualenv rewrite ===

See [[https://github.com/pypa/virtualenv/pull/697|virtualenv issue #697]].
Please contact the Packaging WG to ask us to estimate how much one of these improvements would cost; we'll get back to you within a few business days.

Packaging improvements that could be funded

The Fundable Packaging Improvements repository lists specific things that

  1. the Python packaging community wants
  2. are fairly well-scoped
  3. would happen much faster if the Packaging Working Group got funding to achieve them (through donations or grants/directed gifts)

That list was previously on this wiki page and has now been moved to GitHub.

Please contact the Packaging WG to ask us to estimate how much one of these improvements would cost; we'll get back to you within a few business days.

Fundable Packaging Improvements (last edited 2020-06-05 22:32:24 by SumanaHarihareswara)

Unable to view page? See the FrontPage for instructions.