Packaging Working Group
The Packaging Working Group is a volunteer work group of the Python Software Foundation.
Agenda
The purpose of this working group is to support the larger efforts of improving and maintaining the packaging ecosystem in Python through fundraising (including a sponsorship program) and disbursement of raised funds. It largely focuses on efforts such as PyPI, pip, packaging.python.org, setuptools, and cross-project efforts.
Resources
- Discussion: Slack and mailing list. The archives are set to private since there is voting.
Accounting: We rely on the PSF's donation and accounting mechanisms to raise funds and disburse them to the selected recipients.
Project ideas: Fundable packaging improvements
Governance
Decisions on what fundraising and projects/efforts to support will be done by a simple majority and in the case of a tie, will escalate to the PSF Board. See the PSF Packaging WG Charter.
Administration and Contact
Donald Stufft <donald@python.org> (chair)
Dustin Ingram <di@python.org> (co-chair)
Nicole Harris <n.harris@kabucreative.com> (co-chair)
To contact the Packaging WG, email <packaging-wg@python.org>.
Members
- Nick Coghlan
- Ee Durbin
- Thea Flowers
- Sumana Harihareswara
- Nathaniel J. Smith
- Tzu-ping Chung
- Jannis Leidel
- (others will be added as they accept their invitation to the WG)
Meetings
As needed.
Current Projects
Fundraising
The Packaging Working Group is seeking sponsorships and grants to raise funds for fundable packaging improvements.
Sprints
We run PackagingSprints at conventions and as standalone events. We're open to companies and organizations hosting sprints and work weeks to help us move packaging forward; get in contact with a Working Group member.
PyPI Malware Reporting and Response project
The PSF has received funding from the Center for Security and Emerging Technology (CSET) to develop and improve the infrastructure for malware reporting and response on PyPI.
Summary: Develop an API that allows malware reporting and define the criteria for automated consensus based takedown and soft-deletes of packages
Schedule: One year
Meetings and Updates: See below.
Meetings and status updates:
Meeting/update |
Type |
Date |
Meeting minutes |
June 02, 2023 |
|
Discourse Post |
June 21, 2023 |
|
PyPI Blog Post |
June 22, 2023 |
|
Meeting minutes |
July 03, 2023 |
|
Meeting minutes |
July 19, 2023 |
|
Meeting minutes |
July 19, 2023 |
|
Meeting minutes |
July 20, 2023 |
|
Meeting minutes |
July 20, 2023 |
|
Meeting minutes |
July 21, 2023 |
|
Meeting minutes |
July 25, 2023 |
|
Meeting minutes |
July 26, 2023 |
|
Meeting minutes |
July 27, 2023 |
Past projects
PyPI Organization Account
The Python Software Foundation, with the Packaging WG's approval, funded a project to deploy organization account features in PyPI.
Summary: Organization accounts in PyPI will allow organizations to create accounts, manage users, manage projects and set permission levels for a team.
Schedule: 16 weeks that commenced on April 1, 2022
High level Roadmap: PyPI Organization Account High-level Roadmap
Detailed Roadmap: PyPI Organization Account Detailed Roadmap
Code and discussion: GitHub repository for Warehouse, and Discourse forum
Project Board: PyPI Organization Account Project Board
Deployment: pypi.org.
Manager: Shamika Mohanan
Meetings and Updates: See below.
Meetings and status updates:
Meeting/update |
Type |
Date |
Meeting minutes |
April 1, 2022 |
|
Meeting minutes |
April 5, 2022 |
|
Discourse Post |
April 8, 2022 |
|
Meeting minutes |
April 12, 2022 |
|
Meeting minutes |
April 19, 2022 |
|
Meeting minutes |
April 26, 2022 |
|
Meeting minutes |
May 3, 2022 |
|
Meeting minutes |
May 10, 2022 |
|
Discourse Post |
May 10, 2022 |
|
Meeting minutes |
May 17, 2022 |
|
Meeting minutes |
May 24, 2022 |
|
Meeting minutes |
May 27, 2022 |
|
Meeting minutes |
May 31, 2022 |
|
Discourse Post |
May 31, 2022 |
|
Meeting minutes |
June 1, 2022 |
|
Meeting minutes |
June 3, 2022 |
|
Meeting minutes |
June 7, 2022 |
|
Meeting minutes |
June 14, 2022 |
|
Meeting minutes |
June 21, 2022 |
|
Meeting minutes |
June 28, 2022 |
|
Meeting minutes |
July 5, 2022 |
|
Meeting minutes |
June 6, 2022 |
|
Meeting minutes |
July 12, 2022 |
|
Meeting minutes |
July 19, 2022 |
|
Meeting minutes |
July 26, 2022 |
|
Meeting minutes |
Aug 02, 2022 |
|
Discourse Post |
Aug 03, 2022 |
|
Meeting minutes |
Aug 05, 2022 |
|
Meeting minutes |
Aug 05, 2022 |
|
Meeting minutes |
Aug 08, 2022 |
|
Meeting minutes |
Aug 08, 2022 |
|
Meeting minutes |
Aug 08, 2022 |
|
Meeting minutes |
Aug 09, 2022 |
|
Meeting minutes |
Aug 16, 2022 |
|
Meeting minutes |
Sep 06, 2022 |
|
Meeting minutes |
Sep 13, 2022 |
|
Meeting minutes |
Sep 27, 2022 |
|
Meeting minutes |
Oct 4, 2022 |
|
Meeting minutes |
Oct 11, 2022 |
|
Meeting minutes |
Oct 14, 2022 |
|
Meeting minutes |
Oct 20, 2022 |
|
Meeting minutes |
Oct 20, 2022 |
|
Meeting minutes |
Oct 21, 2022 |
|
Meeting minutes |
Oct 21, 2022 |
|
Meeting minutes |
Oct 24, 2022 |
|
Meeting minutes |
Oct 25, 2022 |
|
Meeting minutes |
Oct 27, 2022 |
|
Meeting minutes |
Nov 09, 2022 |
Warehouse: Facebook gift
The Packaging Working Group applied for and is receiving a gift from Facebook to implement & deploy security features for Warehouse (PyPI's codebase).
Summary: Cryptographic signing of artifacts, and malware detection. See announcement blog post, and the milestone description on GitHub.
Schedule: As of 2 January 2020, the PSF has hired contractors to carry out this work, and has commenced work.
Roadmap: WarehouseRoadmap
Code and discussion: GitHub repository for Warehouse, Zulip livechat, and Discourse forum.
Deployment: pypi.org.
Testing: To be determined
Manager: Ee Durbin
Meetings and Updates: See below.
Meetings and status updates:
Meeting/update |
Type |
Date |
Python Package Index - Python Software Foundation's TUF key generation and signing ceremonies |
Live video stream of ceremony |
October 30th, 2020 |
Meeting notes |
June 20th, 2019 |
|
Blog post |
August 28th, 2019 |
|
Blog post |
September 25th, 2019 |
|
Kickoff - 2019 Q4 RFP Milestone 2 - Automated Detection of Malicious Uploads |
Meeting notes |
December 11th, 2019 |
Blog post |
February 3rd, 2020 |
|
Blog post |
March 4th, 2020 |
|
Online talk |
March 14th, 2020 |
Dependency resolver and user experience improvements for pip
The Packaging Working Group applied for and is receiving funding to work in 2020 on the design, implementation, and rollout of pip's next-generation dependency resolver. The donors funding this work are the Chan Zuckerberg Initiative (USD$200,000) and Mozilla Open Source Support (USD$207,000).
Summary: Complete pip's next-generation dependency resolver, and do user experience research and design to improve pip's usability and debuggability
Schedule: The PSF chose contractors to carry out this work in late 2019/early 2020, and commenced work in early 2020. In July 2020 the team delivered pip 20.2, which includes a beta of the new resolver. The team shipped the new resolver as default in pip 20.3, in November 2020. The work will end in December 2020/early January 2021.
Roadmap: Pip2020DonorFundedRoadmap
Code and discussion: GitHub repository for pip, Zulip livechat, and Discourse forum.
Testing: A mix of automated testing and a series of general public beta periods.
Manager: Sumana Harihareswara
Meetings and Updates: See below.
Meetings and status updates:
Meeting/update |
Type |
Date |
Meeting notes |
December 23rd, 2020 |
|
Meeting notes |
December 22nd, 2020 |
|
Meeting notes |
December 16th, 2020 |
|
Meeting notes |
December 15th, 2020 |
|
Presentation |
December 10th, 2020 |
|
Meeting notes |
December 9th, 2020 |
|
Meeting notes |
December 8th, 2020 |
|
Meeting notes |
December 2nd, 2020 |
|
Meeting notes |
December 1st, 2020 |
|
Meeting notes |
November 25th, 2020 |
|
Forum post |
November 30th, 2020 |
|
Blog post |
November 30th, 2020 |
|
Meeting notes |
November 24th, 2020 |
|
Meeting notes |
November 23rd, 2020 |
|
Meeting notes |
November 18th, 2020 |
|
Meeting notes |
November 17th, 2020 |
|
Meeting notes |
November 11th, 2020 |
|
Meeting notes |
November 10th, 2020 |
|
Meeting notes |
November 2nd, 2020 |
|
Meeting notes |
October 28th, 2020 |
|
Meeting notes |
October 27th, 2020 |
|
Meeting notes |
October 21st, 2020 |
|
Meeting notes |
October 20th, 2020 |
|
Meeting notes |
October 12th, 2020 |
|
Meeting notes |
October 7th, 2020 |
|
Meeting notes |
October 6th, 2020 |
|
Podcast interview |
October 2nd, 2020 |
|
Meeting notes |
September 30th, 2020 |
|
Software Developers Journey Podcast interview with Sumana Harihareswara |
Podcast interview |
September 29th, 2020 |
Video on YouTube |
September 29th, 2020 |
|
Meeting notes |
September 29th, 2020 |
|
Meeting notes |
September 23rd, 2020 |
|
Meeting notes |
September 16th, 2020 |
|
Blog post |
September 15th, 2020 |
|
Meeting notes |
September 9th, 2020 |
|
Meeting notes |
September 8th, 2020 |
|
Podcast interview |
September 2nd, 2020 |
|
Meeting notes |
September 2nd, 2020 |
|
Meeting notes |
August 26th, 2020 |
|
Meeting notes |
August 19th, 2020 |
|
Meeting notes |
August 12th, 2020 |
|
Meeting notes |
August 5th, 2020 |
|
Podcast interview |
August 3rd, 2020 |
|
Blog post |
July 30th, 2020 |
|
Meeting notes |
July 29th, 2020 |
|
Meeting notes |
July 22nd, 2020 |
|
Meeting notes |
July 15th, 2020 |
|
Blog post |
July 13th, 2020 |
|
Meeting notes |
July 8th, 2020 |
|
Meeting notes |
July 1st, 2020 |
|
Meeting notes |
June 24th, 2020 |
|
Meeting notes |
June 17th, 2020 |
|
Meeting notes |
June 10th, 2020 |
|
Meeting notes |
June 3rd, 2020 |
|
Meeting notes |
May 27th, 2020 |
|
Podcast.__init__ episode "Dependency Management Improvements In Pip's Resolver - Episode 264" |
Podcast interview |
May 25th, 2020 |
Meeting notes |
May 20th, 2020 |
|
Meeting notes |
May 20th, 2020 |
|
Meeting notes |
May 18th, 2020 |
|
Meeting notes |
May 14th, 2020 |
|
Meeting notes |
May 13th, 2020 |
|
Meeting notes |
May 11th, 2020 |
|
Meeting notes |
May 7th, 2020 |
|
Developer team syncup, and pip UX/resolver collaboration notes |
Meeting notes |
May 5th-7th, 2020 |
Blog post |
April 30th, 2020 |
|
Meeting notes |
April 30th, 2020 |
|
Meeting notes |
April 29th, 2020 |
|
Meeting notes |
April 28th, 2020 |
|
Meeting notes |
April 23rd, 2020 |
|
Meeting notes |
April 22nd, 2020 |
|
Forum post |
April 20th, 2020 |
|
Meeting notes |
April 16th, 2020 |
|
Meeting notes |
April 14th, 2020 |
|
Meeting notes |
April 9th, 2020 |
|
Meeting notes |
April 8th, 2020 |
|
Meeting notes |
April 4th, 2020 |
|
Meeting notes |
April 2nd, 2020 |
|
Meeting notes |
March 28th, 2020 |
|
Meeting notes |
March 27th, 2020 |
|
Meeting notes |
March 26th, 2020 |
|
Blog post |
March 23rd, 2020 |
|
Meeting notes |
March 19th, 2020 |
|
Meeting notes |
March 17th, 2020 |
|
Meeting notes |
March 12th, 2020 |
|
Meeting notes |
March 10th, 2020 |
|
Blog post |
March 5th, 2020 |
|
Meeting notes |
March 5th, 2020 |
|
Meeting notes |
March 3rd, 2020 |
|
Meeting notes |
February 27th, 2020 |
|
Presentation "Python Packaging and Science" from CZI convening |
Presentation |
February 25th, 2020 |
Meeting notes |
February 20th, 2020 |
|
Meeting notes |
February 19th, 2020 |
|
Meeting notes |
February 17th, 2020 |
|
Meeting notes |
February 13th, 2020 |
|
Meeting notes |
February 7th, 2020 |
|
Meeting notes |
February 6th, 2020 |
|
Meeting notes |
January 29th, 2020 |
|
Meeting notes |
January 23rd, 2020 |
|
Meeting notes |
January 8th, 2020 |
|
Blog post |
December 4th, 2019 |
|
Blog post |
November 11th, 2019 |
Warehouse: OTF grant
The Packaging Working Group applied for and received a performance-based contract (like a grant) from the Open Technology Fund to implement & deploy security, localization, and accessibility improvements for Warehouse (PyPI's codebase).
Summary: See March 13 2019 blog post.
Roadmap: On Read the Docs.
Schedule: Several contractors worked, paid by PSF using the OTF funds, from March 2019 till October 2019. As of 8 October 2019, OTF-funded contractors have finished security improvements, accessibility and internationalization/localization improvements to Warehouse, and volunteers are working on Milestone 6, "Post Legacy Shutdown".
Code: GitHub repository.
Deployment: pypi.org.
Testing: WarehousePackageMaintainerTesting
Manager: Sumana Harihareswara
Meetings and Updates: See below.
Meetings and status updates from the OTF grant-funded project:
Meeting/update |
Type |
Date |
Blog post |
January 17th, 2020 |
|
Forum post |
January 17th, 2020 |
|
Forum post |
October 8th, 2019 |
|
Forum post |
September 8th, 2019 |
|
Forum post |
September 8th, 2019 |
|
Podcast interview |
September 4th, 2019 |
|
Meeting notes |
August 29th, 2019 |
|
Podcast.__init__ Episode 225: Security, UX, and Sustainability For The Python Package Index |
Podcast interview |
August 19th, 2019 |
Inspect PyPI event logs to audit your account's and project's security |
Blog post |
August 15th, 2019 |
Early Aug. update on 2FA, API keys, audit log, & a11y work |
Forum post |
August 6th, 2019 |
Meeting notes |
August 2nd, 2019 |
|
Meeting notes |
July 31st, 2019 |
|
Blog post |
July 31st, 2019 |
|
Forum post |
July 25th, 2019 |
|
Forum post |
July 17th, 2019 |
|
Forum post |
July 3rd, 2019 |
|
Meeting notes |
June 24th, 2019 |
|
Blog post |
June 18th, 2019 |
|
Forum post |
June 8th, 2019 |
|
Meeting notes |
June 7th, 2019 |
|
Blog post |
May 30th, 2019 |
|
Forum post |
May 22nd, 2019 |
|
Mailing list post |
May 2nd, 2019 |
|
Forum post |
May 2nd, 2019 |
|
Forum post |
April 3rd, 2019 |
|
PyPI security work: multifactor auth progress & help needed |
Forum post |
March 22nd, 2019 |
Meeting notes |
March 22nd, 2019 |
|
Commencing security, a11y, & i18n improvements to PyPI for 2019 |
Blog post |
March 13th, 2019 |
Meeting notes |
March 11th, 2019 |
|
Blog post |
December 20th, 2018 |
|
PyPI Security and Accessibility Q1 2019 Request for Proposals Update |
Blog post |
December 19th, 2018 |
PyPI Security and Accessibility Q1 2019 Request for Proposals period opens |
Blog post |
November 19th, 2018 |
PyPI Security and Accessibility Q1 2019 Request for Information period opens |
Blog post |
October 30th, 2018 |
Warehouse rollout
The Packaging Working Group supported the implementation & deployment of Warehouse (PyPI 2.0) to replace the legacy code base that powered legacy PyPI. Announced on PSF blog in January 2016; see its history in this April 2018 LWN article.
Summary: PSF blog post about the MOSS grant.
Roadmap: WarehouseRoadmap. As of 30 April 2018, the Warehouse team has shut down the legacy PyPI installation, and -- on a volunteer basis -- is working on Milestone 6, "Post Legacy Shutdown".
Code: GitHub repository.
Deployment: pypi.org.
Testing: See the PSF blog post about testing for the beta. (Previously: WarehousePackageMaintainerTesting, PSF blog post about testing package maintainer functionality.)
Manager: Sumana Harihareswara
Meetings and Updates: See below.
Meetings and status updates from the MOSS-funded project:
Meeting/update |
Type |
Date |
PSF announcement of $170,000 MOSS award to improve sustainability of PyPI |
Blog post |
November 27, 2017 |
Meeting notes |
Monday, Dec. 4, 2017 |
|
Mailing list post |
Thursday, Dec. 7, 2017 |
|
Developer experience audit walkthrough |
in-person meeting |
Tuesday, December 12, 2017 |
Meeting notes |
Tuesday, Dec. 19, 2017 |
|
Meeting notes |
January 10, 2018 |
|
Mailing list post |
Jan. 15, 2018 |
|
Mailing list post |
Jan. 23, 2018 |
|
Meeting notes |
January 29, 2018 |
|
Mailing list post |
Jan. 30, 2018 |
|
Meeting notes |
February 5, 2018 |
|
Mailing list post |
Feb. 6, 2018 |
|
Meeting notes |
Feb. 12th, 2018 |
|
Warehouse: package manager features & question about advertising |
Mailing list post |
Feb. 13, 2018 |
Standup, bug triage, & milestone schedule update meeting |
Meeting notes |
Feb. 20th, 2018 |
Mailing list post |
Feb. 21st, 2018 |
|
Meeting notes |
Feb. 26th, 2018 |
|
Blog post |
Feb. 26th, 2018 |
|
Warehouse update: a week of testing, polish, & infrastructure |
Mailing list post |
Feb. 27th, 2018 |
Meeting notes |
March 6th, 2018 |
|
PyPI & Warehouse update: redirecting & shutting down legacy by end of April |
Mailing list post |
March 7th, 2018 |
Meeting notes |
March 12th, 2018 |
|
new stuff overview, beta next week, user tests, & other Warehouse updates |
Mailing list post |
March 14th, 2018 |
Meeting notes |
March 19th, 2018 |
|
PyPI/Warehouse: infrastructure hardening & the CAPTCHA conundrum |
Mailing list post |
March 20th, 2018 |
Meeting notes |
March 20th, 2018 |
|
Blog post |
March 26th, 2018 |
|
Blog post |
March 26th, 2018 |
|
Mailing list post |
March 28th, 2018 |
|
Meeting notes |
April 2nd, 2018 |
|
PyPI/Warehouse update: new advice & launch, shutdown dates |
Mailing list post |
April 3rd, 2018 |
Meeting notes |
April 10th, 2018 |
|
PyPI/Warehouse (short) weekly report: Progress towards launch milestone |
Mailing list post |
April 10th, 2018 |
Mailing list post |
April 11th, 2018 |
|
Statuspage report |
April 16th, 2018 |
|
Blog post |
April 16th, 2018 |
|
Meeting notes |
April 17th, 2018 |
|
Mailing list post |
April 18th, 2018 |
|
Meeting notes |
April 23rd, 2018 |
|
PyPI update: legacy shutdown 30 April, new classifiers page, seeking funding |
Mailing list post |
April 24th, 2018 |
Podcast interview |
April 27, 2018 |
|
Statuspage report |
April 30th, 2018 |
|
Mailing list post |
April 30th, 2018 |
|
Meeting notes |
April 30th, 2018 |
|
Mailing list post |
May 1, 2018 |
|
Podcast interview |
May 2, 2018 |