pip test infrastructure planning

23 January 2020

Participants: Pradyun, Sumana, Ernest

Agenda

Today: mostly exploratory/explanatory about what is running now

Then: look into what the long-term goals are

So:

  1. Pipelines
  2. Pain points
  3. Goals

Pipelines

There are numerous different pipelines

Documented already: https://pip.pypa.io/en/latest/development/ci/

Pipelines:

Pradyun: We use Azure, GitHub, and Travis as our 3 CI providers -- see matrix at https://pip.pypa.io/en/latest/development/ci/

Pain points

Wait time

Pain point # 1: reduce time between a pull request submission and GitHub reporting the success:

Ernest: pretty much every project with a big enough matrix runs into that

Inconsistent test environment

Pain point # 2:

Sumana: other projects have surely resolved this somehow?

Pradyun: Django has their own CI on Jenkins with a pretty big matrix

Sumana: is it ridiculous for PSF to run our own Jenkins?

Ernest: Linux, Mac, and Windows targeting -- PSF Infra doesn't have any that is on Mac and Windows.

So this is why most project end up relying on Travis/Azure/GitHub Actions

Similar to the wheel-building block.

We don't have the capacity to run the farm necessary

Thus, PAIN POINT #2 is outside the scope of this project -- future fundraising opportunity

Automation snags and permissions

Pain point #3:

Ernest: PyPA bot and BrownTruck bot on Heroku

the 2 projects have GitHub users that Donald has pw for

Goals

A bunch of Warehouse improvements come to mind, such as:

Do any of these Warehouse improvements seem to be on the horizon?

Ernest: Yanking is closest to implementable. There's a PEP & a draft PR from Donald.

2-step is next closest.

Automated test of installability -- much further away (complexity)

How do we hit point #1?

Pradyun's idea: Speeding up the test suite https://github.com/pypa/pip/issues/4497

(Pradyun is considering putting this up as a GSoC project -- test suite stuff -- working with PSF GSoC Admins)

Sumana: Could we ask Azure and Travis for more slots?

Ernest: Azure more likely. Travis: hard to get in contact with, especially since acquisition

Ernest: we can ASK.

What the current parallel jobs count is for Azure pipeline:

PyPA has a single pool for a bunch of projects.

(They're a sponsor via in-kind donation)

Conclusions

CONCLUSIONS

PackagingWG/2020-01-23-pip (last edited 2020-01-27 17:19:58 by SumanaHarihareswara)

Unable to view page? See the FrontPage for instructions.