UX-developer collab meeting on 18 May

Nicole, Bernard, and Pradyun

Discussion:

• Resolution impossible & resolution too deep - Bernard

  • What situations lead to that failure?
  • UX has reached out to projects with complex dependencies
    • give us examples of when pip fails?
  • Bernard has put together a blog post to try to maybe get responses from wider community
    • But -- pip doesn't know what exception occurs.... and there are 250K Python packages in existence on PyPI. We're trying to gather quantitative data using a qualitative method. What needs to happen (bluesky):
      • pip should be able to know what packages cause resolution impossible exceptions
      • so it can inform itself of what needs to happen (e.g., "a message needs to be displayed to usee, here what's gone wrong, here is how to fix it")
      • but right now pip doesn't know
    • so either:
      • pip needs to have that ability (for package A, "you tried, but it depends on B, B depends on C v1, and D depends on C v2, so impossible")
      • OR, user needs to provide info to us in some automated or semiautomated way. And this needs to be of course consensual and informed.
      • Bernard/Nicole want to do some research about the community's reaction re: opt-in automated collection of data:
      • Discussed with Pradyun, he said:
        • How is this different from what PyPI downloads tells us?
        • It would be non trivial to implement this
        • Is asking the user for information when there is a failure better than automated data collection?

    • this is based on discussions with an expert in Yocto, plus looking at Debian's popularity contest (popcon) & Homebrew

  • Bernard and Nicole happy to talk about this now
• Nicole would like to discuss collecting all of the output that pip currently prints, UI messages
• Nicole would like to run Pradyun through -- Nicole has tried out poetry, to see how it runs, and has a few questions about results she got. (May be in a smaller group)
• Pradyun is interested in prioritizing for the week
• Pradyun has a bugfix to release today (pip 20.1.1)

• Nicole requested logging information - Pradyun is going to share all of the ways that pip outputs information to the user via a gist- https://gist.github.com/pradyunsg/d76a2c4dfe3147b4ebc3aed9b1ce785e

  • We need to identify which messages are shown often
  • We need to identify which messages are repeated often and take up a lot of real estate on the terminal (e.g. looped messages)
  • We need to identify where we *could* provide/expose additional information to the user

• Nicole asked Pradyun why poetry was unable to resolve a dependency conflict that pip can resolve (see https://gist.github.com/nlhkabu/dd09051652a277eaec1d6f599d0e05f1). This is probably due to poetry using pubgrub and throwing information away too early.

  • In this case, pip is doing a better job

  • Nicole asked for an example of where poetry *can* resolve dependency conflicts - there is an example in the poetry readme: https://github.com/python-poetry/poetry#dependency-resolution

Nicole noted that poetry has some nice features regarding printing dependency trees, and showing up to date and out of date packages. If we wanted similar features, how much work would that be? Pradyun: about a week. Not a huge development workload.

Bernard is having trouble finding failing package combinations for pipenv. Pradyun has an issue tracker for this: https://github.com/pradyunsg/zazo/issues \o/

Auditing pip commands in order to know

• useful to document "this is confusing behaviour"
• Pradyun's confusion is how would we do it
• Pradyun is trying to work out how that would happen
• it could be bash scripts: not sure how to map the expected behaviour

Is it worth doing?

• anything touching --upgrade is worth looking at

• pip list

• pip show

these need "improvements" poorly designed things that you don't expect to happen. It might be worth

• Spending time looking into these is useful. Paul has done this. Maybe do KT about this.
• Pradyun is grouping broader issues into one umbrella
  • We can fix this one thing or make broader fixes that fix more things

Double-edged sword

how do you test this?

man file

• contains what the command would do

how would we test the commands?

• write a test script
• see if it has passed the test case

Why not using pubgrub: https://python.zulipchat.com/#narrow/stream/218659-pip-development/topic/PubGrub.20question/near/197679316

pubgrub can't do <something about dependency>. There are more things you can do with python deps that are not possible with pubgrub.

• If we can't use it - can as still get the information we were going to get from pubgrub? Yes there is, it's a matter of someone writing it (and being mentally tortured!). pip's requirements are more than what pubgrub can do.