Wednesday 19th August Participants: * Georgia (not available, but left notes in advance) * Bernard * Pradyun * Sumana * Nicole Agenda: * Nicole: * Can we pay for https://matomo.org/ for 3 months (for the pip docs)? * How much money? * 100 euro or USD for 3 months * Looks like 49 EUR/month for ~1,000,000 views/month (pip sees around 300,000 views) * GA Suggestion/Note: PSF uses Google analytics on other platforms, so maybe we can use Google Analytics on the docs as well and connect with a different property-id to the same account? Matomo is better from a privacy perspective, but might be easier to just use tools PSF is using elsewhere. * Pradyun's notes: * pip's docs don't have any analytics beyond what is provided by RTD today. * we could opt into Read The Docs's beta for analytics -- https://pasteboard.co/JmYdTbP.png shows you a screenshot of the current analytics from Read the Docs (without the beta feature) * Conclusion: * TODO: Pradyun to opt in to RTD beta. Nicole to ask Jan if this is enough. If not, we will fall back to Matomo. * FYI, we are going to add an email link to the pip docs for collecting user feedback. See # Ernest - can we have a PYPA email for this? * GA: We have `pip-research@simplysecure.org` or we could make a group on pypa.io, e.g. `ux@pypa.io` or `.org` ? * TODO: Sumana to ping Ernest about this * To discuss - collecting feedback via UI (in documentation) * Forthcoming GitHub issue. * Would be nice to be able to collect data from users using documentation. * Iteration 1: an email link for "was this helpful to you". * Iteration 2: a widget so they can submit while reading docs. Can we get funded dev time for this? probably no. Can we ID someone in community who can work on this with us? * TODO: Nicole to follow up with Eric about this * https://github.com/pypa/pip/issues/8714 UX design: what should pip output when it is backtracking during dependency resolution? #8714 * Need to develop a plan of action. * Want to be done w/ UX research by end of Aug * Potential workflow * Pradyun develops a few options/examples (mockups) * UX team provides feedback on this, based on best practices and experience. * no feedback loop w/ users. * Nicole: Not ideal, but reasonable * Bernard: Given the constraint of time, it's [best/good]. Pradyun, please only mock this up (don't implement it in code). Low fidelity is better :) * Sumana: "sketches", not prototypes. * If I can think abotu what a decoration might be, two distinct approaches (examples of decorations) * Nicole: depends on verbosity. * Poetry presents no output. * Give just a small amount of information - e.g. what version is being tried and if it is rejected or accepted. (Thanks!) * Bernard: information hierarchy and verbosity * This is a difficult situation, because there will be somebody who will want to see all of the information all of the time. * Most users only care about "worked" vs "didn't work" -- question of verbosity. provide the least amount of information (we can test that w/ people) -- if it's missing information, we can add it in. if we add more thing, people will be less inclined to suggest removing bits. * Minimal is better at the beginning. * verbosity: give a lot of information / give a little * sequence: stage the process is going thru, and which components the user really needs details on. e.g. * stage 1: high or low verbosity: Is it important for the user to know about this? That it is happening, or what the details are? * stage 2: high or low verbosity: is this important to the user or not? Maybe not -- maybe it's more pip internal. Log to a log file, but don't display on screen. * stage 3: high or low verbosity: maybe this is very important for user to see * end: high or low verbosity: maybe this is very important for user to see * TODO: Pradyun to make 2 mockups by early next week (~24-25th) * https://github.com/pypa/pip/issues/8664 New resolver: What performance is acceptable, and are we there yet? #8664 * Pradyun: I think this is closely tied to https://github.com/pypa/pip/issues/8380 (`ResolutionTooDeep` error). * Pradyun: I will be posting more thoughts in a comment on #8664 (what performance is acceptable) issue -- not done yet. :( * `ResolutiontooDeep` happens when pip is trying multiple options because of conflicts. Ths is closely tied to being perceived as too slow. The heuristic we need to figure out to determine when we are taking too long will inform the error message, and vice versa: feedback from users on when pip is too slow will help us set the threshold. We are not 100% sure of how to determine the threshhold for too-slow perf or trying too many options and being slow. * Bernard has a question: Is this connected to the number of iterations the resolver goes thru resolutions? Previously (from memory!) it was set to a "low" (hundred?) number, it was then set high (millions?). * Was 100, is 2,000,000 now. https://github.com/pypa/pip/pull/8275 * Sumana: yes to bernards question, it is directly related. It is the cut-off point. Changed to 2M. The more attempts we go thru the greater the chance of finding a proper resoluton, but also there are things that pip does that people perceive to be slow. the spikeyness of taking long to do something is because of that threshold. PG is interested in not pulling decisions out of nowhere and choosing a number to set this to. What are the benchmarks we should reach for good performance. We've been gathering data on what performance users are getting. * S did a test on her laptop and it took 100% more time in some cases to update something. We're gathering artisanal data - we need moar data! * Start by putting together manual speed tests to compare how fast the new one is vs old one. * based on that we decide a level of speed that is acceptable - that will then help us drive improvements.