This page explains the duties of reviewers on the program committee. Committee members can add questions and observations to this page. Do not refer to specific proposals on this page, because this page is publicly visible.

Reviewers have logins on http://submit.python.org, just like proposal authors. Reviewers have the additional privilege of being able to view other proposals and to record comments and votes on them.

Proposals will be assessed using the [ http://www.iam.unibe.ch/~oscar/Champion/ Identify the Champion] process. Basically, reviewers vote using a +/- 1, +/- 0 scale. To be accepted, a paper must have at least one +1 vote; each person who votes +1 on a paper is a paper's champion.

When reviewers log in, their web page will list their own proposals as well as a bunch of hyperlinks to various specialized lists of proposals. The most important list is the one of proposals still needing reviews, at http://submit.python.org/reviews_needed. Reviewers can then view those proposals and record their votes along with additional comments.

Be sure to read the most recent version of the proposal; check the date and time of the file. Users can upload multiple revisions of their proposal.

When you mark a review as publicly visible, it gets e-mailed to the author the same day (not immediately -- it's a cron job that runs every so often), so the author can update the proposal, fix a problem, etc. After the reviewing is over, authors are mailed all comments on their proposal, whether marked as publicly visible or not.

If you want your comments to be e-mailed to the proposal's author right away, you MUST uncheck the "keep comments private" box in the review form.

If you have suggestions that would improve a proposal you should normally make them visible, allowing the author to respond before the acceptance decisions are made (hopefully with an updated proposal).

In previous years, authors were e-mailed the comments on their proposal after the final decisions were made. You should write your comments with this in mind. Please try to avoid unnecessarily negative comments, and don't be rude.

Reviewers should join the pycon-pc mailing list, which is for program committee discussions.

When a proposal is submitted, three reviewers are randomly assigned to it. Those assignments aren't enforced, so reviewers should feel free to skip proposals assigned to them that they don't feel competent to review. Conversely, reviewers should be reviewing proposals assigned to other people, and doing so is greatly encouraged.

QUESTION: Is there a way for a reviewer to find out which proposals he or she was assigned to?

Once you've logged in to submit.python.org, under the 'Your assigned reviews' header is a list of proposals assigned to you that you haven't voted on yet (if you just posted a comment with no vote, the proposal will remain on this list). There's currently no page that lists everything that was assigned to you, whether you voted on it or not.


CategoryPyCon2006

PyCon2006/Reviewing (last edited 2008-11-15 14:00:51 by localhost)

Unable to edit the page? See the FrontPage for instructions.