(This page has been moved to http://us.pycon.org/TX2007/ReviewerNotes; the contents of this page might be outdated.)

This page explains the duties of reviewers on the program committee. Committee members can add questions and observations to this page. Do not refer to specific proposals on this page, because this page is publicly visible.

Reviewers have logins on http://us.pycon.org/apps07/proposals/, just like proposal authors, and can view the list of submitted proposals. Reviewers can record comments and votes on proposals.

Proposals will be assessed using the Identify the Champion process. Basically, reviewers vote using a +/- 1, +/- 0 scale. To be accepted, a paper must have at least one +1 vote; each person who votes +1 on a paper is a paper's champion.

Once all the proposals have received votes from three different reviewers, we'll hold an IRC meeting to make the final selections. Not all the committee members show up for this chat, though all members are certainly welcome to participate. Papers that only get +1 or +0 votes are usually automatically accepted, and papers with only -1 or -0 votes are usually rejected. Most of the discussion is spent looking at papers with a mix of positive and negative votes, or papers without a strong +1.

When reviewers log in, their web page will list their own proposals as well as a bunch of hyperlinks to various specialized lists of proposals. The most important list is the one of proposals still needing reviews. Reviewers can then view those proposals and record their votes along with additional comments.

Be sure to read the most recent version of the proposal; check the date and time of the file. Users can upload multiple revisions of their proposal.

When you mark a review as publicly visible, it gets e-mailed to the author the same day (not immediately -- it's a cron job that runs every so often), so the author can update the proposal, fix a problem, etc. After the reviewing is over, authors are mailed all comments on their proposal, whether marked as publicly visible or not.

I you want your comments to be e-mailed to the proposal's author right away, you MUST uncheck the "keep comments private" box in the review form. If you have suggestions that would improve a proposal you should normally make them visible, allowing the author to respond before the acceptance decisions are made (hopefully with an updated proposal).

In previous years, authors were e-mailed the comments on their proposal after the final decisions were made. You should write your comments with this in mind. Please try to avoid unnecessarily negative comments, and don't be rude.

Reviewers should join the pycon-pc mailing list, which is for program committee discussions.

When a proposal is submitted, three reviewers are randomly assigned to it. Those assignments aren't enforced, so reviewers should feel free to skip proposals assigned to them that they don't feel competent to review. Conversely, reviewers can review proposals assigned to other people, and doing so is greatly encouraged.


CategoryPyCon2007

PyCon2007/ReviewerNotes (last edited 2008-11-15 14:00:43 by localhost)

Unable to edit the page? See the FrontPage for instructions.