|
Size: 5047
Comment:
|
Size: 3429
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| (This page has been moved to http://us.pycon.org/TX2007/ReviewerNotes; the contents of this page might be outdated.) |
|
| Line 3: | Line 5: |
| Reviewers have logins on http://submit.pycon.org, just like proposal authors. Reviewers have the additional privilege of being able to view other proposals and to record comments and votes on them. |
Reviewers have logins on http://us.pycon.org/apps07/proposals/, just like proposal authors, and can view the list of submitted proposals. Reviewers can record comments and votes on proposals. |
| Line 9: | Line 11: |
| at least one +1 vote; each person who votes +1 on a paper is a paper's champion. | at least one +1 vote; each person who votes +1 on a paper is a paper's champion. Once all the proposals have received votes from three different reviewers, we'll hold an IRC meeting to make the final selections. Not all the committee members show up for this chat, though all members are certainly welcome to participate. Papers that only get +1 or +0 votes are usually automatically accepted, and papers with only -1 or -0 votes are usually rejected. Most of the discussion is spent looking at papers with a mix of positive and negative votes, or papers without a strong +1. |
| Line 14: | Line 18: |
| proposals still needing reviews, at http://submit.pycon.org/reviews_needed. | proposals still needing reviews. |
| Line 38: | Line 42: |
| to them that they don't feel competent to review. Conversely, reviewers should be reviewing proposals | to them that they don't feel competent to review. Conversely, reviewers can review proposals |
| Line 41: | Line 45: |
| ''QUESTION: Is there a way for a reviewer to find out which proposals he or she was assigned to?'' Once you've logged in to submit.pycon.org, under the 'Your assigned reviews' header is a list of proposals assigned to you that you haven't voted on yet (if you just posted a comment with no vote, the proposal will remain on this list). There's currently no page that lists everything that was assigned to you, whether you voted on it or not. |
|
| Line 51: | Line 46: |
| == Scheduling the Meeting == Please make a note against each time you would NOT be available to meet and discuss the submissions. Note that all times are Eastern Standard. ''SteveHolden'' ''I'm available any time. --amk'' === Tuesday Jan 11 === 8am N/A - MichaelBernstein 9am iffy - MichaelBernstein iffy - MichaelChermside 12 noon 1 hour only - SH N/A - Brett N/A - tim N/A - George N/A - MichaelChermside 6 pm N/A - George iffy - Brett N/A - Aahz 8 pm === Wednesday Jan 12 === 8am N/A - MichaelBernstein 9am iffy - MichaelBernstein iffy - MichaelChermside 12 noon 1 hour only - SH 2 hours only - Brett N/A - tim N/A - George N/A - MichaelChermside 5pm N/A - itamar, until 9pm 6pm N/A - tim N/A - George N/A - Brett N/A - jim 7 pm N/A - jim 8 pm N/A - George Available 'til 9:15 - Garry 1 hour only - Brett N/A - jim 9 pm N/A - jim === Thursday Jan 13 === 8am N/A - MichaelBernstein 9am iffy - MichaelBernstein iffy - MichaelChermside 12 noon 1 hour only - SH N/A - tim N/A - George N/A - MichaelChermside 6pm N/A - Aahz 8 pm N/A - MichaelBernstein, for the rest of the evening === Friday Jan 14 === 8am N/A - MichaelBernstein 9am iffy - MichaelBernstein iffy - MichaelChermside 12 noon 1 hour only - SH N/A - tim N/A - George 2 hours only - Brett N/A - MichaelChermside 6pm not until 7:00 EST - Brett N/A - Aahz 8 pm iffy - Aahz N/A - MichaelBernstein, for the rest of the evening ---- CategoryPyCon2005 |
CategoryPyCon2007 |
(This page has been moved to http://us.pycon.org/TX2007/ReviewerNotes; the contents of this page might be outdated.)
This page explains the duties of reviewers on the program committee. Committee members can add questions and observations to this page. Do not refer to specific proposals on this page, because this page is publicly visible.
Reviewers have logins on http://us.pycon.org/apps07/proposals/, just like proposal authors, and can view the list of submitted proposals. Reviewers can record comments and votes on proposals.
Proposals will be assessed using the [http://www.iam.unibe.ch/~oscar/Champion/champion.html Identify the Champion] process. Basically, reviewers vote using a +/- 1, +/- 0 scale. To be accepted, a paper must have at least one +1 vote; each person who votes +1 on a paper is a paper's champion.
Once all the proposals have received votes from three different reviewers, we'll hold an IRC meeting to make the final selections. Not all the committee members show up for this chat, though all members are certainly welcome to participate. Papers that only get +1 or +0 votes are usually automatically accepted, and papers with only -1 or -0 votes are usually rejected. Most of the discussion is spent looking at papers with a mix of positive and negative votes, or papers without a strong +1.
When reviewers log in, their web page will list their own proposals as well as a bunch of hyperlinks to various specialized lists of proposals. The most important list is the one of proposals still needing reviews. Reviewers can then view those proposals and record their votes along with additional comments.
Be sure to read the most recent version of the proposal; check the date and time of the file. Users can upload multiple revisions of their proposal.
When you mark a review as publicly visible, it gets e-mailed to the author the same day (not immediately -- it's a cron job that runs every so often), so the author can update the proposal, fix a problem, etc. After the reviewing is over, authors are mailed all comments on their proposal, whether marked as publicly visible or not.
I you want your comments to be e-mailed to the proposal's author right away, you MUST uncheck the "keep comments private" box in the review form. If you have suggestions that would improve a proposal you should normally make them visible, allowing the author to respond before the acceptance decisions are made (hopefully with an updated proposal).
In previous years, authors were e-mailed the comments on their proposal after the final decisions were made. You should write your comments with this in mind. Please try to avoid unnecessarily negative comments, and don't be rude.
Reviewers should join [http://mail.python.org/mailman/listinfo/pycon-pc/ the pycon-pc mailing list], which is for program committee discussions.
When a proposal is submitted, three reviewers are randomly assigned to it. Those assignments aren't enforced, so reviewers should feel free to skip proposals assigned to them that they don't feel competent to review. Conversely, reviewers can review proposals assigned to other people, and doing so is greatly encouraged.