Contribution is the main criterion for judging a paper. A paper is a contribution if it has two properties: originality and validity.
- The originality of a paper is the degree to which the ideas presented are significant, new, and interesting. Most papers are to some degree extensions or variations of previously published work; really groundbreaking ideas are rare.
• Is the contribution timely or only of historical interest? • Is the topic relevant to the venue’s typical readership? • What is missing? What would complete the presentation? Is any of the material unnecessary? • How broad is the likely readership? • Can the paper be understood? Is it clearly written? Is the presentation at an adequate standard? • Does the content justify the length?
Most papers have an explicit or implicit hypothesis—some assertion that is claimed to be true—and a proposal or innovation. Try to identify what the hypothesis is. The quality of a paper can be reflected in its bibliography. For example, how many references are there?
When you recommend that a paper be accepted, you should:
- Convince yourself that it has no serious defects.
- Convince the editor that it is of an acceptable standard, by explaining why it is original, valid, and clear.
- List the changes, major and minor, that should be made before it appears in print, and where possible help the author by indicating not just what to change but what to change it to; but if there are excessive numbers of errors of some kind, you may instead want to give a few examples and recommend that the paper be proofread.
- Take reasonable care in checking details such as mathematics, formulas, and the bibliography.
When you recommend that a paper be rejected, or recommend that it be resubmitted after major changes, you should:
- Give a clear explanation of the faults and, where possible, discuss how they could be rectified.
- Indicate which parts of the work are of value and which should be discarded, that is, discuss what you believe the contribution to be.
- Check the paper to a reasonable level of detail, unless it is unusually sloppy or ill-thought.
In either case you should:
- Provide good references with which the authors should be familiar.
- Ask yourself whether your comments are fair, specific, and polite.
- Be honest about your limitations as a referee of that paper.
- Check your review as carefully as you would check one of your own papers prior to submission.