1. Introduction
In this tutorial, we’ll explain what peer review is, what types there are, and how to respond to it.
2. Peer Review
Each research article goes through several phases upon submission:
The most important phase is that of peer review. In it, the editor in charge asks a couple of renowned scientists to read the submitted paper and give their recommendations, pointing out its strengths and weaknesses. This process ensures the papers are of high quality and conform to the scientific standard of the day.
Based on the feedback, the editor accepts or rejects the paper or asks the authors to resubmit it after implementing a minor or major revision.
3. Types of Peer Review
There are three main types of peer review depending on the authors’ and reviewers’ anonymity:
- single-anonymized
- double-anonymized and
- open review
3.1. Single-Anonymized Review
This is probably the most common type. Here, the reviewers know the authors’ names, but the authors don’t know who the reviewers are:
This way, the reviewers can be more honest as their identities are hidden, so they don’t have to fear retaliation from the authors if they give negative feedback or recommend rejection.
3.2. Double-Anonymized Review
However, the single-anonymized type isn’t bias-free. What if the authors are big names in the world of science? In that case, the reviewers may be less likely to raise concerns over the experimental design, results, or conclusions, even if those concerns are legitimate and the paper has flaws. Similarly, a reviewer can be harsher than necessary towards newbie researchers or authors from a country or institution for which the reviewer holds prejudice.
These biases can leak into a review unconsciously, so in the double-anonymized type, the authors are anonymized, too, and the reviewers don’t know who wrote the paper they’re reviewing:
That way, we expect the reviewers to be more objective and focus on the papers, not the names.
3.3. Open Review
However, just because the names aren’t there doesn’t mean the authors (or reviewers) can’t infer who the other party is. There aren’t many researchers in niche fields, so we’ll likely be able to guess the reviewer(s) and authors by looking at the topic and references. Similarly, every author develops their writing style, based on which a researcher can guess who wrote the paper (or the review).
Further, some authors have complained that some reviewers take advantage of their anonymity and write unreasonably harsh reviews or aren’t very careful while reading the papers. Taking their anonymity away means that reviewers must ensure they provide quality feedback and read the paper carefully, judging it by its merits only. So, in this type, no name is anonymized. The reviewers know who the authors are, and the authors know who the reviewers are:
However, this is the least common type of peer review.
3.4. Variations and Other Types
Depending on the journal, publisher, and conference policy, we may encounter other types or variations of the three main explained above.
For example, some journals may keep reviewers anonymous during the review but reveal their names after publication. Other journals will ask the reviewers if they want to be listed as such in case the paper gets accepted.
Another type is collaborative review, in which a group of scientists provides one unified review report, or an author works with a reviewer until the paper is recommended for publication.
4. How to Respond?
It depends on the feedback and the reviewers’ requests, but a good practice is to take some time to cool off after reading it the first time. The thing is that negative feedback may put us in a defensive mood. Writing the response from that perspective isn’t likely to get us far. It could turn into attacking the reviewers and claiming they’re wrong without considering that they may be right.
Instead of arguing with our reviewers, we should convince them and the editor that our paper deserves to be published. So, we need to be respectful even if they aren’t.
4.1. Explain Everything
So, after giving it a day or two (or even more), we can revisit the review and try to be as objective as possible. Each review point should be analyzed thoroughly (with co-authors).
In general, there are two courses of action for each feedback point:
- do what the reviewer asks
- decline to do what the reviewer wants
Whatever we decide, the decision should be stated and explained in a separate document in which we prepare our response. It should explain every change to the reviewed submission.
4.2. Examples
Let’s say a reviewer asks us to conduct a follow-up experiment to check something. If we do this, we should say something along these lines:
- We did the additional experiment as you asked. The results are shown in Figure 4 on page 18 and discussed in Section 4 on page 19. We interpret them as follows: …
If we think no additional experiment is necessary, we should explain why that’s the case. For example:
- We considered your suggestion to do a follow-up experiment to check if parameter A influences the response variable on the particular level of parameter B you singled out. The experiment is certainly doable, but we believe it is unnecessary because that specific parameter B value occurs very rarely in practice. There was no significant interaction between A and B in our main experiment.
Even if we decline to do what the reviewer asked, we should explain in the main text why we didn’t do that and similar experiments. That’s because future readers might ask the same question as the reviewer who suggested an additional experiment.
4.3. Response Document
Some journals specify the format of response documents. In that case, we should follow their guidelines.
However, if the format isn’t specified, we’re free to format the document as we want to, but we should keep it simple. It’s sufficient to go over each point, repeat it, and then write the response and what action was taken to respond to their comment. Additionally, let’s not forget to thank the reviewers for their constructive feedback and list the main changes at the beginning of the document. Here’s an example:
Dear reviewers,
Thank you for your constructive feedback. In the continuation of this document, please find our responses.
We used the analysis technique that reviewer #1 suggested, but the results did not change.
Reviewer #1
- Comment 1: You should have compared the confidence intervals instead of sample means in Section 3.
- Response: Thank you for pointing this out. We corrected this in our revised manuscript. The lower bound of the mean in the experimental group is higher than the upper bound of the mean in the control group, as illustrated in Figure 4 in Section 3. Therefore, we kept the conclusion that the acceleration technique (evaluated on the samples in the experimental group) yields better performance gains than the standard method (evaluated on the control group samples).
- Action: We computed the 99% confidence intervals using the normal approximation method and checked if they intersect. Figure 4 in Section 3 was updated, as well as the last paragraph in the same section.
- Comment 2: …
…
4.4. Clarity and Language
Even if a reviewer doesn’t understand something and lists it as a flaw, it’s a good practice to consider rephrasing the sentences and paragraphs in question instead of just responding that the reviewer’s interpretation is wrong.
Reviewers are our “focus group” readers, so if they don’t understand something, it indicates that other readers might misinterpret our words. So, we should work on improving clarity in such cases.
Finally, we should be formal and respectful, and our response document should be grammatically correct.
5. Conclusion
In this article, we talked about peer review and explained its main types. They differ in the anonymity of reviewers and authors.
We also gave some advice on how to respond to it. The key point is to be respectful and back the response to every feedback point with data or theory.