review process

Instruction:

This is an opportunity for students to reflect on and distill some of the key concepts in the course and is different to a formal essay or report. After a series of group activities, it is also your opportunity to earn grades solely on your own thoughts and ideas.

We ask you to reflect the nature of the peer review process and your group activities as an approach to illustrating it. In Week 2 we discussed the peer review process and then ran through group activities participating in a peer review process as authors and as reviewers. Reflect on your own knowledge of the peer review process in theory (i.e., information that was delivered in lessons, linked resources, plus any further reading) as well as your own experience of the practice of peer review (i.e., the three phases of group activity). Reflect on whether the peer review process is the best possible approach for reporting climate science findings. Did your peer review process lead to a more rigorous paper in the end? Did your peer reviewing of others’ papers give you any insights on how much work goes into vetting papers prior to publication or what pitfalls there may be in the process? Outline the relative merits of the peer review process, as you see it. In your opinion, how could it be improved?

Please read the information below thoroughly

The peer review process

Science talks a lot about ‘peer reviewed’ findings as being the fundamental currency of our understanding on a topic. For most of you, you probably have a rough idea of what peer review is and why it is important, but here we will try to lay out a more detailed explanation of the process and point out some of its advantages and flaws.

When a scientist does some research and decides it is significant enough to share with the rest of the scientific community, they do this by publishing it in the form of a scientific article in an appropriate journal. These journals generally draw their reputation from the thoroughness of this review process.  A high quality journal in the Earth sciences such as Geophysical Research Letters rejects 30% of the articles submitted to it. Meanwhile Nature, arguably the most prestigious journal in science, rejects a staggering 93% of papers submitted to it. So how does someone get a paper published?

The first step for a researcher is to decide which journal would be the most appropriate place to publish this research, write it up, and submit it.  It is first assessed by an editor. The editor’s role is to be the point of contact between the authors and the reviewers, and to make the initial decision on whether or not the paper warrants consideration (does it fit the scope of the journal? Does it appear to be novel, quality research?). At this point the article is either rejected or it is sent out to several (usually 2-3) anonymous reviewers (see the flow chart below for a summary).  It is important for the authors not to know who the reviewers are so that reviews can be candid.

The reviewers read the submission and write a thorough critique (a “review”). The editor reads these reviews and then makes a decision, usually either: 1 – the article merits publication in its current state (rare on the first go); 2 – the article is seriously flawed or inappropriate for the journal and is rejected; or, 3 – the article may be publishable after revisions. This third situation is the most common; in this case the authors must address the reviews point-by-point and either make the requested alterations or counter the criticisms. Typically, an article will go through 2-3 rounds of revisions during the review process over a period of several months. If you’re unlucky however, this process can go on for much longer.

Peer review subjects the ideas, methodology, assumptions and interpretations to the critical scrutiny of other experts, who try to find and correct errors, confusing language or claims not supported by the evidence.  It is a screening process that aims to minimise the chance that a published article contains egregious errors, and almost always improves the paper compared to how it was originally written.

The process is not perfect.  Reviewers or editors can occasionally be biased in favour of, or against certain findings or authors, although having several reviewers mitigates against this.  Reviewing manuscripts takes a lot of time and doesn’t lead to glory for the (anonymous) reviewer, so sometimes reviewers don’t put in the required effort, and flawed papers occasionally sneak through.  While efforts have been made to address these problems, many agree that there is a lot of room for improvement. If you’d like to know more about the history and issues surrounding the peer review process, Nature has a particularly good article on the topic.

The IPCC

The Intergovernmental Panel on Climate Change (IPCC) is an international body that was established in 1988 by the World Meteorological Organisation (WMO) and the United Nations Environment Programme (UNEP). Its purpose is to review and assess the science related to climate change so that governments and policymakers have up-to-date information on the basic science, the impacts of climate change, future risks, and options for mitigation and adaptation. The aims of these reports are to be policy relevant, but not prescriptive – to help inform government policy by spelling out what different actions might lead to, but not dictate which policies are wrong or right. Participation in the IPCC is open to any member country of the WMO or UN, which allows it a unique opportunity to draw in scientific experts from all over the world to contribute. Currently, the IPCC has 195 members and its assessment reports represent perhaps the largest and most successful examples of international scientific collaboration in history. If you would like to know more about the structure of the IPCC, you can find it here on their website or read a short explainer here.

The IPCC‘s work falls into three main categories or ‘working groups’.

1. The first covers the ‘Physical science basis’ and its reports summarise our understanding of the climate system as relevant to current climate change.
2. The second working group addresses ‘Climate change impacts, adaptation and vulnerability’ and seeks to inform governments and other policy makers allowing them to plan for the current impacts and potential future risks of climate change.
3. The third working group addresses ‘Mitigation of climate change’. This concerns things like the state of the world’s energy markets, ways of reducing carbon emissions, population and economic considerations, and more recently, the discussion of geoengineering.

The combination of the output from these working groups produces assessment reports every several years to stay up-to-date with new advances in the science. The last report, the Sixth Assessment Report (AR6), was released in 2021/2. The summary video below was released by the IPCC to coincide with AR6.