Using Journal Timing Charts to Improve Manuscript Flow: A Closer Look at Editors’ and Reviewers’ Performance

February 3, 2022

By: Sherrie Hill and Kristen Overstreet

In today’s post, we will continue to dig deeper into the time to initial decision.  Many journals report the time to initial decision as the mean (average) time to initial decision by year.  While this does give insight into how the journal is performing, this is just a high-level overview of what is occurring at this stage in the workflow.  The time to initial decision is impacted by the performance of multiple people, including those in the editorial office, the Editor-in-Chief (EIC), the Editors, and the Reviewers.

In our previous blog, we discussed several EIC timing charts that can give insight into the process.  In our next post, we will be examining editorial office performance charts.  Today, we are going to be looking at editor and reviewer timing charts to gain a better understanding of their performance and time taken in peer review.  We will look at these charts under a scenario where the editors invite reviewers and give their recommendations on a manuscript to the Editor-in-Chief, who makes the final decisions.

The first timing chart to give more information about the (e.g., associate, section) editors’ and reviewers’ collective performance is the time from editor assignment to initial recommendation.  Time taken by the editorial office and the Editor-in-Chief has been removed.

In our example, we can see the combined editor and reviewer performance time is improving from 2019 to 2020.  While this is great news, it is not easy to understand where the improvements came from or how we can make additional improvements.

The time-to-initial-decision-by-editor chart is a good starting point for more in-depth analysis. This chart makes it obvious which editors are significantly slower or faster than their peers. While a much faster editor might seem to benefit the journal, a closer look might show their rejection rate is too high, which is a detriment.

When we look at the editors with the fastest and slowest times to initial decision, on the charts above and below, two editors are stand-outs on both charts: McDonald and Myers. McDonald takes a longer time to reach an initial decision and has an extremely low rate of recommending rejection. Perhaps additional training on identifying low quality submissions prior to sending for full peer review would decrease the editor’s and reviewers’ workload and improve their time to initial decision. Myers’ rate of recommending rejection is very high, and their time to initial decision is faster than their peers. Myers may be rejecting too many papers without full peer review. Although peer review is a slower process, it may reveal more manuscripts that are worthy of publication despite needing extensive revisions.

Though we have looked closer at the editors’ performance in the last two charts, they still haven’t told us everything.  There are many factors that influence the timing during this stage of peer review, including the editors’ experience levels; the condition of the incoming submissions; the breadth, diversity, and quality of the journal’s reviewer database; the time of year; the editors/reviewers’ work commitments outside of their journal work; the number of assignments per year; changes in journal policy; etc.  Though it would be difficult and time consuming to produce charts characterizing all the different factors impacting this timing stage, there are a few easy-to-produce charts that will help us gain some insight into the process.

The first of these charts is the time from editor assignment to the time the first reviewer is invited, broken down by editor.  This should flag editors who may not be on top of their assignments.  It should be noted that editors may need more time to evaluate submissions requiring statistical analysis or submissions on complex research topics, which will delay their decision to send out for peer review and invite the first reviewer, as will a greater number of assignments.

Again, we can readily determine our fastest and slowest editors using this chart.  Most of the editors are taking more than two weeks before inviting the first reviewer.  As a group, the editors might need to review their procedures and determine if there is a way to streamline the process and shorten the time needed to get manuscripts into peer review.  It would also be useful to identify why editors Mohammed and Jones are taking longer than their peers to invite reviewers.  Is it due to their subject area or their workload, or are their other factors coming into play?

Another chart that can yield insight is the time-from-last-review-completed-to-initial-recommendation, broken down by editor.  As with the previous chart, identify how the majority are performing.  For those who are slower, ask again, are they routinely handling submissions that are more complex than their peers or is their workload higher?  If not, editor training, experience, and/or how often they are going into the submission system may be having a significant impact.

In our example, this group of editors seems to be performing well. They are submitting their recommendations within one to two weeks after the reviewers have submitted their comments. It is probably worth doing an investigative analysis of the four slower editors. We will describe that process in a future blog post. We should produce a chart to show the number of assignments per editor. From that chart below, you can see that Editor Santos only had 2 assignments during the time frame we have been looking at. Sometimes editors can become disengaged when they handle only a few submissions.

The time a manuscript spends with peer reviewers has a significant impact on time to initial decision. Some journals place an emphasis on the average (mean) time taken for a reviewer to complete their review (below on the left). Though this is an important indicator, for today’s analysis we will focus on total time in peer review, which begins when the first reviewer is invited and ends when the last review is submitted.

In the chart above (right) the 2020 average (mean) shows the total time in peer review increased by 9 days from the previous year.  Some reasons for this could be: increased volume of submissions; reviewer fatigue; editor fatigue; editors having difficulty selecting appropriate reviewers; editors taking longer to invite alternate reviewers if a reviewer declines or fails to respond to the invitation; environmental conditions, such as the recent COVID pandemic; etc.  Though the timing of this stage has a dramatic impact on the time to initial decision, it can be harder for a journal to influence this timing through changes to journal policy or procedures.  In a future post, we will make suggestions for improving the total time in peer review, as well as individual reviewer time.

Gaining a clear understanding of how your editors and reviewers are performing will help you implement policies at your journal to reduce the time to initial decision.  Many small adjustments can result in significant changes in your journal’s performance.