Fast Facts:
If possible, editor charts should be included in your report. It helps the team to understand how the editor assignment loads are changing over time, as well as any editor who might need extra help.
Including reviewer charts can give insight into the health of the reviewer pool. You can include information on how the reviewers are responding to invitations, where your reviewers are located, and if they compete the reviews they agree to perform.
How to Design an Effective Editorial Report: Part 4
September 1, 2021
By: Jason Roberts & Sherrie Hill
In this, our last post in the series on designing effective editorial reports, we will look at some editor and reviewer charts that you may want to include in your editorial reports.
Editor Charts
The editorial team often wants to understand how the journal’s editors are performing. One of the most common charts that you can include is the time from editor assignment to initial decision. This chart reports the mean time for all editors as compared to previous years. From this chart, you can tell how your team, as a whole, is performing. Your editorial team may prefer to use the median values.
However, it is often beneficial to look at individual editor performance. Two key indicators are the time from editor assignment until the first reviewer is invited. Another data point that is helpful to understand is the time from when the last review was submitted until the initial decision was rendered. From these charts, you can locate pinch points in your peer review process, which lead to longer overall times for manuscripts to receive their initial decisions.
Another performance indicator concerns assessing how effective your editors are at securing reviewers for manuscripts going through peer review. You could, for instance, report out the mean (average) number of reviewers per manuscript each editor is inviting. Editors with higher numbers of reviewers per manuscript might be having trouble identifying people in the journal’s reviewer pool who would be likely to agree to review. Data may also suggest a geographic bias or limitation in the willingness of Editors to widen their personal pool of potential reviewers. However, there are confounders to this interpretation. The situation could actually reflect the fact that the subject area is especially niche with a very limited pool of reviewers. If an Editor is assigned manuscripts at a certain time of the year this may also impact performance (such as during holiday seasons). Therefore, it is best to also report the number of manuscripts assigned to each editor for the same time period, so that the number of reviewer invitations per manuscript can be better understood. If all editors have higher than expected numbers for this variable, it might indicate that your reviewer database needs cleaning or more reviewers need to be recruited.
You might consider including information on the number of unique reviewers that your editors invited for a given time period. Low numbers of unique reviewers might indicate that editors are going back to the same people over and over, which can lead to reviewer burn out. Additionally, using the same reviewers too often could cause your published content to be skewed toward the views of those few people. Again, you always need to consider the number of editor assignments for the same time period, as you would expect higher numbers of unique reviewers for editors with more assignments. If an editor only had a few assignments in the time period, you would know not to be concerned about a low number of unique reviewers. The number of unique reviewers per editor is a time intensive chart to produce, however, this chart can be easily generated in Origin Reports.
One very effective editor performance chart is the editor rejection-rate bubble chart. This particular chart not only shows each editor’s rejection rate, but it also shows the average time it takes editors to reach an initial decision and the number of assignments they had during that time period. In our next blog post, we will be discussing in more detail how to interpret this bubble chart.
Reviewer Charts
One common indicator of individual reviewer performance concerns responses to the journal’s reviewer invitations. If you choose to include this type of chart in your editorial report, you should note the number of invitations sent as well as the responses to those invitations. You can choose to report out how many reviewers agreed/declined/were uninvited/were terminated. You can also report how many reviewers completed reviews/declined after agreeing/were uninvited/were terminated. If you choose to report the number of completed reviews, be sure to also include the number of in-progress reviews, if any. Conversion rate charts, such as the number of reviewers who completed reviews compared to the number of reviewers who agreed to review are also very helpful. Read this blog post to learn more about “Reviewer Conversion Rate Charts”.
If your journal is trying to improve its regional diversity, you may want to report on the number of reviewer invitations by country. It might also be a good idea to report on the number of submissions by country as a reference point. As your submissions from different regions increase, your journal may want to recruit more expert reviewers from those regions as well, so that there are reviewers in your reviewer pool who have a better understanding of the topics of interest in that region. Be sure to color match the country colors between charts to make interpreting the relationships easier.
You may also want to report on reviewer performance, such as the mean (average) time taken for a reviewer to complete a review this year as compared to previous years or the total time with reviewers by year. The latter is often calculated from the time that the first reviewer accepts an invitation to review that manuscript until the last review is completed.
You should select editor and reviewer charts that either give information relating to your journal’s goals or to highlight areas of concern. Keep in mind who your audience is and what information they need to perform their roles related to the journal.
This concludes our series on how to design an effective editorial report. We hope that you found something in these posts that you can use to improve your editorial reports. All of these charts are available in Origin Reports, as well as many more charts that can help you uncover trends and report your key data points. Remember, Origin Reports was designed to help make your journal reporting easier and faster, so that you can spend more time interpreting your data!
