Using Journal Timing Charts to Improve Manuscript Flow: Time to Final Decision, Editorial Office Timing, and Adding Statistics
Using Journal Timing Charts to Improve Manuscript Flow: Time to Final Decision, Editorial Office Timing, and Adding StatisticsBy: Sherrie Hill and Kristen OverstreetMarch 2, 2022In the last post in this series on timing charts, we will look at the time from submission to final decision and the time a manuscript spends specifically in the hands of the editorial office.Time to Final DecisionThe time to final decision chart includes all the stages from initial submission to final decision and shows the total time taken for each of those stages. The data gives authors an idea of how long it will take to reach a final decision, and it gives the journal another indicator to track when looking for whether improvements are needed to shorten the time a manuscript spends in the peer review process.The time to final decision chart (below on the right) shows a very similar trend to our previous time to initial decision (below on the left), though there is a slight increase in the 2020 time to final decision.Just as we broke down the timing charts for initial decisions, we should break down the time to final decision. Since there are elements that the journal has less control
Origin Reports’ Chart of the Month – June: Reviewer Conversion Rate Charts
Origin Reports’ Chart of the Month - June:Reviewer Conversion Rate ChartsBy: Jason Roberts & Sherrie HillThe conversion rate charts are used to quantify how individuals invited to peer review respond.Invited-to-agreed conversion rate vs. invited-to-completed conversion rateInvited-to-agreed conversion rate and the invited-to-completed conversion rate are typically used to examine the health of your reviewer database. Invited-to-agreed is a measure of the percentage of the invitations to review that led to an agreement to review. Invited-to-completed is a measure of the percentage of the invitations to review that led to completed reviews. The invited-to-agreed conversion rate chart, of course, does not show whether the reviewers who agreed ultimately submitted their review.Higher conversion rates indicate a healthy reviewer database containing individuals seemingly willing to provide peer reviews when asked more often than not. Furthermore, higher conversion rates may also indicate that your editors are effective at selecting reviewers who are more likely to agree to provide a peer review. Conversely, a lower conversion rate possibly indicates that many individuals currently in your reviewer database are less predisposed to agree to peer review. It should be noted that for an incomplete (current) year, especially early in the year, the conversion rate might be lower
Why You Shouldn’t Recycle Historical Summary Data in Your Charts
Why You Shouldn't Recycle Historical Summary Data in Your ChartsBy: Jason Roberts & Sherrie HillWhen creating a new chart, you should not 'harvest' data from previous charts created by someone else. Unless it was explicitly documented, it is not possible to know what they counted, included, or excluded. When performing data calculations, it can be difficult to match someone else's summary data.If you created the chart above using historical data for years 2017-2019, would you be able to explain why it appears that there was a significant increase in submission in 2020? Was this increase expected based on what the editorial office saw in 2020? Did the previous managing editor calculate submissions differently? For instance, they may have excluded editorials or other commissioned materials.A better practice is to start over, using raw data from that time period and calculating all of the values yourself so you can be sure all reported data points have been obtained in the same way.Additionally, most manuscript handling systems store data relating to the submissions, editors, and reviewers in a number of different data tables. (Databases are made up of data tables. Data tables are two dimensional objects [rows and columns of data], which is
Origin Reports’ Chart of the Month – May Submissions by Country Map
Origin Reports’ Chart of the Month - MaySubmissions by Country MapBy: Sherrie Hill & Jason RobertsMost journals are trying to expand their readership to a larger worldwide audience. One of the ways this can be done is to present content from authors from a wide range of countries. To help track this data, journal offices typically use the corresponding author’s country to report the number of submissions by country to their editorial board.With the required data set identified, the question then becomes “What is the most effective way to present this data to your editorial board?” Predominantly, this type of data is reported in tables or bar charts. Tables are good at showing the data in an organized manner, though it is more difficult to visualize trends just by looking at the numeric values. Bar charts show trends much better than tables, however, if your report already includes several bar charts, it is hard to make the regional data stand out from all the other charts and unless you add labels, specific numbers can be hard to discern.Origin Reports gives you the option of showing your submissions by country on a world map. The country’s color is determined by the
Is it Possible to Love MyReports Even More?
Is it Possible to Love MyReports Even More?By: Sherrie Hill & Kristen OverstreetMyReports is a feature that allows you to save your chart parameters in a Custom Report, which can be updated as needed by simply uploading your most recent data set.We have now launched version 2 of MyReports, which includes many improved features to give you more control over how you present your data. MyCharts and MyReports are now more closely linked to make it clearer which charts are included in your custom reports and how the charts and reports are linked.Create Custom Versions of Standard ReportsOrigin Reports’ Standard Reports have been designed by editorial report experts. You can choose to start with one of our standard reports, modify it, and then save it as a Custom Report, rather than creating a custom report from scratch.New or improved features:See All of Your Saved Charts in MyChartsNow you can see all the charts that you have saved in one place. In version 2 of MyReports, when you add a chart to a report, it is automatically added to MyCharts as well.MyCharts Shows Links to MyReportsYou can now see which of your saved charts in MyCharts are being utilized in a
You Know, You Don’t Have to Figure It All Out Again
You Know, You Don’t Have to Figure It All Out AgainBy: Sherrie Hill & Kristen OverstreetThere are times in our professional lives when we find a tool that is amazing, but the tool requires some time to learn how to use it. Maybe, it is learning to create a new type of extraction report in your peer review system or learning a new software application that will help you do your job better. These tools are great and we can’t wait to use them again, but perhaps we only need to use it a few times a year. Then in the gap between the time you spent reading the instructions or sitting in virtual training and the time you need it again, you have a sinking realization that you have forgotten quite a bit of what you knew just a few months ago. I think we have all had that moment when we have to decide if it is worth trying to re-learn the ‘new’ shorter process or if it will take less time to return to the slower but more familiar way of doing things. Luckily, this isn’t something that you will need to worry about with Origin Reports. Origin
Origin Reports’ Chart of the Month – April Interpreting Bubble Charts
Origin Reports’ Chart of the Month - AprilInterpreting Bubble ChartsBy: Sherrie Hill & Jason RobertsBubble charts are a great way to show multiple aspects of your data in a visually interesting way. Once your audience becomes accustomed to bubble charts, they will quickly take in the various aspects of the chart to gain a deep understanding of the data.Today we are going to look at a percent rejected bubble chart for all initial decisions for a fictitious journal in a given time frame. We will discuss the importance of the size of the bubbles, as well as the color.Bubble sizeFirstly, you will notice that the size of the bubble is related to the number of initial decisions rendered on the manuscripts; the data are pulled from the country of the submitting author in a given time period. For instance, in the figure above, 36 initial decisions were rendered in 2020 on manuscripts originating from China. 24 (67% of the 36 initial decisions) of these were rejected. Larger bubbles indicate countries that received more initial decisions. Bubble size acts as a visual shortcut so that your readers can easily see which countries received the greatest number of initial decisions during the
Origin Reports’ Chart of the Month – July: Decision Tree for Initial Decisions
Origin Reports’ Chart of the Month - July:Decision Tree for Initial DecisionsBy: Sherrie Hill & Jason RobertsDecision trees provide an instant visual summary of the outcomes of your peer review process, offering details that may otherwise get buried when data are presented in a tabular form. Decision trees offer an overview of a process and details of that process all at once. They can be displayed either vertically or horizontally. The figure above is an example of a vertical decision tree that outlines the initial decision ratios for a journal. When discussing a vertical decision tree, you should move from the top to the bottom and discuss the elements of each row in comparison.In this example, the blue box shows the number of manuscripts that received a decision during the set time frame, in this case the year 2019, as shown in the chart’s Parameters section.The next row separates the manuscripts that received immediate decisions from the manuscripts that went through peer review. Immediate decisions are decisions that are rendered by Editors without any input from peer reviewers.When creating an initial decision tree, you must decide how you want to handle manuscripts that do not have an actual “immediate reject”
How to Design an Effective Editorial Report: Part 4
How to Design an Effective Editorial Report: Part 4By: Jason Roberts & Sherrie HillIn this, our last post in the series on designing effective editorial reports, we will look at some editor and reviewer charts that you may want to include in your editorial reports.Editor ChartsThe editorial team often wants to understand how the journal’s editors are performing. One of the most common charts that you can include is the time from editor assignment to initial decision. This chart reports the mean time for all editors as compared to previous years. From this chart, you can tell how your team, as a whole, is performing. Your editorial team may prefer to use the median values.However, it is often beneficial to look at individual editor performance. Two key indicators are the time from editor assignment until the first reviewer is invited. Another data point that is helpful to understand is the time from when the last review was submitted until the initial decision was rendered. From these charts, you can locate pinch points in your peer review process, which lead to longer overall times for manuscripts to receive their initial decisions.Another performance indicator concerns assessing how effective your editors are at
The How and the Why of Collecting Identity Data: Part 1
The How and the Why of Collecting Identity Data: Part 1By Kristen OverstreetSeptember 20, 2021 It’s Peer Review Week 2021! This is an important event, celebrated each year in September, to focus on a specific aspect of peer review through educational opportunities, information dissemination, and open discussion. This year’s theme is “Identity in Peer Review”.Peer Review week website At Origin Reports, that makes us think about capturing data around authors’ and reviewers’ identities, which can include the following:Countries of Residence – so we can determine the ratio (or balance) of authors to reviewers in various geographic locations,Roles for Individuals – how many authors are also reviewers?ORCiDs – how many of our authors have included ORCiDs (orcid.org) in their user accounts? How many have added it to their manuscripts for publication? Should we start requiring new system users to include an ORCiD in their accounts?Expertise – what are the areas of expertise for our authors and reviewers? Are there gaps we need to fill?Ensuring diversity, equity, and inclusion for our authors, reviewers, and editorial board members, raises all manner of privacy questions regarding the collection of “identity” data, such as gender, race, ethnicity, age, and disability. By collecting this information, we could
The How and the Why of Collecting Identity Data: Part 4 Collecting and Evaluating Identity Data
The How and the Why of Collecting Identity Data: Part 4Collecting and Evaluating Identity DataBy Kristen Overstreet and Sherrie Hill November 19, 2021 How to collect identity dataIn this post, of a multipart series on identity data, we will discuss considerations for when you begin collecting and interpreting the data. Some organizations choose to collect identity data through independent software applications (e.g., SurveyMonkey), while others use their journals’ submission systems. There are pros and cons to either.At the 2021 ISMTE Global Event in October, Dr. Teo Pulvirenti, from the American Chemical Society, discussed these pros and cons. Collecting data through an independent software application ensures that there will be limited access to the raw data collected, since you can easily control who within your organization will have access. Because the data are not stored within the submission system, reviewers and authors might feel more comfortable revealing sensitive identity data. However, because it is a separate application, it does represent an additional process that must be completed by people who are typically very busy. Therefore, your journal might find that the response rate is lower than it would be for questions asked within the submission system.Some submission system vendors are working on better
Using Journal Timing Charts to Improve Manuscript Flow: A Closer Look at Editors’ and Reviewers’ Performance
Using Journal Timing Charts to Improve Manuscript Flow: A Closer Look at Editors’ and Reviewers’ PerformanceBy Sherrie Hill and Kristen OverstreetFebruary 3, 2022In today’s post, we will continue to dig deeper into the time to initial decision. Many journals report the time to initial decision as the mean (average) time to initial decision by year. While this does give insight into how the journal is performing, this is just a high-level overview of what is occurring at this stage in the workflow. The time to initial decision is impacted by the performance of multiple people, including those in the editorial office, the Editor-in-Chief (EIC), the Editors, and the Reviewers.In our previous blog, we discussed several EIC timing charts that can give insight into the process. In our next post, we will be examining editorial office performance charts. Today, we are going to be looking at editor and reviewer timing charts to gain a better understanding of their performance and time taken in peer review. We will look at these charts under a scenario where the editors invite reviewers and give their recommendations on a manuscript to the Editor-in-Chief, who makes the final decisions.The first timing chart to give more information
Reporting on Editor Performance
Reporting on Editor Performance By Sherrie Hill & Jason RobertsFor a journal office, it is very important to understand how each of your editors is performing. Are all of your editors evaluating manuscripts in the same way? Are some of the editors slower than the bulk of the editors? Are some of your editors over or underutilized? In Origin Reports, we have developed a chart that presents all this information in both graphical and tabular form. The Editor Performance bubble chart in undoubtedly information dense. Indeed, it may seem daunting initially. There are three important points to remember to help break down the data:The size of the bubble relates to the number of papers handledThe x-axis shows the mean number of days it takes for a given editor to handle a paper from assignment through reviewer selection to the delivery of either a recommendation or decisionThe y-axis holds information on the rejection rate.To simplify the chart, this illustrative example uses the Editor Name filter to limit the information displayed to only show four editors. Looking at the y-axis (Rejection Rates) we see that editor Chen has the highest rejection rate of the four editors and rejected all the manuscripts that were assigned
Using Journal Timing Charts to Improve Manuscript Flow
Using Journal Timing Charts to Improve Manuscript Flow By Sherrie Hill and Kristen Overstreet January 19, 2022 It’s time to consider what types of information we will present to our boards at upcoming editorial board meetings. Two charts that are usually included are time to initial decision and time to final decision. These excellent charts give insight into the performance of a journal over a certain time period. Our editorial boards often ask, “How can we improve these times?” <img src="https://images.squarespace-cdn.com/content/v1/575e4e9be707eb86c44df175/7e4f1dd0-b085-4544-a6d0-5e5697b56a19/Figure+1+1-19-22.jpg" alt="" /> <img src="https://images.squarespace-cdn.com/content/v1/575e4e9be707eb86c44df175/05a36488-fa7e-4547-b21e-4c60d6835c85/Figure+2++1-19-22.jpg" alt="" /> In order to answer their question, we must first understand our data. Thus, let’s first consider the time to initial decision chart. In this chart, we are looking at the time from submission of a manuscript to the time an Editor makes a decision. During this time frame, the submission may be handled by the editorial office, the Editor-in-Chief, an associate editor and/or a section editor and/or a methods editor and or a statistical editor, and by peer reviewers. There are many components in the time to initial decision. For a journal to make a significant improvement in their time to initial decision, a breakdown by stage must be examined. Example Peer Review Workflow In our example peer review workflow, the
A Date is a Date is a Date…. Maybe Not
By: Sherrie Hill and Kristen OverstreetMarch 30, 2022When you create charts to present your journal data, it is important to note which of the many possible dates you used. For example, when creating a chart to show initial decisions, are you using the submission date or the date of the initial decision as your reference date?Does it really matter?Actually, the reference date you select is very important because the date determines the way that you will group the submissions. If you were tasked with putting apples into groups and counting the apples in each group, you can easily see how different your counts would be depending on how you set up your groups. If you grouped the apples by color, you would get different counts than if you grouped them by apple variety or by size. The dates that we use to group our submission data works the same way. If we group all the manuscripts by submission date, we will get different counts than we would if we used different dates, such as initial decision date, final decision date, date the editor was assigned, or date the reviewer was invited. So, what reference date should you select when creating
Diving Deeper into Your Submission Data
Diving Deeper into Your Submission Data By Sherrie Hill and Kristen OverstreetDecember 23, 2021As we approach the end of the year, editorial office staff are beginning to think about their journals’ 2021 submission volume and how it compared to previous years. Submission volume for a journal can be a critical indicator of a journal’s health, but the actual submission number is just the surface information. We can dive deeper into the data and find trends that can be used to steer a journal in new directions.The trend of the submission volume is of particular interest. Everyone wants to know if the submissions are trending upward, staying relatively flat, or decreasing. However, keep in mind any special circumstances that occurred in that year. Were there an unusual number of special issues? Were more conference presenters invited to submit manuscripts? Or were all conferences for that year canceled due to COVID? In 2020, did your journal experience a significant increase in submissions due to researchers having more time to write because they were working from home or was there a significant decrease because clinical researchers were working overtime on COVID-related issues? It is imperative to keep a log of significant events and changes
The Importance of Chart Parameter Lists
The Importance of Chart Parameter ListsBy: Sherrie Hill & Kristen OverstreetWhy should you show the parameters on your charts? Because it is a way to document how the chart was created so that it can be correctly interpreted and reproduced.Consider the three charts above. Though all three charts were created from the same data set using the same date range, they appear to show very different results. The reason for this difference is that filters were applied to the bottom two charts to allow the editorial office to answer specific questions.Interpretation issues can result from producing charts without listing the parameters used to create them. Firstly, it is not possible to tell if the last data point shown on the charts above represents a full year of data or only a partial year. This can confuse any possible trends, since some data are very sensitive to influences of partial years, such as the submission volume, the number of accepted manuscripts, the reviewer response to invitation conversion rates, etc.If any filters were applied, that should be recorded directly on the chart. Many people believe it is sufficient to note the filters used to create the chart within the written report. Unfortunately,
The How and the Why of Collecting Identity Data: Part 2
The How and the Why of Collecting Identity Data: Part 2 By Sherrie Hill October 21, 2021 During this year’s Peer Review Week, there was a lot of conversation about encouraging diversity in authors, reviewers, editors, and editorial boards, which led to conversation about the importance of ensuring collection of accurate information, respect of all people being polled, and data privacy. To learn more about best practices for collecting identity data, I attended several of the sessions about Diversity, Equity, and Inclusion at the ISMTE 2021 Global Virtual Event . I found the session “Using Data to Promote DEI: You Can’t Improve What You Don’t Measure,” presented by Anna Jester from eJournal Press and Dr. Teodoro Pulvirenti from the American Chemical Society, to be very insightful in regard to considerations for collecting identity data. I also read the Scholarly Kitchen blog post by Katie Einhorn, Steph Pollock, and Nick Paolini, titled “Guest Post – Introducing Demographic Questions during Manuscript Submission at the American Psychological Association”, which provided additional items for consideration. I have described below what I learned and the additional questions raised. Why We Want the Data Many journals would like to increase the diversity of their stakeholders (authors, reviewers, editors, editorial board, etc.), and
