Fast Facts:
Identity data from authors, reviewers, editors, and the editorial board can be beneficial to journals as they attempt to ensure that underrepresented groups are adequately represented.
Before collecting any identity data, your journal should be clear about the goals for the data collection.
Collected data should be anonymized for storage and distribution.
The How and the Why of Collecting Identity Data: Part 2
October 21, 2021
By: Kristen Overstreet
During this year’s Peer Review Week, there was a lot of conversation about encouraging diversity in authors, reviewers, editors, and editorial boards, which led to conversation about the importance of ensuring collection of accurate information, respect of all people being polled, and data privacy.
To learn more about best practices for collecting identity data, I attended several of the sessions about Diversity, Equity, and Inclusion at the ISMTE 2021 Global Virtual Event . I found the session “Using Data to Promote DEI: You Can’t Improve What You Don’t Measure,” presented by Anna Jester from eJournal Press and Dr. Teodoro Pulvirenti from the American Chemical Society, to be very insightful in regard to considerations for collecting identity data. I also read the Scholarly Kitchen blog post by Katie Einhorn, Steph Pollock, and Nick Paolini, titled “Guest Post – Introducing Demographic Questions during Manuscript Submission at the American Psychological Association”, which provided additional items for consideration. I have described below what I learned and the additional questions raised.
Why We Want the Data
Many journals would like to increase the diversity of their stakeholders (authors, reviewers, editors, editorial board, etc.), and they need to start with the questions, “What do they plan to do with the data and what problems are they trying to solve?” According to Dr. Pulvirenti, the American Chemical Society (ACS) is hoping to invite new voices and perspectives to their editorial boards. They would also like to increase visibility of underrepresented groups. According to Einhorn, Pollack and Paolini, authors of the blog post mentioned above, the American Psychological Association (APA) is concerned that groups such as Black, Indigenous, and People of Color (BIPOC), women, gender non-conforming people, members of the LGBTQ+ community, and members of the disability community have been excluded from scholarly publications in the past, which “is harmful to both the integrity of the scientific review and publication process”, as well as to the careers of these people.
Determining What Information to Collect
Before collecting data, you need to determine what you want to discover about the current state of your journal. Are you trying to determine if your editorial board has racial diversity? Do you want to know if your journal is attracting young professionals or whether it has an adequate number of expert reviewers? Do you want to make sure that people of different gender identities are equitably represented in your reviewer pool? Do you want to determine if your journal has a global reach? By establishing your goals for the collected data, you will be able to create a list of needed identity data points, determine which groups within your organization or journal community you are going to survey, and what questions you will need to ask.
First, Do No Harm
Identity data is sensitive and needs to be collected in a way that is transparent and also respectful to the respondents; stored where it cannot be accessed by anyone, ever, other than those involved in the data collection, in the way that was explained to the respondents; used only in the way and for the purpose(s) that was described to the respondents; and reported only in the aggregate. No one reading the results should ever be able to identify the person who provided the data. Data should be anonymized for storage and the original surveys destroyed if they point to the specific person who provided the information. Respondents should always have the option not to answer a question and /or not to complete the survey. The place where the data are stored must be secure and require at least one level of password protection.
How to Use the Collected Data
After determining why you want the data and what specific data to collect, you can set short- and long-term goals for how you will use the data. These goals should be determined before sending out the survey and they should be explained to those who are being asked to complete it. The data should not be used for any purposes not explained to the survey participants.
Here are some examples for goals for how you could use the data (from the American Chemical Society and American Psychological Association):
- Develop more inclusive language to be used throughout the submission and peer-review process.
- Benchmark where you are and set goals to develop more representative editorial boards and author pools.
- Identify potential biases in the peer review process by assessing data such as acceptance and rejection rates for authors with different identity characteristics.
- Expand the reviewer pool by recruiting people from underrepresented groups.
- Determine if topics related to underrepresented groups are more likely to be rejected because of the underrepresentation of those groups among the reviewers and/or editors.
- Gain information to train editors, reviewers, and staff to recognize bias in their own and others’ behavior.
- Develop a broader marketing strategy to reach individuals in underrepresented groups.
Once you have your data, your organization or journal should be prepared to report out on the aggregate data collected and goals set / improvements made as a result of the information. This ensures not only a transparent process, but may also encourage fuller participation in future surveys.
In our next post, we will focus on the process of designing the survey to collect the data. We would love to hear your thoughts and questions on these topics.
