In anticipation of Peer Review Week 2020, and in consideration of the theme ‘Trust in Peer Review’, Delta Think surveyed broadly across a two-week period in August to determine whether COVID-19 has impacted perceptions of preprints. The survey was open to everyone – from Publishers to Librarians to Researchers to the Lay Public with an interest in scientific output.

We entered into the survey with open minds, though a few recurring themes circulating in news outlets were on our minds including: the idea that both traditional journal submissions and articles posted to preprint servers have spiked in the last six months, owing to COVID-19; and the hunch that perhaps increased traffic indicated a correlated spike in trust, or conversely, that challenged findings making their way into mainstream news might have reduced trust. Was the reputation associated with preprints in jeopardy? Or could they play a critical role in speeding up science at a time so critical for the global health community? With these early loose hypotheses and questions, we launched the survey.

We explore the survey results below. In the spirit of transparency, we have posted the underlying data to Zenodo. It’s also important to note that while this was a quick survey, and is not meant to include a representative sample from the participating audiences, it provides interesting top-level findings and points to areas that may be ripe for a further investigation or a deeper dive in future.

Survey Population Demographics

In total, we collected 371 responses to the survey. Of this population:

  • Over 72% identified as either a Publisher, Librarian or Researcher
  • A majority of responses came from North America (over 55%)
  • Second-highest participation from Europe and Central Asia (25%)
  • East Asia and Pacific and Rest of World account for close to 20% of responses

Of this multi-disciplinary population:

  • Nearly 85% work in Biological Sciences and Medicine
  • 53% are affiliated with Engineering, Physical Sciences, or Chemistry
  • Over 40% claimed association with Arts & Humanities and Social Sciences

Note: Respondents could select multiple disciplines.

We limited the survey to respondents who were already familiar with preprints by offering an initial question to gauge familiarity. Those who were unfamiliar exited the survey at this point (6% of those starting the survey).

Trust Balances

When it comes to the question of trust in preprints, the average trust level came in almost exactly at the midpoint at 51 out of 100, where 100 indicates the highest trust and 0 indicates no trust. But a few notable variances stand out in the details. Taking a closer look at Authors vs. Non-Authors, we find Authors overall more trusting of preprints, and if we look by role, those who identified as Researchers are the most trusting group of all. Publishers, on the other hand, are the least trusting.

Figure 1: All responses to, “How much do you trust the findings presented in a Preprint manuscript?”

COVID-19 Impact on Behavior & Trust

Next we asked how the COVID pandemic has changed their perspectives on preprints. About 60% of all respondents reported that their trust has remained unchanged despite COVID-19. Still, Non-Authors were more likely than Authors to report a decrease in trust, and Publishers were more likely than Librarians and Researchers to report a decrease in trust.

Looking at specialties, Arts & Humanities and Social Science affiliated respondents were slightly more likely to report a trust increase, while Engineering, Physical Science & Chemistry, and Biological Sciences and Medicine were more likely to report a decrease in trust.

Figure 2: Specialty responses to: “In light of the COVID-19 pandemic, has your trust of Preprints increased, decreased, or remained the same in the last year?”

When asked why their trust increased, those with increased levels primarily reported that speed became more important than pre-publication peer review.

When respondents reported that trust had decreased, overall they pointed to a perceived quality problem and worried that many preprints being published now do not present well-constructed studies or findings. They also noted that traditional pre-publication review is necessary now more than ever.

Overall Benefits of Preprints

The greatest benefits of preprints, according to all respondents, are:

  • Speed
  • Openness
  • Connection to facilitating open peer review

A deeper dive into filters by author status, specialty, role, and region mostly showed speed and openness as the top two benefits. Different groups, however, picked various options as their third most important benefit. Authors wanted increased visibility for their research; Engineering, Physical Sciences & Chemistry chose increased cooperation in research across labs; Biological Sciences and Medicine wanted to facilitate open peer review.

Some interesting findings appeared in the regional breakouts as well. Those from the Rest of World countries selected increased visibility of their research as well as the researcher as the #1 benefit, and East Asia appreciated that it was cost-efficient or free more so than other regions.

Figure 3: Region specific breakdown of top Preprint benefits
(respondents could chose up to three)

Conclusion

Nearly seven months into the COVID-19 pandemic, trust in Preprints mostly holds steady for the majority of respondents to this survey, although respondents in Engineering, Physical Science & Chemistry, and Biological Sciences and Medicine had higher levels of doubt, and those in Arts & Humanities and Social Sciences were more likely to express an increase in trust.

And overall, Authors & Researchers were the most positive about preprints of all the groups.

Shoring up preprint review quality, increasing preprint reviews, and increasing awareness of checks already in place on preprint servers, could go a long way toward increasing trust in preprints and growing activity. While the speed and openness of preprints emerge in the data as clear, universally agreed benefits, improving review seems the prudent path forward toward enticing the research community to wholeheartedly embrace these benefits.

Delta Think wishes to thank all those who participated in the survey. For each survey completed, we donated $5 to Project HOPE, up to $1,000.


This article is © 2020 Delta Think, Inc. It is published under a Creative Commons Attribution-NonCommercial 4.0 International License. Please do get in touch if you want to use it in other contexts – we’re usually pretty accommodating.

TOP HEADLINES

Copyright Clearance Center Launches Open Access Workflow Services – September 16, 2020

“Copyright Clearance Center, Inc. (CCC) announces Open Access Workflow Services, a comprehensive consulting practice providing strategic Open Access (OA) and Transformative Agreement workflow support to publishers, funders, institutions and other key stakeholders in the scholarly communications ecosystem.”

New Springer Nature white paper analyses geographical diversity and usage of OA books – September 10, 2020

“Open access (OA) books are reaching more countries and have greater usage and higher citation numbers than non-OA books. A new analysis collaboratively produced by Springer Nature and COARD (Collaborative Open Access Research & Development) presents these and other key findings in a new white paper that explores how OA affects the geographical diversity of readers.”

New white paper explores how the transition to Open Access could be made more equitable for the Global South – September 7, 2020

“Given an increasing focus on UN Sustainable Development Goals (SDGs) and a global transition in scholarly publishing models towards Open Access, a newly published white paper: ‘Achieving an equitable transition to open access for researchers in lower and middleincome countries’ examines how the playing field can be levelled from the authorship side also, permitting knowledge production in lower and middle-income countries (LMICs) to find a wider audience.”

Theme of 2020 Open Access Week to be Open with Purpose: Taking Action to Build Structural Equity and Inclusion – August 31, 2020

“The 2020 Open Access Week Advisory Committee is pleased to announce that the theme for this year’s International Open Access Week, to be held October 19-25, will be ‘Open with Purpose: Taking Action to Build Structural Equity and Inclusion.’ … 2020 marks the third consecutive year the theme for International Open Access Week will focus on the urgent need for action on equity and inclusion, underscoring the urgency of continuing to center this work.”

SPARC Europe to launch large-scale Open Education initiative – August 27, 2020

“SPARC Europe is poised to embark on a new, two-and-a-half-year programme to grow Open Education capacity among libraries of higher education in Europe. The project will allow us to gain a better understanding of the European Higher Education library community’s needs and progress in OE, to share knowledge across countries, and to advocate for more OER, OE policy and open infrastructure.”

Springer Nature’s Transformative Agreements enable country level transition to open access – August 25, 2020

“Transformative Agreements (TAs) continue to have an influential and driving impact on enabling a country to flip to a fully open access (OA) environment sustainably and effectively. The latest data from Springer Nature on the growth of OA publishing as a result of its national TAs illustrates that any country with an agreement in place with the publisher, has the ability to flip to full OA.”

The Open Research Library Reaches New Milestones, Seeing Growth in Content, Usage and Accessibility Features – August 24, 2020

“The Open Research Library (ORL), the hosting platform that makes scientific book publications and other high-quality academic Open Access (OA) content freely available to anyone, anywhere in the world, has made great strides since its official launch in January 2020.”

OA JOURNAL LAUNCHES

September 16, 2020

De Gruyter transforms journal “Bibliothek Forschung und Praxis”to Open Access via Subscribe to Open Model

“Together with the editors and the National Contact Point Open Access OA2020-DE, De Gruyter is transforming the leading library journal Bibliothek Forschung und Praxis (BFP) into open access through the Subscribe to Open model. After a successful transformation, all journal content will be available immediately in gold open access on degruyter.com from 1 January 2021.”

September 16, 2020

Karger Publishers Expands Open Access Program

“The Switzerland-based company has flipped (converted) six subscription journals to the Gold OA model and founded a new OA journal… The newly OA journals are: Intervirology, Ophthalmic Research, Cardiorenal Medicine, Microbial Physiology (formerly Journal of Molecular Microbiology and Biotechnology), Public Health Genomics and Transfusion Medicine and Hemotherapy. In addition, Karger has launched a new OA journal, Glomerular Diseases.”

September 10, 2020

Oxford University Press launches third title in new Oxford Open series, Oxford Open Climate Change

“Oxford University Press (OUP) has today launched the third title in its pioneering new open access journal series with Oxford Open Climate Change. Oxford Open Climate Change is a wide-reaching interdisciplinary journal that aims to cover all aspects of climate change.”

September 8, 2020

Duke University Press Converts Flagship Journal Demography to Open Access

Demography, the flagship journal of the Population Association of America (PAA), will become open access in 2021 as it joins the Duke University Press journals publishing program.”

September 3, 2020

New Open Access Peer-Reviewed Journal Telemedicine Reports Now Accepting Manuscripts

Telemedicine Reports, a new peer-reviewed open access companion to the highly regarded Telemedicine and e-Health, is officially open to receive manuscript submissions. All articles in Telemedicine Reports will be published open access under the CC-BY and CC-BY-NC licenses to ensure broad and rapid distribution and visibility of meaningful telemedicine and telehealth research.”

August 28, 2020

AAAS announces new Science Partner Journal, Space: Science & Technology

“The American Association for the Advancement of Science (AAAS) is pleased to announce the launch of new Science Partner Journal, Space: Science & Technology, published in affiliation with the Beijing Institute of Technology (BIT)… Space: Science & Technology is currently open for submission and will publish under a Creative Commons Attribution License (CC BY).”

August 26, 2020

Announcing a New Journal: Philosophy of Medicine

Published by the University of Pittsburgh, “the mission of Philosophy of Medicine is to serve as the flagship journal for the field by advancing research in philosophy of medicine, by engaging widely with medicine, health sciences and the public, and by providing open-access content for all.”

August 25, 2020

New subscription model may make cell biology journal open access

“The American Society for Cell Biology (ASCB) is launching a new library subscription model for its peer-reviewed basic research journal—Molecular Biology of the Cell—making it one of the first research journals to experiment with Subscribe to Open (S2O). With S2O, all MBoCreaders may have open access in 2021, while the author’s cost will remain low.”