Covid-19 Resources

These Covid-19-related resources are freely available. 

Compiled from the WAME Newsletter. Providing the links does not imply WAME's endorsement. Updated March 19, 2024. Latest content links are preceded by ►. 

Selections from recent news: 

• Early COVID-19 research is riddled with poor methods and low-quality results − a problem for science the pandemic worsened but didn’t create": but is convenience sample science better than no science when a new deadly disease affects the entire world? 

---

• FREE RESOURCES
• JOURNALS, PREPRINTS, AND SPEED OF PUBLISHING
• ACADEMIA AND RESEARCH
• MAY BE OF INTEREST (payment required)

---

 FREE RESOURCES

• Retracted covid-19 research from Retraction Watch

• World Health Organization Covid-19 courses, also available in national languages 

• US Centers for Disease Control and Prevention Covid-19 database of research articles

• US Centers for Disease Control and Prevention "Isn’t Publishing Large Portions of the Covid Data It Collects...the pandemic exposed the fact that data systems at the C.D.C., and at the state levels, are outmoded and not up to handling large volumes of data. C.D.C. scientists are trying to modernize the systems…public health officials in Scotland said they would stop releasing data on Covid hospitalizations and deaths by vaccination status because of similar fears that the figures would be misrepresented by anti-vaccine groups."

• “The COVID-19 Catalog of Training Resources for Health Care Workers is a comprehensive library of around 300 existing training resources developed or made available by various organizations” (hat tip to HIFA) 

• WHO has published an interactive timeline of its response to COVID-19 (hat tip to HIFA)

• Evidence Aid offers Covid-19 collection

• COVID-19 Open Research Dataset (CORD 19) 

• Covid-19 Data Portal, from the European Bioinformatics Institute (EMBL-ELI)

• Cochrane Library
• Hesperian health guide on coronavirus in 27 languages 
• Covid-19 advice for patients in 21 languages from Doctors of the World UK
• Johns Hopkins Coronavirus Resource Center
• US National Institutes of Health Treatment Guidelines
• UpToDate: free access to covid-related content (registration required)
• Clinical resources from American College of Physicians and Dynamed
• Electronic Therapeutic Guidelines (eTG) from Australia's Therapeutic Guidelines International Program : complimentary access available
• UK National Institute for Health and Care Excellence (NICE) Covid-19 rapid guidelines and evidence summaries
• CoVis: a curated knowledge map of seminal works on COVID-19”
• Introducing Rapid Reviews: Covid-19: Massachusetts Institute of Technology creates a new journal “using artificial intelligence tools to rapidly compile scientific findings for open review and sharing. It “will cull articles from preprint formats, add peer reviews and openly share the results regardless of whether the process ends in a traditional decision to publish.”
• US National Academy of Medicine coronavirus resources
• Many subscription journals and publishers have made all Covid-19 content free and created hubs of free access Covid-19 research content, eg, Elsevier, Wiley, Springer Nature, BMJ, Wolters Kluwer, JAMA Network
• MedRxiv, BioRxiv, and ResearchSquare post Covid-19 preprints
• "Free Public Data File of 112+ Million Crossref Records" "A lot of people have been using our public, open APIs to collect data that might be related to COVID-19...we have made a free data file of the public elements from Crossref’s 112.5 million metadata records."
• Johns Hopkins world map of Covid-19 cases and deaths
• Language diversity of areas affected by Covid-19
• US Covid preparedness scores, by state and county
• Covid-19 and Humanitarian Settings: Webinars from the READY Initiative, the London School of Hygiene and Tropical Medicine (LSHTM), the Center for Humanitarian Health at Johns Hopkins University and the Center of Education and Research in Humanitarian Action (CERAH)
• Covid-19 Conversations: Webinars from the American Public Health Association and the National Academy of Medicine

Free access to Covid-19 research: "94 academic journals, societies, institutes, and companies signed a commitment to making research and data on the disease freely available, at least for the duration of the outbreak."

“Joint Statement on transparency and data integrity International Coalition of Medicines Regulatory Authorities (ICMRA) and WHO… ICMRA¹ and WHO call on the pharmaceutical industry to provide wide access to clinical data for all new medicines and vaccines (whether full or conditional approval, under emergency use, or rejected). Clinical trial reports should be published without redaction of confidential information for reasons of overriding public health interest.”

UNESCO calls on representatives from 122 countries to "reinforce scientific cooperation and integrate open science in their research programmes to prevent and mitigate global crises"

Global officials call for free access to covid research

Global Health NOW (from Johns Hopkins Bloomberg School of Public Health) daily newsletter 

Institute for Global Health's “First Friday” webinar: Too Much, Too Crazy: A Pandemic, A Presidential Election, and Global Health Reporting

“Modeling COVID-19 scenarios for the United States… Universal mask use could save an additional 129,574 (85,284–170,867) lives from September 22, 2020 through the end of February 2021, or an additional 95,814 (60,731–133,077) lives assuming a lesser adoption of mask wearing (85%), when compared to the reference scenario.” However, rates of mask wearing had already increased by the time is was published, plus a paywalled critique in Wall Street Journal

“Let’s Do Better: Public Representations of COVID-19 Science…we conclude with ten recommendations aimed at key actors involved in the communication of COVID-19 science, including government, funders, universities, publishers, media and the research communities.” From the Royal Society of Canada Task Force on COVID-19

“[US Centers for Disease Control] officials describe intense pressure, job threats from Trump White House...the Trump White House and its allies repeatedly ‘bullied’ staff, tried to rewrite their publications and threatened their jobs in an attempt to align the CDC with the more optimistic view of the pandemic espoused by Donald Trump."

---

 JOURNALS, PREPRINTS, AND SPEED OF PUBLISHING

►"Early COVID-19 research is riddled with poor methods and low-quality results − a problem for science the pandemic worsened but didn’t create": but is convenience sample science better than no science when a new deadly disease affects the entire world?

"Does haste make waste? Prevalence and types of errors reported after publication of studies of COVID-19 therapeutics...Only one erratum contained a serious error: a typographical error which could have misled readers into believing that the treatment in question had serious adverse effects when in fact it did not."

"The effects of communicating uncertainty around statistics on public trust...communicating uncertainty around present COVID-19 statistics in the form of a numeric range (versus no uncertainty) may lead to slightly lower perceived trustworthiness of the number presented but has no impact on perceived trustworthiness of the source of the information...[T]his minimal impact of numeric uncertainty on trustworthiness is also present when communicating future, projected COVID-19 statistics (Study 2; N = 2,309). Conversely...statements about the mere existence of uncertainty, without quantification, can reduce both perceived trustworthiness of the numbers and of their source."

►"Plagiarism among COVID-19 articles in infectious disease journals between 2020 and 2021...Similarity reports were manually checked for events of true plagiarism using an 80% threshold, performed via human judgment...41.61% (n = 129) of manuscripts were deemed plagiarized out of a total of 310 papers that were analyzed. Plagiarism was identified in 35.07% of reviews (n = 47), and 46.6% of original research (n = 82)...The discussion section of these articles was the most problematic area, with the average number of copied sentences in that section being 6.25 ± 10.16. The average time to judge all manuscripts was 2.45 ± 3.09 min. Among all the plagiarized papers, 72.09% belonged to papers where the similarity report was ≤ 15% (n = 93)."

"Research integrity during the COVID-19 pandemic: A book excerpt... [W]e would like to review the COVID-19 pandemic through the prism of the 3R’s of research integrity: rigor, reproducibility, and responsibility...What could have been done better? What needs to improve?"

"Publisher looking into COVID vaccine paper with ‘serious flaws’...A controversial paper on the safety and immunogenicity of an Iran-made COVID-19 vaccine is being investigated by the U.S.-based publisher Wiley... But in a commentary published in May 2023, in the same journal, Donald Forthal, chief of the University of California, Irvine’s Division of Infectious Diseases, expressed surprise that 'a manuscript containing so many serious flaws would have been accepted for publication following peer review, and given these issues, a retraction may be in order.' ”

"Science corrects two 2022 papers about the origins of COVID-19, one by Worobey et al and the other by Pekar et al." [Retraction Watch]

"Highly cited Lancet long COVID study retracted and republished...The retraction and replacement notice describes the steps the journal editors took in their investigation, which involved statistical review of the data changes the authors requested, a clinician’s review of the differences between the 6-month and 1-year follow-up questionnaires, review of the Chinese versions of the questionnaires by The Lancet’s China editors, and asking the China-Japan Friendship Hospital, with which corresponding author Bin Cao is affiliated, to verify the corrected data. All aspects of the investigation turned out in favor of the authors."

►The unfortunate rewards of spreading misinformation on social media: False information about vaccines and climate change, generative AI, and the widening gap between Democrats and Republicans in trusting scientific experts to make policy decisions were addressed in a 3 day conference "organized by the Nobel Foundation and the National Academy of Sciences."

"How covid-19 bolstered an already perverse publishing system...Medical journals halved their turnaround times in the first half of 2020...Rubin says that NEJM’s editors had to abandon their usual full checks and balances in the acute phase of covid article triage and were publishing things they normally wouldn’t, including case reports, small trials, and uncontrolled studies...More threatening to people’s trust in journals were high profile retractions...while major publishers including Elsevier, Springer Nature, and Wiley continue to make covid content freely available, only about half of papers on the climate crisis are similarly available...much of the bad behaviour in the eagerness to publish, especially in top journals, is down to incentives and a lack of oversight in academic promotion systems."

"Retracted Covid-19 articles: significantly more cited than other articles within their journal of origin...The retracted articles were cited an average of 44.8 times, which was significantly higher than the average CiteScore (p = 0.01). Between June and November, retracted COVID-19 articles gained a total of 728 new citations, presence of “withdrawn” or “retracted” before article title did not affect citation rates. COPE guidelines for retraction statements were not met for 32% of articles."

Frontiers issues Expression of Concern on Microbiome-Based Hypothesis on Ivermectin’s Mechanism in COVID-19: Ivermectin Feeds Bifidobacteria to Boost Immunity: “Starting in December 2022, commenters on PubPeer posted questions about the paper, such as why it referred to data that it did not appear to provide, and missing or inappropriate citations...As is typical for Frontiers, but not considered best practice, the original article page does not link to the expression of concern or otherwise indicate it exists.” (ICMJE and others recommend that Expressions of Concern be linked to the original article.)

►Editorial decision time before and after onset of the COVID-19 pandemic: “We obtained the publication history of 339,199 papers submitted between January 1st, 2019 and June 1st, 2021 from Elsevier." Editorial decision time was "measured by the difference in dates between the revised date and the accepted date of a paper...the pandemic significantly decreases the editorial decision time of revised papers by 8.9 days on average...editors of higher-tier (than lower-tier) journals were more likely to be entitled to the increased editorial speed...editors of social science journals experienced a larger increase in editorial speed than editors of science journals...Coinciding with the policy relaxation, the editorial decision time gradually returned to the pre-COVID-19 level."

“How did the scientific publication system respond to the Covid-19 pandemic?...Concerning review quality, some extremely dubious articles on Covid-19 treatment managed to be published, some of which were in relatively prestigious journals. Regarding retractions (contrary to initial reports), no increase in retractions rate was evident as time passed and the system reached its “steady state”. As for preprints, problems associated with their use are the practice of “retracting” (withdrawing) preprints with no adequate retraction standards and how to evaluate preprint quality effectively.”

"Publication of COVID-19–Related Articles Initially Presented as Preprints…We identified 3343 COVID-19–related preprints posted on medRxiv in 2020." As of October 2022, 1742 (52.1%) of those preprints had been "subsequently published in the peer-reviewed literature."

PLOS issues expressions of concern for 49 papers by Didier Raoult, “the French scientist who became controversial for promoting hydroxychloroquine for treating COVID-19…while it investigates potential research ethics violations in the work."

Environmental Research published a previously retracted paper about masks in children raising levels of inhaled carbon dioxide, without the previous retraction being mentioned (the study was retracted for questions about methodology and validity), despite being told by the authors it had been retracted. Important scientific issues in this case aside, shouldn’t a paper that has already been retracted cite the retraction and address the concerns if it’s republished? Are post-retraction republication guidelines needed?

"Journal editor in chief who published controversial Covid papers [including one claiming ‘billions of lives are potentially at risk’ with Covid-19 vaccines] resigns...The reasons for his resignation, as stated by Dr Domingo, are 'my personal frustration by the decrease in the Clarivate IF, and the unpleasant and serious problems generated on my daily mood as a consequence of the e-mails/Internet regarding documents (submitted or published in FCT) on the safety of the COVID-19 vaccines.'"

‘A very worrying scenario’: Internal documents on India Covid-19 vaccine raise troubling questions about approval process…regulators endorsed the vaccine, called Covaxin, despite discrepancies in the number of clinical trial participants. Moreover, questionable changes were made to the trial protocols — which are established procedures for testing a vaccine or medicine — to expedite the approval process…the number of people enrolled in the Phase 1 portion of the trial differed from what was later published in a medical journal. There were also important changes made to the protocol for Phase 2 testing, when immunogenicity data from the previous Phase 1 stage were not yet available.”

Should preprints be included in meta-analyses? “Consistency of covid-19 trial preprints with published reports and impact for decision making: retrospective review...Apart from two (3.3%) of 60 comparisons, point estimates were consistent between meta-analyses including preprints versus those excluding preprints as to whether they indicated benefit, no appreciable effect, or harm. For nine (15%) of 60 comparisons, the rating of the certainty of evidence was different when preprints were included versus being excluded—the certainty of evidence including preprints was higher in four comparisons and lower in five comparisons.”

“Transparency and reporting characteristics of COVID-19 randomized controlled trials...Transparency was poor. About half of trials were prospectively registered (n = 140, 56%); 38% (n = 95) made their full protocols available, and 29% (n = 72) provided access to their statistical analysis plan report. A data sharing statement was reported in 68% (n = 170) of the reports of which 91% stated their willingness to share. Completeness of reporting was low: only 32% (n = 81) of trials completely defined the pre-specified primary outcome measures; 57% (n = 143) reported the process of allocation concealment. Overall, 51% (n = 127) adequately reported the results for the primary outcomes while only 14% (n = 36) of trials adequately described harms. Primary outcome(s) reported in trial registries and published reports were inconsistent in 49% (n = 104) of trials; of them, only 15% (n = 16) disclosed outcome switching in the report. There were no major differences between preprints and peer-reviewed publications. Of the 130 RCTs published as preprints, 78 were subsequently published in a peer-reviewed journal. There was no major improvement after the journal peer review process for most items.”

Citing retracted papers still a problem: “The 212 retracted papers [on Covid-19] investigated were cited 2697 times, a median of seven times per paper. A quarter of these retracted papers reported clinical findings relevant to patient care – almost 90% of citations of these papers referenced the retracted paper without mentioning it had been retracted, and 80% were published after the retraction.”

Differences between Covid-19 and non-Covid-19 retractions: speed of retraction, study design (Covid-19 more likely to be modeling studies), and reason for retraction, with non-Covid-19 studies more likely to be “initiated by editors, journals, or publishers than COVID-19 studies.” Covid-19 first and last authors had more years from their first publication to their retraction.

Covid-19 preprints with high Altmetric scores are more likely to be published in a year and by high impact journals, so does a study being published in a high-impact factor journal make it highly cited, or is the journal more likely to receive the studies with high citation potential and publish them? This study implies the latter – the studies make the journal’s high impact factor.

Authors submit a rebuttal of Covid-19 mRNA research claims to the publishing journal, which ultimately rejects it. What next? Another journal: “Here, we describe why this article, which contains unsubstantiated claims and misunderstandings such as ‘billions of lives are potentially at risk’ with COVID-19 mRNA vaccines, is problematic and should be retracted. We report here our request to the editor of [Food and Chemical Toxicology] to have our rebuttal published, unfortunately rejected after three rounds of reviewing. Fighting the spread of false information requires enormous effort while receiving little or no credit for this necessary work, which often even ends up being threatened. This need for more scientific integrity is at the heart of our advocacy, and we call for large support, especially from editors and publishers, to fight more effectively against deadly disinformation.”

"Journal says ivermectin study met standard for ‘credible science’" but two authors already had ivermectin studies retracted

“Lessons from the COVID-19 pandemic and recent developments on the communication of clinical trials, publishing practices, and research integrity: in conversation with Dr. David Moher...[CONSORT and other similar initiatives] 'have helped a little ... the evidence is that there are still about 50% of reports of randomized trials that tell you nothing about the randomization process,' a less exciting reality, 'I think we still have a way to go.'...Dr. Moher highlights the paper by London and Kimmelman in which the authors discuss “pandemic research exceptionalism” and how the pandemic has paved the way for questionable research practices at the expense of rigorous, high-quality research [12], which ultimately affects patients and health systems [12]. He also cites work by Goldacre et al. highlighting selective reporting in 58 of 67 trials published during a 6-week period in prominent journals that endorse CONSORT [13]. The authors found that over 300 new, non-prespecified outcomes were silently added to the final trial reports [13]. In another study based on 262 trials reported in the most prominent oncology journals, only 11% reported all 10 essential elements about the intervention (e.g., drug name, dose, route)."

"Reliability of citations of medRxiv preprints in articles published on COVID-19 in the world leading medical journals...Among the 205 research articles on COVID published by the four major medical journals in 2020, 60 (29.3%) cited at least one medRxiv preprint. Among the 182 preprints cited, 124 were published in a peer-reviewed journal, with 51 (41.1%) before the citing article was published online and 73 (58.9%) later. There were differences in the title, the data or the conclusion between the preprint cited and the published version for nearly half of them. MedRxiv did not mentioned the publication for 53 (42.7%) of preprints...More than a quarter of preprints citations were inappropriate since preprints were in fact already published at the time of publication of the citing article, often with a different content. Authors and editors should check the accuracy of the citations and of the quotations of preprints before publishing manuscripts that cite them."

“Comparative analysis of retracted pre-print and peer-reviewed articles on COVID-19…
The median (IQR) retraction time for pre-print and peer-reviewed articles was 29 (10-81.5) days and 139 (63-202) days (p = 0.0001).” [preprint]

How expedited were Covid-19 papers? "We found that when the early bird effect [‘extremely short publication delays for scientific manuscripts on new and ‘hot’ topics at the outset of a newly emerging event’] and data incompleteness are taken into account, COVID-19 related research papers show only a moderately expedited speed of dissemination as compared with the pre-pandemic era. Medians for peer-review and production stage delays [ie, turnaround time] were 66 and 15 days, respectively, and the entire conversion process from a preprint to its peer-reviewed journal article version took 109.5 days."

“We create a dataset of pre-prints posted to arXiv, bioRxiv, and medRxiv between 28 January 2020 and 30 June 2021 that are related to COVID-19. We extract the text from these pre-prints and parse them looking for keyword markers signaling the availability of the data and code underpinning the pre-print. For the pre-prints that are in our sample, we are unable to find markers of either open data or open code for 75% of those on arXiv, 67% of those on bioRxiv, and 79% of those on medRxiv.”

"Pivot into COVID-19 research eases as publishing surge starts to level off...the number of pandemic-related papers appears set to decline this year, after explosive and unprecedented growth in 2020 and 2021...In key disciplines such as infectious diseases and public health, the proportion of new papers devoted to COVID-19 appears to be flattening out...And in fields more distant from pandemic science, the share of COVID-19 papers is declining, suggesting researchers are returning to their core interests."

Covid-Vitamin D paper retracted after “randomization” turns out to be alternate allocation – why Methods details need to be clear before acceptance

“The chronic growing pains of communicating science online…The same profit-driven algorithmic tools that bring science-friendly and curious followers to scientists’ Twitter feeds and YouTube channels will increasingly disconnect scientists from the audiences that they need to connect with most urgently. Moving forward, conquering this challenge will require partnerships among the scientific community, social media platforms, and democratic institutions.”

“Publishing of COVID-19 preprints in peer-reviewed journals…How do Covid preprints “COVID-19-related preprints were more likely to be published within 120 days since the deposition of the first version (OR = 1.96, 95% CI: 1.80–2.14) as well as over the entire observed period (OR = 1.39, 95% CI: 1.31–1.48). Submission-to-acceptance was by 35.85 days (95% CI: 32.25–39.45) shorter for COVID-19 articles.”

“Tracking changes between preprint posting and journal publication during a pandemic…We assessed preprints from bioRxiv and medRxiv that had been posted and subsequently published in a journal through April 30, 2020, representing the initial phase of the pandemic response…We found that the total number of figure panels and tables changed little between preprint and published articles. Moreover, the conclusions of 7.2% of non-COVID-19–related and 17.2% of COVID-19–related abstracts undergo a discrete change by the time of publication, but the majority of these changes do not qualitatively change the conclusions of the paper.”

“[W]e have shown that the dramatic rise in COVID-19 publications was accompanied by a substantial decrease of non-COVID-19 research” publications.

Some authors of a covid meta-analysis on ivermectin, now retracted, “present a subgroup meta-analysis to assess the effects of stratifying by trial quality on the overall results. The stratification is based on the Cochrane Risk of Bias measures and raw data analysis where possible. The results suggest that the significant effect of ivermectin on survival was dependent on largely poor-quality studies. According to the potentially fraudulent study (risk ratio [RR], 0.08; 95% CI, 0.02–0.35), ivermectin improved survival ~12 times more in comparison with low-risk studies (RR, 0.96; 95% CI, 0.56–1.66). This highlights the need for rigorous quality assessments, for authors to share patient-level data, and for efforts to avoid publication bias for registered studies.”

“Ivermectin papers slapped with expressions of concern”: Two 2021 meta-analyses of ivermectin receive Expressions of Concern after a study incorporated in them is retracted. However, the studies receiving the Expressions of Concern don’t mention them. Do Expressions of Concern need to be flagged in a method similar to retractions?

“Rise of the preprint: how rapid data sharing during COVID-19 has changed science forever… A staggering 19,389 articles about COVID-19 were shared in the first four months of the pandemic, a third of which were preprints, unvetted and unfiltered for all to see.”

Authors of Covid preprint showing negative Omicron outcomes associated with vaccine "updating findings with totally different results"

“How bad research clouded our understanding of Covid-19…It was this team that investigated the paper, in the journal Viruses, that found that ivermectin was a highly effective treatment but that turned out to have a data set that was just the same 11 patient records copied over and over…The study authors said, “After revising the raw data we realised that a file that was used to train a research assistant was sent by mistake for analysis… ‘[Another] paper claims to describe a trial in which patients were randomly allocated to treatments. This is not true. Extreme differences are seen between groups across multiple variables such as oxygen level, blood pressure, and SARS-CoV-2 test results before they even got their first dose of medication.’.. One simple but effective test for data fabrication is whether numbers have a typical distribution of “trailing digits” — for example, whether they end in “1” about as often as they end in “0.” Humans, when making up numbers, tend to cluster around certain end digits. The randomized control trial from Iran fails this test, with very pronounced digit clustering to a degree that can’t have occurred by chance…”

"‘Science is flawed’: COVID-19, ivermectin, and beyond…Trials of favipiravir, another repurposed COVID-19 medication, have recently been retracted due to data concerns...There are now nearly a dozen studies looking at whether vitamin D has a benefit in COVID-19 that have been corrected or withdrawn entirely over the last 18 months."

“Potential Partiality in Scientific Publishing…We started exploring favoritism in research publications following up the hydroxychloroquine saga…Among articles supporting the use of hydroxychloroquine, a so-called meta-analysis was published in New Microbes and New Infections while the scope of this journal really does not match with therapeutic issues. So, we took a closer look at this journal and found that 35 percent of all articles were published by at least one author on the current editorial board…Looking at other authors of the meta-analysis, we found that Didier Raoult has signed [authored] 235 of the 728 articles published in New Microbes and New Infections, making him the most prolific author of this journal. Didier Raoult is not part of the editorial board, but as the director of the Institut Hospitalo-Universitaire Méditerranée Infection, six members of the editorial board ([including] the editor-in-chief and the deputy editor-in-chief) report to him.” The study on which this Q&A was based is available here.

Qualitative interviews with Australian “scientists and communicators” regarding scientific miscommunication: “Recommended solutions from within the scientific community included: rewarding research translation, encouraging standardised study design, increasing use of automated quality assessment tools, mandating study protocol registration, transparent peer review, facilitating wider use of open access and use of newer technologies to target public audiences. There was disagreement over whether preprints were part of the problem or part of the solution.”

 “Science misinformation alarms Francis Collins as he leaves top [US National Institutes of Health] job…’Somewhere along the way, our political hyperpolarization began having a lot of really dangerous consequences, where in many instances we seem to have lost a sense of how to tell the difference between a fact and an opinion — or some Facebook post that’s, frankly, a lie. That’s truly dangerous. That’s another epidemic that is not going to go away even if we triumph over COVID-19. We need to figure out what happened here, and how to bring ourselves back to a place where our nation has a more stable future.’ ”

Even journals are disseminating problematic research in preprints (tweets never were a good way to provide nuance and caveats)

Covid retractions are now up to 205, including a poorly conducted meta-analysis and an abstract

 What happened with the ivermectin studies? “Claims about the drug are based on shoddy science—but that science is entirely unremarkable in its shoddiness”

 “Ivermectin for COVID-19: addressing potential bias and medical fraud”

 “Inconsistent and incomplete retraction of published research: A cross-sectional study on Covid-19 retractions and recommendations to mitigate risks for research, policy and practice… Sources of articles post-retraction were preprint servers, ResearchGate and, less commonly, websites including PubMed Central and the World Health Organization.”

 “Gender gap in journal submissions and peer review during the first wave of the COVID-19 pandemic. A study on 2329 Elsevier journals… the pandemic could have exacerbated existing inequalities by imposing additional obstacles in terms of time and effort investment by women just as the demand for research was growing unprecedentedly.”

 Covid vaccine myocarditis lessons learned: “[A]nyone can make an error but there are several simple ways to minimize your chances of making one…plot the data…use your common sense… Respect the responsibility and accountability that comes with authorship.”

 How COVID-19 is Changing Open Access in Canada parts 1 and 2

Ivermectin “meta-analyses based on summary data alone are inherently unreliable”: One study, “the results of which represented more than 10% of the overall effect in at least two major meta-analyses,” with “several irregularities in the data that could not be consistent with them being experimentally derived,” has been withdrawn from the preprint server where it was posted. The other has “unexpected stratification across baseline variables in another randomized controlled trial for ivermectin6, which were highly suggestive of randomization failure.”

What is a journal’s responsibility to release information about covid? A peer reviewer learned of Covid’s risks before the rest of the world, asked (via the editors) the authors to notify the authorities, but “[a]n editor told him the researchers replied to the journal the next day, saying they would like to disclose their findings but could not do so without permission from the Chinese government.” What’s the remedy? “Many journals and research organizations, including The Lancet and the Chinese Center for Disease Control and Prevention, have signed a statement published by Wellcome in January 2020 that promised, among other things, to make newly submitted manuscripts available to WHO immediately.”

Did COVID change publishing? “[E]arly predictions that preprints would dominate the COVID-19 literature—and reshape all scientific publishing after the pandemic—so far appear incorrect. Through May, only about 5% of all peer-reviewed journal articles about COVID-19 papers started out as a preprint…” On the other hand: “Coates and Polka’s study found that 28.7 percent of COVID-19 preprints were cited in at least one news article, compared to 1 percent of preprints unrelated to the pandemic. They also discovered that preprints were cited in 52 of 81 COVID-19 policy documents from three large health authorities—the European Centre for Disease Prevention and Control (ECDC), the United Kingdom Parliamentary Office of Science and Technology (UK POST), and the World Health Organization (WHO).”

“WHO COVID-19 library contains hundreds of papers from hijacked journals…My review found 383 papers from three hijacked journals in the database that were imported from Scopus. Following a post on Retraction Watch about how hijacked journals duped Scopus, Scopus removed these papers, but they are still in the WHO library.”

Would rapid-review boards help speed peer review in a crisis? Perhaps, but who knows which reviewers will be needed for the next crisis?

"Huge [preprint] study supporting ivermectin as Covid treatment withdrawn over ethical concerns" including "numerous data errors, discrepancies and concerns."

“COVID-19 affects men and women differently. So why don’t clinical trials report gender data?... [O]ut of 45 COVID-19 randomized controlled trials whose results were published by December 2020, only eight reported the impact of sex or gender… If funding agencies or trial registries required sex-disaggregated reporting, that could motivate researchers to bake it into their trials.”

"Covid-19: Sputnik vaccine rockets, thanks to Lancet boost...Journals risk being used in place of regulators when they publish studies of novel vaccines that have not yet been authorised by a major regulator. Chris van Tulleken argues that peer review is inadequate to decide the risk-benefit ratio of new drugs”...”These obvious errors and the uncritically glowing editorials would be worrying enough under normal circumstances. But they are particularly concerning given that, at the time of publication of the phase III trial, no major regulator had even received an application for marketing. By comparison, every other covid-19 vaccine with a phase III trial published in a high impact journal had already been submitted to or authorised by a major regulatory authority...And the Lancet paper seems to have given other countries confidence. Before publication of the phase III trial, 16 countries had authorised Sputnik V for use—now, over 40 have authorised it...Most of the 40 countries using the vaccine ahead of EMA authorisation are low and middle income countries without well resourced, independent regulators. Understandably, because of the desperate global shortage of vaccines approved by a major regulator, they may have had no choice but to rely on the Lancet’s vetting of the science. But despite its international reputation, is the Lancet’s peer review process adequate for this?"

What were covid studies retracted for? “We examined 68 removed articles and determined that many of the articles were removed for unknown reasons (n = 22) or as duplications (n = 12); 24 papers were retracted for more significant reasons (data integrity, plagiarism, reporting or analysis, and IRB or privacy issues). The majority of removed papers were from the USA (n = 23) and China (n = 19).”

“Covid-19 and the evolution of scientific publishing…The experience that the editorial and review processes are gaining during the covid-19 era might be beneficial, as journals adopt best practices for the rapid, transparent, and thorough triage, review, and publication of manuscripts. The combination of accelerated review and social media platforms could result in true scientific dialogue—a back and forth between the authors and the readers mediated by journals.”

Systematic review vs meta-analysis? Why the distinction matters for mask recommendations

"Bibliometric analysis of highly disseminated COVID-19 research papers...This review of the most widely disseminated COVID-19 studies highlights a preponderance of low-quality case series with few research papers adhering to good standards of reporting."

From Retraction Watch: “John Ioannidis removes a criticized appendix in a COVID-19 paper.”

Open and Faster Scholarly Communication in a Post-COVID World...How has covid affected research communication?

“Let’s do better: public representations of COVID-19 science: we conclude with 10 recommendations aimed at key actors involved in the communication of COVID-19 science, including government, funders, universities, publishers, media, and the research communities.”

“Research integrity in the COVID-19 era: insights from Retraction Watch co-founder Ivan Oransky… there have been 87 retractions of COVID-19-related papers to date. That number isn’t all that different from what you would expect to see given the number of papers – and preprints – that have been published.”

“3 Ways the Pandemic Has Made the World Better: [T]he explosion of preprints is sometimes portrayed as the downfall of formal peer review. It’s the opposite. No process that allows more insight into how the sausage gets made can avoid a glimpse of its less tasteful elements, but what we need to change is how we relate to science, not try to go back to the stilted, slow pre-pandemic world. We should embrace the extraordinary and robust process of open science and more peer review, as well as its dynamism, even as we establish new guardrails to contain its energy.”

“The pandemic has turned reviewers from gatekeepers into coaches…Reviewers of Covid papers were less likely to ask for additional data or experiments, or for major restructuring. Instead, where they identified deficiencies and gaps they tended to ask for conclusions to be modified and qualified. Reviewers of Covid-related papers also seemed to place greater emphasis on the social and medical relevance of research, rather than on its pure methodological soundness or novelty. All in all, this suggests the crisis has seen an emphasis on getting work out there as quickly as possible. This touches on a long-standing debate on the function of journal peer review and the task of reviewers.”

Has covid made peer review kinder? “Faster but more cooperative”

How the pandemic changed editorial peer review – and why we should wonder whether that’s desirable

Preprint on Vitamin D treatment removed from Lancet server

"Scientists said claims about China creating the coronavirus were misleading. They went viral anyway."

Covid conflicts of interest? “As testing and the Oxford AstraZeneca vaccine are hailed as UK pandemic successes, why won’t Oxford University or the government disclose the ‘long list’ of financial interests of a high profile researcher at the centre of both?”

A letter to editors on covid and suicide rates, from the International COVID-19 Suicide Prevention Research Collaboration: “[O]ver the last few months we have seen a number of sensational headlines and news stories based on reports of academic articles and wanted to alert you to the potential impact this may have on suicide risk more widely.” What not to publish to avoid contributing to the problem.

How much covid research is strong enough to use for health policy? “Only 1/27 studies passed all of the above checks, and 4/27 were rated as overall appropriate.” [preprint]

Preprint to publish: “The COVID-19 preprints that became peer-reviewed journal articles were often submitted to journals concurrently with the posting on a preprint server, and the entire publication cycle, from preprint to the online journal article, took on average 63 days. This included an expedited peer-review process of 43 days and journal’s production stage of 15 days, however there was a wide variation in publication delays between journals. Only one third of COVID-19 preprints posted during the first nine months of the pandemic appeared as peer-reviewed journal articles.” [preprint]

Surgisphere retractions continue to be cited: “[O]f the 200 papers examined by Science—all published after the retractions—105 inappropriately cited one of the disgraced studies. In several cases it was a primary source for a meta-analysis combining multiple studies to draw overarching conclusions. In most, the studies were cited as scientific support or context.”

How do media outlets identify preprints? “Communicating Scientific Uncertainty in an Age of COVID-19: An Investigation into the Use of Preprints by Digital Media Outlets”

“Are too many scientists studying Covid?”

“How a torrent of COVID science changed research publishing — in seven charts” including topics covered, peer review time, growth of preprints, publication by country, and gender disparities. “Around 4% of the world’s research output was devoted to the coronavirus in 2020, according to one database. But 2020 also saw a sharp increase in articles on all subjects being submitted to scientific journals — perhaps because many researchers had to stay at home and focus on writing up papers rather than conducting science. Submissions to publisher Elsevier’s journals alone were up by around 270,000 — or 58% — between February and May when compared with the same period in 2019, one analysis found¹. The increase was even higher for health and medicine titles, at a whopping 92%.”

“Rapid Translation of COVID-19 Preprint Data into Critical Care Practice”: Australian researchers studied use of corticosteroids before and after preprint and final paper publication of the RECOVERY trial, which demonstrated that dexamethasone improved mortality in patient with covid who required respiratory support. “For the period prior, corticosteroids were used in 29.5% of patients, increasing to 92.0% after the preprint release (absolute difference, 62.5% [95% CI, 51.3% to 73.6%]; p < 0.001). After formal publication of the trial results, 95.8% of patients received corticosteroids, similar to that observed in the preprint period (absolute difference, 3.8% [95% CI, -5.4% to 13.1 %]; p = 0.275).” Their abstract concludes, “[P]reprint release of the RECOVERY trial findings led to an almost immediate, dramatic change in corticosteroid use in critically ill COVID-19 patients across Australia. Although the rapid translation of medical evidence is beneficial in cases where the intervention is of proven benefit, preprint release of trial findings prior to peer review, creates a risk of inappropriate global translation. Fortunately, in this scenario, independent findings from other research groups identified a similar signal7, and potentially thousands of critically ill COVID-19 patients were advantaged globally, through the rapid dissemination of these results.” Of course, there are concerns about preprints being consumed without appropriate caveats. 

Does peer review differ when the topic is covid? This researcher studied whether peer review comments are qualitatively different between covid and non-covid papers, using the open peer reviews provided in BMJ and eLife. They were, specifically “diversity between Covid-19 and non-Covid-19 related articles, including fewer requests for additional experiments, more cooperative comments, and different suggestions to address too strong claims. In general, the findings suggest that both reviewers and journal editors implicitly and explicitly use different quality criteria to assess Covid-19 related manuscripts, hence transforming science’s main evaluation mechanism for their underlying studies and potentially affecting their public dissemination.”

More than half of PubMed and preprint covid papers (in English or Chinese) did not report original research: 4190 publications (56.1%) did not include any data or analytics (comprising expert opinion pieces)…the exponential growth of COVID-19 peer-reviewed articles was mainly driven by publications without original data (mean 261.5 articles ± 51.1 per week) as compared with original articles (mean of 69.3 ± 22.3 articles per week). Original articles including patient data accounted for 713 (9.5%) of peer-reviewed studies.”

Would adherence to reporting guidelines have revealed the problems that led to the Lancet and NEJM Surgisphere COVID retractions? While most STROBE components were reported, RECORD (REporting of studies Conducted using Observational Routinely-collected health Data) and RECORD-PE (RECORD for Pharmacoepidemiology) components were not. “Important aspects of the methods unique to research using routinely collected health data were not reported, including variables used to identify exposure, outcome and confounders, and validation of the coding or algorithms. RECORD also requires a detailed description of the database population (ie, how data were collected in the population from which the study database is created), and the description and accuracy of linkage of databases.” Regardless, as the authors point out, “The lack of description of the underlying Surgisphere database should have raised concerns amongst peer reviewers and editors.”

How did published research in the “big 5” change with covid? “[W]e characterised various aspects of COVID‐19‐related articles published in the five leading general medical journals with the highest impact factors (Web of Science) compared with an equivalent period in the preceding year…In circumstances which usually require consent, just under half of the COVID‐19 studies did not explicitly state consent was obtained, despite clear recommendations by the International Committee of Medical Journal Editors…There was a near three‐fold increase in the proportion of studies that published corrections, perhaps reflecting the observed reduction in time from submission to publication observed in the one journal for which these data were available. It is likely this figure is an underestimation, given that corrections and retractions would be expected to continue over time. Three COVID‐19 studies were retracted.”

Retraction Watch’s Top Retractions of 2020 -- Covid anyone?

Surprising no one, preprints addressing Covid were published in peer-reviewed journals more quickly than other topics [preprint]

Medical journals, covid, and politics: “[W]hile editors of these prestigious journals have taken political positions, the international and national associations and committees of medical journal editors have so far not stuck their necks out by taking positions on the handling of the pandemic and interference in scientific processes by the political leadership. In India, no medical journal or association of medical journal editors has taken a strong critical position on the government’s handling of either of these issues. It is not clear to what extent this silence is due to a fear of reprisals, or to the belief that medical journals should stay “apolitical” ─ though to be apolitical is also a political position, sometimes equated with collusion and silent support. So there is a long way to go before medicine and science push back the intrusion of irrationality and untruth brought about by certain types of politics.”

“Covid-19: politicisation, 'corruption,' and suppression of science… The UK’s pandemic response provides at least four examples of suppression of science or scientists… How might science be safeguarded in these exceptional times? The first step is full disclosure of competing interests from government, politicians, scientific advisers, and appointees, such as the heads of test and trace, diagnostic test procurement, and vaccine delivery. The next step is full transparency about decision making systems, processes, and knowing who is accountable for what.”

“Academic Publishing in Nepal during the COVID-19 crisis

Does rapid publication also reduce the ability to identify duplicate publications?

Open science solutions to flaws in Covid research literature [preprint]

Data sharing in the Covid era: “ 'I'm hoping this will raise awareness of the possibilities of interdisciplinary research and show researchers that there is a different way of approaching research and learning from other studies…I believe that all publishers must require researchers to put data in either a discipline-specific or general repository…then we need to have quality data curation across the board, involving institutional libaries [sic] and data curators.'”

Several journals publish editorials related to the US election including NEJM, Science, Nature, and BMJ, on Covid and beyond. Wired comments, and Columbia Journalism Review writes on Lancet’s political commentary.

When politics interfere in science: the US, India, and Turkey

“The Surgisphere Scandal: What Went Wrong?” A comprehensive review of Surgisphere and its founder, Dr Sapan Desai, including a helpful timeline

Reviews of covid retractions: As of 31 July 2020,19 published articles and 14 preprints about COVID-19 have been retracted, withdrawn, or an expression of concern has been issued” and “According to this search [as of June 18, 2020], the most common reasons for retraction were concerns, issues, and errors in results and/or conclusions, followed by concerns, issues, and errors in data.”

“Are Publishers Learning from Their Mistakes? …in the wake of any single retraction, publishing houses face near-term incentives that are similar to those of an airline after a disaster: rather than looking for the root causes of the problem, and investing to fix them across a portfolio, it is in their near term interest to address the issue narrowly, perhaps sometimes assign blame elsewhere, and try to move forward as quickly as possible. But in the long-run, avoiding scrutiny is good neither for airline safety nor for the industry as a whole. As a result, the federal government created the National Transportation Safety Board (NTSB), to serve as a third-party that investigates airline accidents, assigning responsibility and recommending future corrective actions fearlessly. Perhaps the scholarly publishing sector needs an NTSB-like partner to conduct regular ‘post-mortems’ so that the sector as a whole can learn lessons from each incident, address shortcomings pro-actively, avoid repeating the same mistakes again, and establish confidence that editorial practices and trustworthiness are continuously improving...The sector needs not only to ask for trust but also to make sure that it is continuously earning it every day.”

“Evidence, rationality, and ignorance: Agnotological issues in COVID-19 science”: Open questions around Covid serve “as a fertile breeding ground for agnotological strategies, whereby scientific studies are deliberately or unintentionally designed to create distractions or draw conclusions that are not supported by research findings. Researchers, public health authorities, and healthcare workers should be equipped to identify such agnotological strategies, distinguish them from scientific fraud, and avoid drawing misleading inferences based on an irrational adherence to hypotheses and a lack of criticism of implausible results.”

Covid and air quality: When journals review papers beyond their wheelhouse “[S]atellite data often come with caveats, quality flags, and recommendations on how to use or not use the data (e.g., accounting for transport by winds or screening for clouds, if they are present). Scientists routinely access this information in users’ guides and algorithm theoretical basis documents, but nonexpert users either don’t know to look for these resources or may not have access to them.”

“COVID-19 Blamed for Weaker Research Published by Top-Tier Journals in 2020”: Authors presenting at the virtual European Society of Cardiology Congress 2020 analyzed articles published between January and April 2020 in NEJM, Lancet, and JAMA, compared with the same period in 2019. They “assessed quality using four parameters: randomized design, defined primary hypothesis, defined primary endpoint, and GRADE quality of evidence score.” They found that fewer studies were randomized, “the study hypothesis and primary endpoint were less frequently defined,” the GRADE score indicated that the studies were weaker. When they excluded Covid-19 research, they found no difference in study quality year to year. The presenting author acknowledged, “The issue is that this is an unknown disease and there was an urgent need to have some evidence, because some evidence is better than no evidence.”

Characteristics of articles on Covid-19 published as of April 2, 2020: “Limitations were reported in 42 out of 1165 abstracts (3.6%), with 10 abstracts reporting actual methodological limitations.”

The physician behind Surgisphere and retracted covid studies

“Science, uninterrupted: Will COVID-19 mark the end of scientific publishing as we know it?”

“How the COVID-19 crisis has prompted a revolution in scientific publishing…Right now, the production, vetting, and editing of a manuscript still happens behind closed doors, locked inside journal editing screening processes. We envision a world in which all of that takes place out in the open, and a marketplace of tools, services, and community-driven initiatives help make science better by engaging more people to participate in the process. In this world, we might have an off-the-shelf service that translates a public scientific paper for a general audience. We could have a community of scientists to help ensure that gene-variant names in another study aren’t misspelled. There could be new technology that would help analyze the connection between results, methods, data, and resources. And, for sure, there would be more novel collaborations that bring important breakthroughs to the forefront when they’re needed most. These ‘overlay services’ built on top of preprints to deliver value to scientists are what made other examples of online collaboration and peer production successful.”

What will Covid mean for clinical trials? “The COVID-19 pandemic has created massive disruptions to clinical trial research across the world. As in other aspects of life, the virus has severely affected the ability to conduct trials in safe and effective ways…Thousands of trials have been suspended or stopped because of the difficulties in continuing under lockdown conditions, even as those restrictions have begun to ease in parts of the world.”

Most Covid information for the public tested around the world is well above recommended reading levels

Two Covid-19 studies based on Surgisphere data have been retracted, one each from the New England Journal of Medicine(NEJM) and the Lancet. Review and commentary ensued; what have we learned?

• From the Los Angeles Times: “Questions about the research first arose in Australia, where it was reported by the Guardian Australia that the Lancet paper cited more COVID-19 deaths in that country than had actually occurred during the paper’s time frame. Surgisphere explained that data from an Asian hospital had been mistakenly added to the Australian figures. The Lancet ran a brief correction. But the glitch prompted closer scrutiny of Surgisphere and its database. Medical authorities wondered how a small company with roughly a dozen employees could have assembled a worldwide patient database with tens of thousands of samples…In the original paper, Mehra states that he and co-author Amit Patel ‘had full access to all the data in the study.’ Now he’s saying that wasn’t so…Mehra and his colleagues will need to explain how they came to rely on a database they can’t vouch for. Desai will have to explain the origins of the database, if it even exists. And the Lancet and NEJM will have to explain how they came to publish papers that they had to retract within two weeks.”
• From BMJ Blogs: “A single author, Sapan Desai, who directed the company Surgisphere, which describes itself as a healthcare data analytics company, was responsible for collecting data from 671 hospitals across 6 continents, and the sole person responsible for acquiring the data and conducting the statistical analyses…In this case, this weakness was plainly evident in the author contributions section of the Lancet manuscript, and could have been raised by the editors or peer reviewers.”
• The African Federation for Emergency Medicine Severity Scoring Tool derived from the database should not be used: “Effective immediately(05 June 2020), AFEM is recommending discontinued use of the Severity Scoring Tool. Having to make such a recommendation is disappointing –we know that there is little contextually-appropriate guidance for the COVID-19 response in low-resource settings, and the AFEM team was excited to share this resource with our community–but it is the only right choice in this scenario…For now we encourage the use of guidelines of the World Health Organization (WHO) for clinical management of COVID-19 which are available for download here: https://www.who.int/publications-detail/clinical-management-of-covid-19”
• From The Guardian: Reevaluating the peer review process: According to a NEJM spokeswoman, “the Surgisphere study ‘received excellent peer reviews, and the reviewers raised relevant questions about the work’,” but “‘We have limited experience with reviewing or publishing studies like this one, which used a large database based on electronic medical records. The reviewers and editors asked the authors questions about the data sources and data validity. The editors accepted the authors’ responses, rather than asking for help from reviewers with expertise in this type of data. In the future, our review process of big data research will include reviewers with such specific expertise.’” A Lancet spokeswoman said “‘all research articles undergo independent, external peer review, including statistical review…the Surgisphere paper was peer reviewed and edited according to The Lancet’s ‘usual editorial procedures’. ‘We are reviewing our requirements for data sharing and validation among authors, and data sharing following publication.’”
• From The Scientist: “As funding is not infinite, it is essential that basic scientific and biomedical research be conducted in a proper fashion, thus economizing resources and minimizing the production of unreliable and misleading results…The intention was that the WHO should set the standard worldwide by imposing compliance to QPBR [Quality Practices in Basic Biomedical Research] on institutions receiving WHO research funding. Clearly, this intention has not been carried through; training in QPBR, though planned and with training material available, was never deployed extensively and research institutions have not been required to implement QPBR prior to receiving funds. We appeal to all funding bodies (WHO, National Institutes of Health, European Commission, Medical Research Council, Gates Foundation, government agencies, etc.) to tie their funding to the implementation of QPBR (or equivalent guidelines) as a first step to assuring that the studies they support generate quality data, that is, accurate, reliable, reproducible, and auditable.”
• From Wired magazine: “Just how historic is the latest Covid-19 science meltdown?: Science meltdowns of this type—and the ‘biggest’ retractions that ensue—occur with shocking regularity. Again and again, over decades, scientists and the public have had their confidence in the enterprise shaken by these sorts of disturbing revelations; and then, again and again, over decades, everyone has been surprised.”
• From Inside Higher Ed: “Rush to Publish Risks Undermining COVID-19 Research…[S]cholarly publishing is grappling with new time pressures and new pressures from the general public. ‘The idea that you run your study, you gather the data, then you take time writing it up, you send it to the journal, the journal sends it to a couple of referees, they all take their time, and six months later the paper gets published -- that is not an attractive possibility in the context of an evolving pandemic.’”
• From the New York Times: “A spokesman for Lancet publisher Elsevier said it will re-assess about 20 additional published articles that contain Surgisphere data. A spokesman for Brigham and Women's Hospital [at Harvard] said its data oversight mechanisms ‘were not applicable’ to the Lancet and NEJM studies, adding it only provides ‘support, guidance and oversight on all agreements that involve the use of our institutional resources, including our patient data.’…‘We are reviewing our procedures, including how we assess research analyzing large datasets based on electronic medical record data,’ said NEJM spokeswoman Jennifer Zeis.”
• Surgisphere founder’s past research Sapan Desai questioned for possible image manipulation: “‘I concur with the allegation that there appear to be numerous small duplicated regions in the photographs,” [said] Mike Rossner, a former managing editor of the Journal of Cell Biology who now runs a consultancy firm called Image Data Integrity.”
• Sapan Desai, coauthor of both papers, previously coauthored publications including “a call to arms for medical publishers and editors to resolve any potential conflicts of interest in order to protect the integrity of scholarship…how the Journal of Surgical Radiology prevents publication fraud. They write, ‘The most serious cause of fraud in medical publishing is manufactured data that authors use to support high impact conclusions’…and “best practices for registry research.”
• “Surgisphere Sows Confusion About [Ivermectin,] Another Unproven COVID-19 Drug… The company behind a now-discredited study on hydroxychloroquine also posted a report that has been cited by Latin American governments recommending ivermectin as a possible coronavirus treatment. Clinicians there say the effects have been extremely damaging.”
• Red flags in the Surgisphere research papers: “Reviewers are expected to critique the science put in front of them, but not necessarily to question whether it was real in the first place… this episode makes it clear that postpublication review and discussion (often facilitated by social media) is now an essential component of the scientific process.”
• ***New York Times: “ ‘This got as much, if not more, review and editing than a standard regular track manuscript,’ Dr. Rubin, the editor in chief of the N.E.J.M., said of the heart study appearing in the N.E.J.M., which was based on a smaller set of Surgisphere data. ‘We didn’t cut corners. We just didn’t ask the right people.’
• Editor in the spotlight: “The Lancet Editor’s Wild Ride Through the Coronavirus Pandemic” and more… “[W]e all made a collective error, and that collective mistake was to believe what we were being told.”
• Hospitals deny providing data to Surgisphere and question why source hospital names weren’t listed in the research report. And why did the coauthors not balk when they were denied access to the raw data?

“COVID-19 research: pandemic versus ‘paperdemic’, integrity, values and risks of the ‘speed science’: Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has ignited another parallel viral pandemic, with science ranging from robust studies to dishonest studies being conducted, posted, and shared at an unprecedented rate. A balance is needed between the benefits of the rapid access to new scientific data and the threat of causing panic or erroneous clinical decisions based on mistakes or misconduct. The truth is that the ‘scientific research has changed the world’ but now, and more than ever, ‘it needs to change itself’.”

Characteristics of covid-19 clinical research registered in WHO’s International Clinical Trials Registry Platform as of March 26, 2020.“While accelerating morbidity and mortality from the COVID-19 pandemic has been paralleled by early and rapid clinical investigation, many trials lack features to optimise their scientific value. Global coordination and increased funding of high-quality research may help to maximise scientific progress in rapidly discovering safe and effective treatments.”

“Will the pandemic permanently alter scientific publishing? The current system would have to shift wholesale to rewarding open, early sharing of findings to give scientists incentives to communicate their work in this way. A few funders and research institutions were already advocating that approach, and the pandemic could nudge them further along this path. But the crisis could unleash other forces that might reshape science communication: not least an economic downturn that could disrupt research budgets, job markets and the scientific-publishing industry.”

“Licences and permanence of COVD19 collections in PubMed Central (PMC)”: Many journals and publishers have made their covid content free. For how many is this change permanent? Articles published by Elsevier, Springer Nature, and Wiley, which comprise the vast majority of articles in the Public Health Emergency collection, the licenses are temporary.

Free Webinar: Scholarly Communication & COVID-19: Closing the Loop for Effective Peer Review: Wednesday, June 24, 2020

A guide to making sense of coronavirus studies

Placing preprints in perspective: Lightly vetted science didn’t start there – consider presentations at scientific conferences and inadequate peer review

“Preprinting a pandemic: the role of preprints in the COVID-19 pandemic” [preprint]

First peer reviewed report on remdesivir: "Preliminary results from the 1059 patients (538 assigned to remdesivir and 521 to placebo) with data available after randomization indicated that those who received remdesivir had a median recovery time of 11 days (95% confidence interval [CI], 9 to 12), as compared with 15 days (95% CI, 13 to 19) in those who received placebo (rate ratio for recovery, 1.32; 95% CI, 1.12 to 1.55; P<0.001). The Kaplan-Meier estimates of mortality by 14 days were 7.1% with remdesivir and 11.9% with placebo (hazard ratio for death, 0.70; 95% CI, 0.47 to 1.04)." 

"JetBlue’s Founder Helped Fund A Stanford Study [by John Ioannidis and others] That Said The Coronavirus Wasn’t That Deadly...But Ioannidis and his coauthors did not disclose that the study was funded in part by Neeleman." The study, posted as a preprint, had a rocky reception.

Will the Covid-19 pandemic change how science is published? More than 7,000 papers on the pandemic—covering everything from virology to epidemiology—have appeared in the past three months (see chart). A fifth of them have come out in the past week alone... Around half of the available scientific work on the pandemic has been released through preprint servers."

"How coronavirus is changing research practices and publishing... A recent analysis of 669 articles published in 14 medical journals found that COVID-19 articles published since January 2020 are taking 57 fewer days on average from submission to publication compared to non-pandemic articles published before October 2019. A separate analysis of 11,686 articles found that roughly 68% of COVID-19 papers are now freely available online. For comparison, it’s estimated that approximately 28% of scientific papers across all disciplines are open access."

Burgeoning preprints evolve with prescreening and some peer review: "In April, the clinical preprint repository medRxiv published between 50 and 100 SARS-CoV-2-related posts daily...As Nature Biotechnology went to press, the repository had received over 1,941 COVID-19 preprints, ranging from clinical trials of therapeutics to diagnostic tests, personal protective equipment, and epidemiology or modeling, and bioRxiv had posted 508 SARS-CoV-2-related preprints. Over the same period, 7,136 COVID-19-related papers appeared in PubMed. This last statistic is notable because the number of COVID-19 preprints (2,449) as a proportion of peer-reviewed papers (7,136) published since the beginning of the year is high (34%)." Some preprints have evolved: MedRxiv preprints are screened before posting for ethics and potential for public harm, and "One recent development has been for journals to post referee reports on the preprint version of a paper under review. A grassroots initiative, PREreview, aims to facilitate constructive feedback, educate a broader global community of reviewers, and convene virtual journal clubs on preprints."

"A pandemic moves peer review to Twitter...Page views to the MedRxiv site are now averaging 15 million a month, up from 1 million before the pandemic... Many of the coronavirus-related papers being posted on MedRxiv are rushed and flawed, and some are terrible. But a lot report serious research findings, some of which will eventually find their way into prestigious journals, which have been softening their stance on previously released research."

"Coronavirus in context: Scite.ai tracks positive and negative citations for COVID-19 literature: Unlike conventional citation-metrics tools, Scite.ai tells users how often a paper has been supported or contradicted by the studies that cite it, as well as how many times it has simply been mentioned. The resulting reports display citations in the context in which they are mentioned, allowing users to assess for themselves how the paper is being cited."

"How swamped preprint servers are blocking bad coronavirus research...BioRxiv and medRxiv have a two-tiered vetting process. In the first stage, papers are examined by in-house staff who check for issues such as plagiarism and incompleteness. Then manuscripts are examined by volunteer academics or subject specialists who scan for non-scientific content and health or biosecurity risks. BioRxiv mainly uses principal investigators; medRxiv uses health professionals...On bioRxiv, this is usually completed within 48 hours. On medRxiv, papers are scrutinized more closely because they may be more directly relevant to human health, so the turnaround time is typically four to five days."

Psychological science in a crisis: It's not rocket science, but perhaps it should be approached with the same care: proposing Evidence Readiness Levels for psychological science in a crisis, based on NASA's (US National Aeronautics and Space Administration) Technology Readiness Levels [preprint]

Social science and Covid-19: Rushing to publication, speaking outside areas of expertise, and the temptation to overclaim

Advice for journalists, well taken for anyone evaluating research: always read the entire paper, ask "dumb" questions to define jargon, ask "smart" questions about the study design and methodology, look for the limitations, determine clinical relevance, quantify results, list adverse effects, identify dropouts, assess alternative treatments and conflicts of interest, seek outside opinions, and avoid language that implies causation unless the research actually shows it.

"Chinese state censorship of COVID-19 research represents a looming crisis for academic publishers: These shared challenges and objectives could be an organising principle for a set of industry standards regarding research censorship, ensuring that all amendments to journal platforms are clear and transparent – as is already the case for retractions, errata and corrigenda. If full, unrestricted, global Open Access is not possible, full disclosure and transparency, where obstacles exist, allows for a better-informed debate on the limits of ‘openness’, which ought to continue beyond the current crisis.

 "That demand for conclusiveness highlights longstanding tensions over the role of a scientific journal. Should it be an arbiter of facts or a generator of new ideas? A keeper of the historical record or a predictor of the future? A private channel for scientists to communicate with one another or a megaphone with which they can reach the public? Or all of the above?...If we put our faith in a single conclusion, it’s easy to feel distressed when it’s amended. If we trust the process, imperfect though it is, we’re better prepared to change with it, which is the most we can hope to do."

"How not to lose the Covid-19 communication war: [T]he seductively simple directive to be 'accurate,' which lies at the heart of science communication, obscures the reality that accuracy is a tenuous notion during a crisis such as this, in which uncertainty reigns. Science that was considered correct at the outset will likely turn out to be incorrect or incomplete, making it difficult to draw a bright line between misinformation and science that is legitimately contested." 

"Crises are no excuse for lowering scientific standards... Although crises present major logistical and practical challenges, the moral mission of research remains the same: to reduce uncertainty and enable caregivers, health systems, and policy-makers to better address individual and public health. Rather than generating permission to carry out low-quality investigations, the urgency and scarcity of pandemics heighten the responsibility of key actors in the research enterprise to coordinate their activities to uphold the standards necessary to advance this mission."

"Compared to articles published in the same journals before the pandemic, turnaround times have decreased on average by 49%. The largest decrease in number of days between submission and publication of articles was due to a decrease in the number of days required for peer review. While the acceleration of journals’ publication process is laudable from the perspective of quick information dissemination, it also raises concerns relating to the quality of the peer review process and the quality of the resulting publications." [preprint]

" [M]ore people will die from Covid-19 because we cannot study drugs more quickly...[O]ur inability to start and run clinical trials faster — whether in normal times or in a pandemic — is a legacy of our decision not to develop the technologies and approaches that would make doing so easier."

" A Number Of Coronavirus Studies Are Now Being Released. How Do We Gauge Their Validity?" [podcast interview with Ivan Oransky]

Retracted covid-19 research from Retraction Watch -- plus selected Covid research: retracted, corrected, withdrawn, questioned (not a complete list):

· "A study which found that aerosolized novel coronavirus could be spread nearly 15 feet — twice what health officials had believed — has been retracted, but the journal isn’t saying why."

· " 'The authors did not mean to suggest that the victim had died, and that the authors do not know for sure and cannot scientifically confirm that the virus moved from the dead body.' "

· "The controversy about the report will only add weight to the arguments from scientists that Sweden is taking risks with its relatively relaxed coronavirus strategy."

· Does air pollution increase the risk of dying from Covid-19? A commentary/peer review

Correcting the research that cites the retractions: "Authors to correct influential Imperial College COVID-19 report after learning it cited a withdrawn preprint"

 European Network of Research Integrity Offices (ENRIO): "Research integrity remains crucial in the COVID-19 crisis, given the need for robust, evidence-based conclusions: Researchers should communicate their work on social and other media responsibly, with professionalism and transparency. Subjective or unfounded interpretations must be avoided and information must not be intentionally omitted...Academic and research organisations in particular, as trusted promoters of good research practices, should be extra vigilant in identifying and clearly exposing poor as well as fraudulent practices to protect the public from flawed interpretations by overenthusiastic, careless or unscrupulous researchers."

"Rapid Registered Reports initiative aims to stop coronavirus researchers following false leads" "[T]he journal Royal Society Open Science is expediting its Registered Report review process specifically for COVID-19 research. In a Registered Report, the study is peer reviewed prior to data collection, with a focus on the methods and analysis plan. The aim of the COVID-19 initiative is to return this initial review to authors within one week of submission. Passing peer review means that authors receive an ‘in principle’ acceptance – if they follow the agreed research plan, their study will be accepted for publication, regardless of the results. Studies will be published open access, with reviews made available alongside the article. For this initiative, Royal Society Open Science is waiving its publication fees."

"Elsevier investigating hydroxychloroquine-COVID-19 paper" "Elsevier has weighed in on the handling of a controversial paper about the utility of hydroxychloroquine to treat Covid-19 infection, defending the rigor of the peer review process for the article in the face of concerns that the authors included the top editor of the journal that published the work." 

Studying drug treatment for Covid-19: research, and publishing: Ivan Oransky comments on CBC (Canadian Broadcasting Corporation) 

"Coronavirus Tests Science’s Need for Speed Limits: 'Science is a conversation'...'Unfortunately people in times of crisis forget that science is a proposition and a conversation and an argument. I know everybody’s desperate for absolute truth, but any scientist will say that’s not what we’re dealing with.' ...Dr. Yammine thinks that preprint servers and peer-reviewed journals should 'encourage scientists to lead the fight against misinformation by making it mandatory to include a plain-language summary outlining the scope and limitations of their work prior to publication,' she said...'Every drug ad has to end with: ‘talk to your doctor,' Dr. Oransky said. 'Maybe every preprint header or primer should end with, ‘talk to your neighborhood statistician,' ' to determine how legitimate a paper is — or is not.”

Responsible coverage of coronavirus preprint research from the Association of Health Care Journalists [webcast]

Mother Jones magazine on fast science, covered quickly: “ 'In a normal news cycle, science gets like less than 1 percent of the coverage...But in today’s news cycle, the only thing in the news is science. So everybody, every economist, every physicist, every tech guru, everybody wants a piece of the attention if they think they can link themselves or link their own pastime, their background, to something that’ll be covered.' ”

"Pseudoscience and Covid-19--we’ve had enough already: Disinformation expert Claire Wardle at Harvard University in Cambridge, Massachusetts, has said, 'The best way to fight misinformation is to swamp the landscape with accurate information that is easy to digest, engaging and easy to share on mobile devices.'... Of course, part of the scientific community’s fight against pseudoscience is keeping its own house in order. Those pushing biomedical conspiracy theories and other nonsense point to legitimate concerns about how research is funded, interpreted and disseminated. Scientific integrity — particularly, refraining from hype and being transparent about conflicts — is crucial. We must promote both trust in science and trustworthy science."

"Problems arise when the public misinterprets [preprints] or puts too much emphasis on something minor." [podcast]

"Journalists are recognizing they’re writing a rough draft of history – and can’t say definitively ‘that’s the way it is’: A story in the Los Angeles Times stated, " 'One thing to keep in mind before we continue: It is possible that the information you read below will be contradicted in the coming weeks or that gaps in knowledge today will soon be filled as scientists continue to study the virus.'"

"The truth is a snowball rolling down a hill, and each new fact changes the snowball, making it bigger and more multifaceted... The changing conclusions journalists can draw from facts about the coronavirus make the weakness inherent in this attitude especially clear, since information is evolving so rapidly... Long-developing, ongoing stories – such as the coronavirus today – are particularly difficult for journalists to convey in the day-by-day, article-by-article style which they and the public are used to... a corrections column, as much as it suggests a news organization’s dedication to truth, really is not adequate to dealing with uncertain, always changing stories, such as COVID-19."

"Science Will Not Come on a White Horse With a Solution: We’ve modeled the progression of the disease, but not the social consequences of the preventative measures that we’re taking... It’s well known from disaster studies that effects on vulnerable populations are much more severe and long-lasting...Social sciences and social problems get lower billing and lesser attention than physical and material causes that we think we can control more easily... Peer review is not good when you need to confront different bodies of knowledge with and against each other. Someone who understands the dynamics of what happens inside a family when you’re cooped up together for week upon week—that person is not going to tell you very much about how a virus acts inside a body, or how quickly contagion spreads if you don’t self-isolate."

COVID-19 and the future of open access: Will "both the upsurge in open practices and the freely available content made available as a result of the lockdown...help nurture a better, more open and less profiteering publishing ecosystem?... it is important to note that paywalls have been lifted temporarily, unilaterally and unsystematically – purely in response to a global pandemic crisis. Once this crisis has passed, or at least when publishers deem it to have passed, there is no suggestion that anything other than business as usual will returnand that paywalls will be re-erected...If things were tough already for small, not-for-profit, and university press publishers, they are going to get worse during the downturn."

How will Covid-19 affect the debate around free access to research and use of preprints? 

"Covid-19 is reshaping the world of bioscience publishing": Or the world's new shapes of publishing are coming into greater relief: "bioRxiv published its first preprint on the novel coronavirus on January 19, 2020, and it has been on a roll ever since, with 33 papers in January, 281 in February, and more than 500 in total as I write this. bioRxiv is currently receiving 25 to 30 scientific papers day on this topic, with the majority now going to medRxiv, the more clinically oriented site...Even the most elite peer-reviewed journals in the bioscience publishing world are modifying their approach in response to the pandemic. The New England Journal of Medicine, for example, is making all of the material it publishes on Covid-19 open access on the web (everything else is behind a subscription paywall), and putting this content in one place on NEJM’s high-traffic website...The Lancet has a preprint section for as yet un-reviewed papers (including an important new Covid-19 paper), and other journals are accelerating the review process."

How are editors adjusting to the consequences of covid-19? "Some editors expect an initial ‘boom’ in papers as scientists newly blocked from entering their labs find themselves with more time to write, edit and respond to reviews — followed by a longer-term slowdown in work. At the same time, many scientists who serve as editors and reviewers are managing their own affected labs while tackling changing demands at home, including childcare and remote working.

"Editorially Surviving an Outbreak": from nomenclature, to drafting letters to authors requesting that their data be sent to WHO, to extreme efficiency 

Covid-19 submissions, peer reviews, and preprints all in massive numbers How does the peer review process keep up? 

"Responding to peer reviews is never fun. It’s harder when COVID-19 shuts down your lab"...and how do you respond to requests for additional experiments on your own manuscripts? 

Paper on covid-19 testing quoted as reason why US didn't import covid-19 tests was retracted and removed from online journal

Ivan Oransky discusses potential risk of rapid covid-19 research on a Economist podcast (listen at minute12:45)

"Strong caveats are lacking as news stories trumpet preliminary COVID-19 research"“Until COVID-19 there was no reason for the press to pay attention (to preprints) because they could count on the journals to curate what they felt was most important, most impactful,” said one of medRxiv’s founders, Joseph Ross, MD, a professor of medicine and public health at the Yale School of Medicine. “But now it’s chaotic. Science is moving really quickly and we’re all trying our best to understand things."

eLife publishes its review criteria for Covid-19 research "We think it is essential that eLife changes how it functions in response to this new reality, so we are planning to take the following steps.

1. Curtail requests for additional experiments during revisions...
2. Suspend the two-month limit on revisions...
3. Make the posting of preprints to bioRxiv or medRxiv the default for all eLife submissions...
4. Extend our 'scoop protection' policy to cover competing work that is published on preprint servers prior to submission...
5. Mobilize early-career researchers."

Medical Journal of Australia on their rapid review of Covid-19 papers: Says Editor-in-Chief Nicholas Talley, "The MJA has stepped up to do its part in this crisis including ultra-rapid review of SARS-CoV-2 papers and pre-print publication so the newest data and viewpoints are released as soon as possible. In addition, all SARS-CoV-2 articles are fully free access for anyone to read. Our medical and structural editors are working from home and the MJA will continue to publish as usual in these extraordinary times. The new rapid publication model risks errors, but being too slow with information sharing is a much bigger risk. We will work to refine the preprints prior to publishing online early and in print, and correct and update along the way." 

The rush to publish on covid-19 continues -- but are all the patients reported unique? Apparently not

---

 ACADEMIA AND RESEARCH

►“Proposed Biosecurity Oversight Framework For The Future of Science: Draft Findings & Recommendations of Two National Science Advisory Board for Biosecurity [NSABB] Working Groups...In developing the findings and recommendations presented in this report, the two NSABB Working Groups tasked with evaluating the [Potential Pandemic Pathogen Care and Oversight] and [Dual Use Research of Concern] oversight frameworks and considered relevant policies and guidance, and consulted with subject matter experts in pathogen research, research administration and oversight, biosafety and biosecurity, biodefense, and national security, among others, from the [US government], federal funding agencies, academic institutions, and scientific and professional societies.” Related Nature News article: ““The big loophole at this stage is that most things never get to the stage of being reviewed” because they aren’t flagged as risky, [Marc Lipsitch, epidemiologist at the Harvard T.H. Chan School of Public Health in Boston, Massachusetts] says. He points to proposed experiments at the NIH that would attempt to evolve a monkeypox virus to see whether certain mutations could make it more lethal to mice, and a study³ in which researchers inserted a gene from the Omicron variant of SARS-CoV-2 into an earlier strain of the virus in an effort to understand how Omicron causes disease. Neither underwent the HHS’s extended-review process.The COVID-19 pandemic has highlighted the difficulty of designing guidelines that protect against research that is unreasonably risky without hampering crucial studies with regulatory red tape. It has also put a spotlight on risky pathogen research, with concerns raised over the safety and transparency of practices at laboratories in China that investigate bat coronaviruses.”

"Scientists Must Combat Scientific Dogmatism...The COVID-19 pandemic has thrust discussions over what role science should play in society to the forefront...We, as a society, need to learn how to have constructive, evidence-based scientific discussions...It is as unscientific to blindly trust scientists as it is to dismiss them. The dysfunction in how the public consumes scientific news can partly be traced to a weakness in our collective scientific [il]literacy."

Was it a lab leak?

• “The media called the ‘lab leak’ story a ‘conspiracy theory.’ Now it’s prompted corrections — and serious new reporting.”
• “Fauci described Indian research on ‘man-made Covid’ as outlandish” and an earlier report on the same study
• See also [subscription required]: Elite journals under scrutiny over role in Wuhan lab leak debate

Were Dutch studies of covid transmission in crowds ethical? “The studies didn't need approval from a medical ethics committee because they didn't meet the legal definition of medical research, a panel at Radboud University Medical Center ruled. But the authors of the open letter say Fieldlab should have followed ethical guidelines for research in the social and behavioral sciences, which stipulate that participants give their informed consent and researchers assess the potential drawbacks for individuals and society.”

"A year after lockdowns began, has research got any kinder? [E]veryone also shares stories of research lost and examples of a creeping unkindness in research culture that could have long-term implications for their research agenda and careers. These include: the inability to negotiate over strict deadlines imposed by institutions, journals, funding agencies and invitations to review or talk, leading to a decision to forgo the opportunity entirely; the increased emotional toll of pandemic teaching; relinquishing the opportunity to apply for promotion owing to a lack of time and energy; publications left unfinished because of a return to a culture of overwork." The author provides recommendations.

“Researchers worldwide have turned to analysing data, planning experiments and writing and reviewing more papers as their lab time has been disrupted…Nearly all respondents (97%) to a global survey of 2,000 researchers reported that the COVID-19 pandemic has affected their work — and half reported ‘significant’ impact — but most are staying productive despite the disruptions.’”

“Scandal over COVID vaccine trial at Peruvian universities prompts outrage: Researchers gave shots to politicians and family members, violating trial regulations — and damaging public trust”

"COVID is amplifying the inadequacy of research-evaluation processes: Systems for assessing scientists’ work must properly account for a lost year of research – especially for female researchers."

“How Has COVID-19 Affected Research Funding, Publishing and Library Budgets? Or, Finding the Truth in the Gap Between Perception and Reality”

US Department of Justice targets potential covid clinical trial fraud: “DOJ’s recent focus on curbing fraud in clinical research trials serves as a warning to sponsors, contract research organizations, and investigators to have robust compliance programs in place to prevent or identify misconduct. In addition to myriad clinical trials for new medications and medical devices treating any number of health conditions, there are currently over 3,000 pending clinical research studies related to COVID-19.”

Getting covid under control requires the public trust. But do biomedical researchers deserve their trust? The author addresses three questions: Are disseminated research results routinely reliable? Does research routinely create increased social value? Is research routinely conducted ethically? He states, “Employing deserved trust to support the public’s trust leads us to consider what it is that the biomedical research community should be trusted to do, examine the evidence about the effectiveness of current safeguards meant to assure that those things routinely get done, and identify new ways to equip individual researchers, research teams, and research institutions to assure that the public’s trust in their research is deserved rather than misplaced.”

How did a “Stanford University medical professor…famous for his rigorous assessments — and frequent debunking — of disease treatments” find himself garnering a “half-million views on YouTube” as well appearing on Fox News and CNN? John Ioannidis argued against covid lockdowns and promoted his own research that disputed covid’s severity. Said Steve Goodman, “Debates among scientists about the evidence are healthy. But if conducted in public the rules change…They can confuse people and undermine the consistent messaging needed for public health. Politicians can also misuse these debates to undermine public health policies they don’t like. The result? Our complete failure to contain covid-19.” Scientific American also published a major correction of a related story.

“The Academic Response to COVID-19”: survey in which “25,307 members of our academic community participated, representing diverse countries, roles, and areas of research.” In addition to covid-related findings, “There is concern about future pandemics, but researchers are equally concerned about climate change, which we can prepare for and mitigate with the help of science.”

Five barriers to publishing during a pandemic: barriers depend largely on where you sit

“International Collaboration During the COVID-19 Crisis… The most affected nations tend to produce the greatest number of coronavirus articles, with output closely coupled to the rate of infection. COVID-19 research has fewer nations and smaller teams than pre-COVID research, a trend which intensifies during the pandemic. “

French professor Didier Raoult, who promoted hydroxychloroquine as an effective covid treatment, without evidence, “will appear before a disciplinary panel charged with ethics breaches, an order of doctors has said.”

“Scholarly Societies in the Age of COVID… we spoke with chief executives and chief publishing officers at 12 large and small societies, some focused on the humanities and social sciences, others on the STEM fields…[to see] how the various sources of activity and value for scholarly societies were being impacted by the pandemic, how they were coping, and what they expected to see in the future.”

“Publish or perish: women in research call for an end to systemic discrimination: The global COVID-19 crisis offers universities the ideal pretext to change their practices and rethink their definition of academic work and its value.…[I]t is not merely a question of reducing expectations with respect to the traditional criteria, but defining new, non-discriminatory criteria that make it possible to appreciate the community, scientific and professional contribution of all faculty.”

From Scholarly Kitchen: “SSP Early Career Development Podcast: Episode 4, The Publishing Industry and the Pandemic Part 1”

What happens when research funding and peer review don’t respond quickly enough to outbreaks like covid?

Australia Academy of Science on open science:"[A]fter the Covid-19 pandemic there can be no return to closed working...Opening up research on the specific topic of the current crisis is really only a gesture. In the current pandemic, researchers are also searching for information from the past literature on many topics, such as on the properties of ventilators and face masks as well as on previous coronaviruses. These papers, especially older ones, have largely not been made available in, for example, the COVID-19 Open Research Dataset (CORD-19) of scholarly literature... This has left us with a research system that is much less efficient than it could be: duplicated funding, duplicated effort, and more time taken to solve big scientific and health problems. We now need a system that not only works for but also exploits the opportunities of a world where research is digital from beginning to end—where seamless connections are the key to fully interoperable research.

“Research Practices in the wake of COVID-19: Busting open the myths around open data…Over a third (32%) of academic researchers report that their research has been ‘extremely’ or ‘very’ impacted by the outbreak of COVID-19. This is higher than those working in professional settings (26%)…The disciplines affected most by COVID-19 at this point in time are those working in Chemistry (47%), Biology (39%), Medicine (36%) and Materials Science (36%). The lowest level of impact was reported in Humanities and Social Sciences (20%)”

The research and teaching academic appointment in the time of Covid: “Will this be the moment that leads to the creation of new employment models, perhaps ones that give equal prestige to teaching and to research? Or, more depressingly, are those academics lucky enough to keep their jobs destined to wilt under ever more unreasonable workload demands?”

Trend of women’s submissions declining and men’s increasing during covid continues

“How COVID-19 is Changing Research Culture: An interview with Daniel Hook, CEO of Digital Science”

"The Universities UK Jisc content negotiation strategy group is calling on major academic publishers to seek reductions of 25% on all agreements in light of the severe financial impact institutions are facing because of the pandemic."

Gender, race and parenthood impact academic productivity during the COVID-19 pandemic: “Here we show the influence of gender, parenthood and race in academics productivity during the pandemic period, based on a survey answered by 3,345 Brazilian academics from various knowledge areas and research institutions. Findings revealed that male academics - especially childless ones - were the least affected group, whereas female academics, especially Black women and mothers, were the most impacted group.”[preprint]

Research Data Alliance (RDA) COVID-19 Working Group Recommendations and Guidelines for Data Sharing: “The work has been divided into four research areas (namely, clinical, omics, epidemiology, social sciences) with four cross cutting themes (namely, community participation, indigenous data, legal and ethical considerations, research software), as a way to focus the conversations, and provide an initial set of guidelines in a tight timeframe. The detailed guidelines are aimed to help stakeholders follow best practices to maximise the efficiency of their work, and to act as a blueprint for future emergencies. The recommendations in the document are aimed at helping policymakers and funders to maximise timely, quality data sharing and appropriate responses in such health emergencies.”

“…Multiplicity of Randomized Clinical Trials for Coronavirus Disease 2019 Launched During the Pandemic…We found a high rate of trial multiplicity, particularly with chloroquines, which are being tested in 143 studies. Although these overlapping trials may afford opportunities for replication and validation, the high degree of multiplicity also enhances the likelihood of finding a positive result through chance alone, potentially resulting in widespread use of an ineffective and possibly hazardous intervention.”

“Data show panic and disorganization dominate the study of Covid-19 drugs…The findings show…that too often studies are too small to answer questions, lack real control groups, and put too much emphasis on a few potential treatments, as occurred with hydroxychloroquine.”

“Some Covid-19 trial sponsors never posted other study results in an EU database. Will they hide data again?…[T]wo-thirds of the 118 Covid-19 trials analyzed are currently run by universities and companies that have no record of uploading clinical study results to the European Union Drug Regulating Authorities Clinical Trials Database, or EudraCT, which trial sponsors are required to do so under EU rules. Of these, 39 trials are being run by sponsors that, in the past, violated requirements to upload results.”

Cherry picking citations: “Study Publications Must Be Up-To-Date in Covid Time & That New Hydroxychloroquine Study”

“Will COVID-19 be evidence-based medicine’s nemesis?... In a complex system, the question driving scientific inquiry is not ‘what is the effect size and is it statistically significant once other variables have been controlled for?’ but ‘does this intervention contribute, along with other factors, to a desirable outcome?’.”

“Assuring research integrity during a pandemic…First, the scientific community, both individuals and professional societies, should actively speak out when they feel the standards for integrity in research…have been compromised…Second, government advisory committees should be fully transparent…, disclosing all real and apparent conflicts of interest and clearly identifying and making open the research that supports their recommendations…Third, editors and authors should be proactive in assuring that the review status of their publications is clearly understood…Fourthly, there are calls for scientists to open up their research to critique early on in the process i.e. before data collection starts so the robustness of the study methods, materials and analytical approach can be first ‘peer reviewed’ either formally or informally.”

What might conferences look like in a post-covid world? “…[L]eadership has taken the approach that they will arrange to have, ‘a trusted healthcare partner’ test each attendee upon arrival, with the testing cost included in the registration fee. All attendees who test negative will then be allowed unfettered access to the conference, without any additional mitigation protocols. Anyone testing positive, will be immediately dismissed.”

US Centers for Disease Control and Prevention (CDC) considerations for colleges and universities during Covid-19

Which of the work, meeting, and educational changes brought by Covid-19 will result in permanent change?

"Waste in covid-19 research...An extraordinary number of covid-19 trials have been registered since the pandemic started. The National Library of Medicine registry ClinicalTrials.gov lists 1087 covid-19 studies, and though some will provide useful information, many are too small and poorly designed to be helpful, merely adding to the covid-19 noise. Of the 145 registered trials of hydroxychloroquine, for example, 32 have a planned sample size of ≤100, 10 have no control group, and 12 are comparative but non-randomised. Outcome measures vary widely, and only 50 seem to be multicentre. Strikingly, only one provides a protocol, and even limited registry details reveal unjustified outcome switching...The imbalance in trial topics is worrying, in particular the paucity of trials on non-drug interventions. Despite non-drug interventions being the mainstay of current mitigation,3 we could find just two trials of masks on ClinicalTrials.gov and none examining social distancing, quarantine effect or adherence, hand hygiene, or other non-drug interventions. Covid-19 research funding mirrors this woeful imbalance."

More on declining research submissions from women during covid: time to be flexible? "Hallam has heard back from a research journal to which she submitted a paper before the lockdown, wanting her to make revisions by early June that she can’t find the time to do. She notes the journal took two months less than usual to respond. 'Perhaps the reviewers aren’t parents,' she says drily. 'I wrote back explaining why I would struggle to make the deadline. The editor declined an extension, saying deadlines were important,' she says.

"Women in science are battling both Covid-19 and the patriarchy: The pandemic has worsened longstanding sexist and racist inequalities in science pushing many of us to say ‘I’m done’": A letter from 35 female scientists

Women and research during the pandemic: what the data say: "Frederickson looked at arXiv studies posted between 15 March and 15 April in 2019 and in 2020. The number of women who authored preprints grew by 2.7% from 2019 to 2020 — but the number of male authors increased by 6.4% over that period.... In 2 of the 3 registered-report repositories, covering more than 14,000 reports with authors whose genders could be matched, Sugimoto’s team found a decrease in the proportion of submissions by female principal investigators from March and April of 2019 to the same months in 2020, when lockdowns started."

"Pandemic researchers — recruit your own best critics ... Finding ways to prove ourselves wrong is a scientific ideal, but it is rarely scientific practice. Openness to critiques is nowhere near as widespread as researchers like to think. Scientists rarely implement procedures to receive and incorporate pushback. Most formal mechanisms are tied to the peer-review and publishing system. With preprints, the boldest peers will still criticize the work, but only after mistakes are made and, often, widely disseminated."

"What Will We Learn About Scholarly Publishing as a Result of COVID-19?" What will it mean for women in research, rapid review of Covid-19 research, and non-Covid-19 research papers?

"The pandemic and the female academic: I'm curious what lockdown will reveal about the ‘maternal wall’ that can block faculty advancement."

"Women academics seem to be submitting fewer papers during coronavirus. ‘Never seen anything like it,’ says one editor. Men are submitting up to 50 percent more than they usually would..."

 Are fewer women submitting research papers during the pandemic? 

Could the covid-19 pandemic lead to a kinder research culture?

"Northern hemisphere universities fear an explosion in academic misconduct as they approach the end of the academic year with little prospect of staging face-to-face examinations. But Dr Ellis said the backup tests cobbled together around the world could turn out to be a blessing in disguise."

How federated authentication was used to provide journal access to remotely-working academics: the complex process of providing individual level access to remotely-working faculty via institutional subscriptions

"China is tightening its grip on coronavirus research: Over the past two months, [China] appears to have quietly intro duced policies that require scientists to get approval to publish — or publicize — their results, according to documents seen by Nature and some researchers." Is this intended to improve quality of published research or control release of information about Covid's origins? 

"China clamping down on coronavirus research, deleted pages suggest: Two websites for leading Chinese universities appear to have recently published and then removed pages that reference a new policy requiring academic papers dealing with Covid-19 to undergo extra vetting before they are submitted for publication."

How is Covid-19 affecting researchers? "Never before, scientists say, have so many of the world’s researchers focused so urgently on a single topic. Nearly all other research has ground to a halt...As a practical matter, medical scientists today have little choice but to study the coronavirus if they want to work at all. Most other laboratory research has been put on hold because of social distancing, lockdowns or work-from-home restrictions."

"How are Academic Institutions Innovating Under Pressure"

"Scientific and Scholarly Meetings in the Time of Pandemic: This not only affects meetings revenue but can have ripple effects, impacting membership renewals, journal article recruitment, and other society activities...Professional networking is perhaps the most challenging aspect of in-person conferences to reinvent online, and yet one of the most important."

How can virtual conferences achieve the in-person networking of the live event?

What does COVID-19 mean for Chinese universities? 

MAY BE OF INTEREST (subscription or log in required)

Chinese Censorship Is Quietly Rewriting the Covid-19 Story

This Questionable Study Caught Fire in Anti-Vaccine Circles. How Did It Get Through Peer Review?

Reporting preprints in the media during the COVID-19 pandemic

Has the pandemic changed research culture – and is it for the better?

How academia shunned the science behind the Covid vaccine

Impact of COVID-19 on the Careers of Women in Academic Sciences, Engineering, and Medicine

After Covid-19 Data Is Deleted, NIH Reviews How Its Gene Archive Is Handled

‘Validity’ of journal impact factor highlighted by Covid effect

Wuhan lab leak 'now the most likely origin of Covid', MPs told

Research done wrong: A comprehensive investigation of retracted publications in COVID-19

---

Sources include Retraction Watch, Healthcare Information for All listserve, Open Scholarship Initiative listserve, and Scholarly Kitchen. For more information contact Margaret Winker, MD, Trustee, WAME.