WAME Newsletter #99, September 27, 2022

Providing the links does not imply WAME's endorsement. Please send any suggestions for content or format to [email protected]. See the Index Page for Newsletters 2020-2022 and the WAME Newsletter Compilation for content from earlier newsletters. Covid resources are also available.

ACCESS

"Removing author fees can help open access journals make research available to everyone...McGill University Library supports a no-fee, OA science journal called Seismica, which publishes peer-reviewed research in seismology and earthquake science....McGill Library is one of many Canadian libraries supporting journals in this manner. Of the nearly 300, no-fee OA Canadian journals we researched, 90 per cent were supported in some way by academic libraries."

"No evidence that mandatory open data policies increase error correction...Using a database of open data policies for 199 journals in ecology and evolution, we found no detectable link between data sharing requirements and article retractions or corrections."

AUTHORS

"Poor English skills? New AIs help researchers to write better...A growing suite of free or low-cost online tools can translate text, check spelling, correct grammar and even detect whether the tone of the text is appropriate, providing crucial aid to people who are not proficient in written English. These tools are powered by the same natural-language-processing models that underlie capabilities such as predictive text and voice-to-text transcription, and are largely targeted at ordinary users. Yet researchers can use them to polish their writing in everything from manuscripts and grant applications to social-media posts. The tools won’t write papers or applications for you, but they can offer scientists — particularly those for whom English is a second language — a strong grammatical steer…But “AI is not magic”, says Nishchay Shah, chief technology officer at Paperpal’s parent company, Cactus Communications in Mumbai, India. Not all the edits will make sense, he says, so researchers should go through the suggested changes one by one."

DIVERSITY, EQUITY, AND INCLUSION

Addressing diversity, equity, and inclusion at the Peer Review Congress 2022 [from Scholarly Kitchen]:

  • "Ana-Catarina Pinho-Gomes...reported on a cross-sectional study of 47 BMJ journals, which found that women comprised less than one third of peer reviewers. Journals with a higher percentage of women as editors, or a woman as editor-in-chief, also had a higher representation of women as peer reviewers.”
  • “Micah Altman and Philip Cohen...compared the editorial board composition of open access versus closed journals, using a dataset of 14,228 journal editorial boards. Across all journal types, less than one third (28.7%) were women, although this varied greatly by discipline...International diversity was higher in editorial boards of OA journals, however, board composition was disproportionately male and US- or UK-centric across both journal types.”
  • "In a study on the geographic diversity of reviewers conducted by Khaoula Ben Messaoud…, of the 257,025 invited reviewers across 21 BMJ Publishing Group biomedical journals. 90,467 (35.2%) agreed to review. ‘Compared with high-income countries, agreement was higher among reviewers from lower middle–income countries (3.26 95% CI, 3.06-3.48) and low-income countries (2.99 95% CI, 2.57-3.48) (P < .001).’”
  • "Holly Falk-Krzesinski…provided a promising update on the multi-publisher Joint Commitment for Action on Inclusion and Diversity in Publishing
  • Is greater representation of individuals from LMICs on editorial boards associated with more publications from LMICs? "[B]ased on their analysis of 45 journals, [Gandolina Melhem et al] concluded that there is indeed a moderate increase."
  • "Annette [Flanagin] et al set out to determine the impact of the JAMA Network Updated Guidance on Reporting Race and Ethnicity in Medical and Science Journals... Of the 249 articles included in their analysis, more reported how race and ethnicity were determined after the implementation of the new guidance, however, race and ethnicity were still underreported compared to other demographic information such as age, sex, and gender."

"Sitting through many hours of presentations, it became increasingly frustrating to hear what is so often unspoken: that researchers struggle to define accountability and responsibility in the context of the scientific process...Perhaps to the casual observer of the scientific publication process it seems to suggest energy is disproportionally expended on the generation of a publication/series of publications rather than into ensuring accuracy, completeness, and transparency."

"Open access to research can close gaps for people with disabilities...The impact of inequitable access to research information cannot be overstated. People with disabilities are grossly underrepresented across science. Although 27% of U.S. adults have a disability, only 9% of the scientific workforce is disabled."

"Women still publish less than men in South Africa...[T]he proportion of women academics grew from 44.1 per cent in 2005 to 49 per cent in 2020. Over that period their share of publications remained 12-14 percentage points lower than their proportional headcount...South Africa also has disparities along racial lines. The contribution of Black female authors increased over the reporting period from 4 percent of all South African-authored papers in 2005 to 18 per cent in 2020. However, the paper found that white women were more than twice as productive in 2020 than Black women academics in terms of research outputs."

EDITING

"Good Publication Practice (GPP) Guidelines for Company-Sponsored Biomedical Research [organized by the International Society for Medical Publication Professionals]: 2022 Update...The GPP guidelines apply to peer-reviewed or peer-oriented biomedical publications, such as manuscripts, meeting presentations, posters, and abstracts, as well as enhanced content, such as plain-language summaries. The current GPP guidelines incorporate guidance on ethics and transparency as well as the planning, development, review, and approval of biomedical publications and policies and procedures that describe these practices." More detailed recommendations available in the Supplement.

"Editors-in-chief of aging journal resign en masse after ‘impasse with the Anatomical Society and Wiley’...Many of those problems, the editors wrote, involved fallout from a steep increase in workload as the journal’s stature has risen...One solution was to pay the volunteer section editors 'based on the number of manuscripts they handle.' That fix, according to the group, was ignored, with a counter-offer of relying instead on ‘early career’ scientists.Meanwhile, the editors-in-chief said they were still being paid for their time at a rate set in 2006: 2,000 GBP, or about $2,619, per year. The letter also points out that Wiley eliminated the editors’ 'modest' budget to travel to meetings in 2022 – an important part of the job, they said – because they missed the deadline to apply for the funds last year."

"We need to talk about editors...Even if an editor starts off well, they may over time start to think 'What’s in this for me?' and decide to exploit the opportunities for self-advancement offered by the position. The problem is that there seems little pressure to keep them on the straight and narrow; it's like when a police chief is corrupt. Nobody is there to hold them to account...we see clearcut instances of paper mill outputs that have apparently been approved by a regular journal editor...some preliminary suggestions: 1. Appointment to the post of editor should be made in open competition among academics who meet specified criteria. 2. It should be transparent who is responsible for final sign-off for each article that is published in the journal. 3. Journals where a single editor makes the bulk of editorial decisions should be discouraged...4. There should be an editorial board consisting of reputable people from a wide range of institutional backgrounds, who share the editorial load, and meet regularly to consider how the journal is progressing and to discuss journal business. 5. Editors should be warned about the dangers of special issues and should not delegate responsibility for signing off on any papers appearing in a special issue. 6. Editors should be required to follow COPE guidelines about publishing in their own journal, and publishers should scrutinise the journal annually to check whether the recommended procedures were followed 7. Any editor who allows gibberish to be published in their journal should be relieved of their editorial position immediately."

Editor of Anesthesia identifies data pattern consistent with fabrication in a submitted manuscript; subsequently published works by the author are retracted

PEER REVIEW

“A quick rundown from the 9th Peer Review Congress…Trust and Transparency in Peer Review Models” [from HighWire], including open peer review pros and cons (with “122 definitions…analyzed and categorized”); Registered Reports with peer review occurring first after the design phase; efficacy of peer reviewer guidance and training; and Systematizing Confidence in Open Research and Evidence (SCORE), “about developing artificial intelligence algorithms that flag potential issues in research papers, particularly in regard to replicability and reproducibility. These algorithms can be used to help peer reviewers and editors detect aspects of a research paper where they need to engage in closer examination. This initiative is still in early phases, with a prototype currently in testing and an open algorithm competition underway.”

Peer Review Week essays [from Scholarly Kitchen]:

  • “We Asked the Community: Is Research Integrity Possible without Peer Review?...Peer review has failed to prevent the publication of racist, homophobic, transphobic, ableist, and otherwise oppressive “research.” So, in this context, research integrity is possible without peer review because peer review doesn’t address this well...Some first steps can include: Learning about how systemic racism operates in scholarly publishing and its impact on communities and scholars of color; Increasing diversity and representation within editorial boards and peer reviewer pools; and Ensuring individuals who hold marginalized identities are participating in the review of research about their communities…[R]esearch integrity is more about the actions of the researcher, than any safeguards we try to put into place. Bad players need to be called-out with greater severity and frequency in order to curb unethical publication practices.”
  • "Is Research Integrity Possible without Peer Review?" A researcher says the answer isn't as clear as one might think: "[p]eer review tends to have generally low levels of inter-rater reliability, making it, statistically speaking at least, relatively capricious."
  • “Does Trust in Research Begin with Trust in Peer Review?...It’s a mistake to let peer review become synonymous with peer-reviewed science articles...The caricature of objective up or down, right or wrong masks too much that is, in fact, about using a human process to get things as right as we can with what we know now...'transparency is about making sure that we are able to reveal the information necessary to bring a level of trust and efficiency to the peer review process.'"
  • "Peer Review and Research Integrity: Five Reasons to be Cheerful"...reproducibility and registered reports, hearings on paper mills, and retractions, and how publishing is using systematic approaches to improve evaluation of research integrity

"The [Elsevier] Peer Review Workbench...is a platform for providing responsible access to interested academics and researchers to the enhanced and enriched Elsevier journal and manuscript metadata inside our ICSR Lab, which is dedicated to conducting evidence-based research on peer review. Researchers and academics can apply to access metadata for manuscripts in Elsevier journals in order to run systematic analyses on peer review processes in different disciplines at scale."

“Does peer review improve the statistical content of manuscripts? A study on 27 467 submissions to four journals” using frequency of statistical terms as a proxy for statistical content. “Our results suggest that manuscripts with both initial lowest or highest levels of statistical content increased their statistical content during the process, whereas desk-rejected manuscripts had comparatively fewer statistical terms in their text. We found that these developments were associated with a higher probability of a manuscript’s acceptance. The availability of reviewer guidelines on statistics on review forms seems to ensure similar initial levels of statistical content among submitted manuscripts but did not have any qualitative implication on manuscript change during peer review. We found that editors were more likely to reject manuscripts when reviewers concentrated more on the statistical content of manuscripts in their reports.”

“Writing constructive peer review reports...10 tips to help you write constructive peer reviews.”

PREDATORY/DECEPTIVE JOURNALS AND CONFERENCES

Predatory conferences continue to spread: “These events are being organized mostly by a relatively small number of large, international organizations, although smaller companies have recently entered the industry. Senthil Gopinath, CEO of the International Congress and Convention Association, a trade group for the association meetings industry, has commented on the scale and impact of predatory conferences: ‘Tens of thousands of terrible quality and sometimes fraudulent conferences are today being promoted around the world, which presents an industrial-scale challenge to bona fide associations and their quality education programs. It’s a global phenomenon, which today impacts negatively on almost every scientific discipline.’” Before signing up, the author recommends: “Research the organization putting on the event...Spend some time on the event website...Consider the sponsors...Connect with the event organizers.”

PREPRINTS

"What is better for your career than a publication? A preprint" The author argues that speed, access, curriculum vitae building, enhanced collaboration opportunities, and early marketing support preprinting as long as the journal to which one will ultimately submit considers preprinted manuscripts.  

PUBLISHING

New US Office of Science and Technology Open Access Policy [aka the Nelson Memo]:

  • "Zero Embargo...the key aspects of the [OSTP] directive include...This is (presently) an unfunded mandate...The memo describes a “public access” policy, not an “open access” policy...The policy applies to all authors on a publication, not just corresponding authors...The memo does not clearly specify what version of research papers this policy will apply to...The memo is not just applicable to original research papers…Immediate public access to the underlying data is required...Metadata is a focus...The policy goes into effect by 2026."
  • From Scholarly Kitchen:
    • "Some Initial Observations and Questions...Neither embargoes – of any length – nor the idea of 'leeway to tailor' plans in light of 'challenges and public interests that are unique to each field and mission combination,' nor a possibility of petitioning for exceptions is contemplated in the Nelson Memo."
    • "OSTP Policy Part I...Publishers who have been paying attention generally assumed that immediate public access to papers and open data requirements were in the pipeline, and that it was a matter of when rather than if. With these policies released, timelines are set, and planning that was hopefully in place can be accelerated. It is noteworthy that this is a set of policies written with seemingly little input from publishers nor the same level of consultation with key stakeholders as was undertaken for the Holdren Memo. The fact that this is now the second consecutive US administration that chose to craft research publishing policy in this manner speaks volumes about the poor relationships between the research publishing industry (particularly the industry’s advocacy organizations) and the federal government."
    • "OSTP Policy Part II...the OSTP Memo outlines that publications not only be made available, but made available “in formats that allow for machine-readability” that also allows for accessibility for the print-disabled. NISO’s JATS standard is called out as an appropriate format in wide use for this...Much of what constitutes the FAIR principles is centered around metadata and identification for discovery, use, and reuse. What is troubling from a systems perspective, however, is that these elements are not equally supported by a directive for funding, in the way that the public access provisions for content may be...Perhaps the attention on these issues at the OSTP level will push national funding toward these systems in a more systematic way, rather than relying on the publishing ecosystem as their primary funding source."
    • "Quantifying the Impact...The policy affects about 31% of US papers and 7% of papers published globally. Some of the papers in scope are already published in a gold-OA format (33%). Converting all underlying papers to a gold-OA format will be a significant contribution toward the global transition to OA (my working assumption for these estimates posits that ¾ of the papers that are not currently gold-OA, will turn gold-OA as a result of the policy). Yet the true impact of the policy may be greater than these numbers imply, given that several of the leading, mostly paywalled scholarly titles (Nature, Science,Cell, PNAS) get more than 40% of their papers from US, federally-funded research...[T]he EU and the UK publish 50% of their papers in a gold-OA format. These papers are in journals that typically charge APCs. Europe’s performance places it well ahead of the USA and China, which publish 30% and 33% of their content, respectively, in a gold-OA format.”
    • "The Outlook for Data Sharing...the Nelson Memo calls for immediate public access to data supporting publications, and asks agencies to develop 'approaches and timelines for sharing other federally funded scientific data that are not associated with peer-reviewed scholarly publications.' "

"Tearing down the academic research paywall could come with a price...The established journals are reluctant to commit to open access, since submission fees may deter potential researchers from sending in their work. And if journals don’t charge submission fees or reader subscriptions, they’ll have to turn to other sources of income, which may be unsustainable in the long run."

"The scramble for science...Audrey Smith and colleagues from the University of Florida reported last year a study of more than 37 000 articles from Elsevier's 'mirror journal' system. In this arrangement, a parent hybrid journal has a gold open access mirror. When the two journals—one open access, one not—were compared, the geographical diversity of authors was significantly lower for open access papers... At Lancet journals, we regularly agree [to] APC waivers. But Smith and colleagues note that in their study waivers were clearly failing to encourage submissions from authors in lower-income settings."

"Beyond the 'Accidental Profession': Bringing More Structure, Equity, and Respect to Scholarly Publishing Employment...mission-driven scholarly publishers thrive in an astonishing variety of habitats. In North America, these include public and private universities and colleges; military service academies; almost every learned society; museums and galleries; non-governmental organizations and think tanks; research institutes; and hundreds of libraries." Plus a Scholarly Kitchen podcast series on early career development in publishing

Stanford professor who sued journal for defamation for publishing an article critical of his research “is appealing a recent court order that he pay the journal’s publisher more than $400,000 in legal fees”

RESEARCH INTEGRITY AND TRANSPARENCY

"Identifying and managing problematic trials: A research integrity assessment tool for randomized controlled trials in evidence synthesis...Through iterative discussion, we developed a research integrity assessment (RIA) tool for randomized controlled trials for the update of this Cochrane review...RIA assesses six study criteria: study retraction, prospective trial registration, adequate ethics approval, author group, plausibility of methods (e.g., randomization), and plausibility of study results."

"Promoting trust in research and researchers: How open science and research integrity are intertwined...We argue that responsible research practices focus more on the rigorous conduct of research, transparency focuses predominantly on the complete reporting of research, and open science’s core focus is mostly about dissemination of research… we provide suggestions on what researchers and research institutions can do to promote a culture of research integrity. We close with a brief reflection on initiatives by other research communities and stakeholders."

"Unprecedented! Incredible! Scientific grant applications are getting heavier on hype...130 out of 139 hype adjectives increased in prevalence by an average of over 1,300% percent in successful applications submitted to the National Institutes of Health between 1985 and 2020."

"Stakeholders’ Experiences of Research Integrity Support in Universities: A Qualitative Study in ...Dutch, Spanish and Croatian universities...Experiences of support differed between countries in relation to: the efforts to translate norms into practice; the extent to which RI oversight was a responsibility of RE structures, or separate RI structures; and the availability of support close to research practice, such as training, responsible supervision, and adequate tools and infrastructure."

“Scientists apply their tools inconsistently. They can learn from the law...The vast majority of neuroimaging studies are underpowered and rarely produce results above noise. The odds that an average neuroscience study is true is 50-50 or lower, according to a 2013 review. And an estimated 50% of studies in biomedicine “have statistical power in the 0–10% or 11–20% range, well below the minimum of 80% that is often considered conventional.” P-hacking and hypothesis fishing are rewarded because they make for more remarkable, and therefore more publishable, results...The legal system, by contrast, is obsessed with questions of proof and evidence. Lawyers have rigid structures to test the hypotheses underlying a legal battle. Lawyers are trained to study and argue both sides of a case, carefully finding holes in each. Evidence unrelated to a case can be thrown out by a judge, who is supposed to be an unbiased referee of competing claims. Science is inconsistent, with ad hoc rules, in comparison.

"How (not) to be held accountable in research: The case of the Dutch integrity code...The conclusion is that this code is based on a flawed conception and an inadequate analysis of the nature and role of values and norms in science."

“[C]ommon problems in western blot figures and methods reporting with recommendations to improve them…Our data show that most published western blots are cropped and blot source data are not made available to readers in the supplement. Publishing blots with visible molecular weight markers is rare, and many blots additionally lack molecular weight labels. Western blot methods sections often lack information on the amount of protein loaded on the gel, blocking steps, and antibody labeling protocol. Important antibody identifiers like company or supplier, catalog number, or RRID were omitted frequently for primary antibodies and regularly for secondary antibodies.” The authors provide descriptions and examples, plus a toolbox for scientists, “teaching slides in English and Spanish, and an antibody reporting template.”

"It takes a laboratory to avoid data loss...Academia heaps most of the burden of documentation and data storage onto individuals, instead of the lab as a whole. At the same time, little, if any, instruction is provided to teach individuals how to properly document and store their data. But labs can mitigate data loss by implementing three simple suggestions...Establish standards...Manage personnel transitions...Maintain institutional knowledge."

"PETA [People for the Ethical Treatment of Animals] Publicizes Reports to OLAW, USDA Inspections, Targets [US National Institutes of Health] NIH Intramural Program...The flurry of recent activity is part of PETA’s strategy to hold NIH and funded institutions’ feet to the fire when there are documented violations of either NIH guidelines for animals used in research or of the Animal Welfare Act (AWA), which a USDA division enforces."

RETRACTIONS

"Reducing the Inadvertent Spread of Retracted Science: recommendations from the RISRS report...The RISRS recommendations are: (1) Develop a systematic cross-industry approach to ensure the public availability of consistent, standardized, interoperable, and timely information about retractions; (2) Recommend a taxonomy of retraction categories/classifications and corresponding retraction metadata that can be adopted by all stakeholders; (3) Develop best practices for coordinating the retraction process to enable timely, fair, unbiased outcomes; and (4) Educate stakeholders about pre- and post-publication stewardship, including retraction and correction of the scholarly record."

SCIENTIFIC MISCONDUCT

“A quick rundown from the 9th Peer Review Congress…Author and contributor misconduct” [from HighWire]:  

  • Di Girolamo et al "looked at self-reported CRediT statements, and measured those statements against two different sets of authorship guidelines to determine authorship authenticity [ICMJE and a "looser adaption" of ICMJE]...approximately one third of the authors did not qualify for full authorship, based on ICMJE guidelines. The percentage dropped considerably when the looser guidelines were applied, but they were still significant. The good news is that the percentages consistently decreased from 2017 to 2021 in both cases.”
  • "Daniel Evanko presented Use of Artificial Intelligence-based Tool for Detecting Image Duplication Prior to Manuscript Acceptance...In a comparison of time required to screen for image reuse, the average analysis time for manual checking was 8 minutes, while the AI-supported process took 4.4 minutes. Effectiveness was also improved using AI, with manual checking detecting 5 duplications in 3 manuscripts, and AI screening catching 11 duplications in 8 manuscripts. One of the findings that I found encouraging is that most duplications were unintentional, and authors were often able to supply the correct image. It seems that many labs use image and data repositories and do not practice good data hygiene."
  • "Publication and Collaboration Anomalies in Academic Papers Originating from a Russian-Based Paper Mill, given by Anna Abalkina, examined the work of a Russian paper mill by looking at the commercial offers to researchers being made by that organization, and then finding published papers that matched those offers. The study found that 451 papers were published in 159 journals. More that 6000 authorship slots were sold to around 800 different scholars from at least 39 countries. Because the paper mill usually targeted different journals each time, it would be hard to detect these fraudulent papers using simple similarity checking."
  • "Effect of Alerting Authors of Systematic Reviews and Guidelines That Research They Cited Had Been Retracted, presented by Alison Avenell...The researchers found 27 retracted reports of clinical trials, and then performed literature searches to find systematic reviews that cited those trials. They found 88 systematic reviews that fit their criteria, and after evaluating the impact of the retracted article, the researchers found that 44% of the reviews were significantly impacted if the retracted trials were removed. The study also evaluated author and editor reactions and actions after being notified multiple times of the retracted trials. The results here were disappointing as very few corrections or even acknowledgement of the notification was made by the authors or the journals."

‘Papermill alarm’ software flags potentially fake papers...The tool, called the Papermill Alarm, was developed by Adam Day, who is director of scholarly data-services company Clear Skies in London, UK. Day says he ran all the titles listed in citation database PubMed through the system, and found that 1% of currently listed papers contain text very similar to that of articles produced by paper mills."

“A physics publisher is retracting 494 papers after an investigation ‘indicated that some papers may have been created, manipulated, and/or sold by a commercial entity’ – aka a paper mill.”

"The Research Conduct Spectrum": “rule bending” practices that may help careers but harm the science

"Detecting anomalous referencing patterns in PubMed papers suggestive of author-centric reference list manipulation [RLM]... Approximately 16% of all authors may have engaged in RLM to some degree. Authors who use 18% or more of their references for self-citation are significantly more likely to have NSC Gini distortions, suggesting a potential willingness to coerce others to cite them."

"Nearly half of respondents…admit to plagiarism" in Moroccan study of "university professors, PhD researchers and postdoctoral fellows, PhD students, scientific graduates and undergraduate students...About half of the respondents admitted to having plagiarised text in some way...Unintentional plagiarism is particularly noted among scholars whose native language is not English."

“Medical journals broaden inquiry into potential heart research misconduct… Three medical journals recently launched independent investigations of possible data manipulation in heart studies led by Temple University researchers…adding new scrutiny to a misconduct inquiry by the university and the U.S. government.” Meanwhile one of the authors involved “fight[s] over a claim of stolen heart research and a start-up that’s selling for $53 million.”

MAY BE OF INTEREST (Subscription or registration required)

Sources include Retraction Watch, Scholarly Kitchen, Open Scholarship Initiative listserve, and STAT Newsletter.

Margaret Winker, MD
Trustee, WAME
***
wame.org
@WAME_editors
www.facebook.com/WAMEmembers