2

Do Sponsoring Organizations Influence Research?¹

To serve as a foundation for the workshop, Lisa Bero, chief scientist at the Center for Bioethics and Humanities, professor of medicine and public health at the University of Colorado Anschutz Medical Center, and the Cochrane Collaboration’s senior editor for research integrity, summarized the evidence on sponsor influence. Research is a cyclical process that involves designing questions and methods, conducting research, and reporting the results, which generates further questions for additional research. If any of these steps fail, the value of the resulting evidence, synthesis, systematic reviews, and health and public health guidelines may be questionable. No aspect of the process should be unduly influenced or biased, she said. Bias is defined as a systematic error, or deviation from the truth, in the results of a study. It can overestimate or underestimate the true effect of an intervention, depending on the research question. It could affect clinical trials on a drug, for example, by producing results and conclusions that overestimate efficacy or underestimate harm.

When studying bias, Bero uses a technique she called “meta-research,” which is research on research. As an example, she cited a Cochrane systematic review in which she and her colleagues examined bias in 75 industry-sponsored drug studies. One analysis explored whether the reported statistical significance of drug efficacy estimates differed by sponsor. In other words, are some studies more likely to have statistically

___________________

¹ This section is based on the presentation of Lisa Bero, University of Colorado Anschutz Medical Center.

significant efficacy results, an outcome that would be favorable for the drug’s developers, when the developers sponsor the trial? This analysis included 25 papers covering almost 3,000 studies and compared studies with drug industry sponsors with all of those with other sponsors, mostly government and nonprofit organizations. Industry-sponsored studies were about 30 times more likely than non-industry sponsored studies to report statistically significant efficacy estimates (Lundh et al., 2017). Bero pointed out that there was no significant difference in the risk of bias between studies with industry sponsors and other sponsors.

Another study she cited examined whether the effect sizes of nutrition study results differed by sponsor. The investigators looked at the association of dairy intake and cardiovascular disease (Chartres et al., 2020) and found that industry-sponsored studies had a larger effect size, a favorable result for dairy intake. “The non-industry-sponsored studies are almost seeing no effect when we combine them, and the industry-sponsored studies have a larger effect,” explained Bero. She added that detecting effect size difference in meta-research studies is difficult because many factors can influence it.

Meta-research studies often examine study conclusions; one of the first meta-research studies looked at whether the conclusions of reviews on secondhand smoke differ by sponsor. Bero and her colleague examined topic, year of publication (to account for the results changing as evidence accumulates), whether or not the review was peer reviewed, and whether the review was sponsored by industry or a different sector. The only factor associated with the conclusions of these reviews was the sponsor. “Tobacco industry-sponsored reviews were almost 90 times more likely to conclude that secondhand smoke was not harmful,” said Bero. “That raised some red flags.”

Similarly, another meta-analysis found that industry-sponsored research or author COIs are associated with results and conclusions that favored the sponsor (Chartres et al., 2016). This affects a study’s results and the authors’ conclusions, said Bero, and is called “funding bias.” Bero noted that the results of a study (not the conclusion) are important for conducting systematic reviews and formulating health and public health guidelines. However, conclusions are often picked up by lay media and read by those wanting a quick summary of evidence. She added that meta-research provides answers about an observed bias in results or conclusions, except when all the studies are sponsored by companies that made the drug. Furthermore, meta-research studies demonstrate that funding bias exists but do not explain the mechanisms by which it can occur.

HOW DOES BIAS HAPPEN?²

Bero and colleagues (Odierna et al., 2013) wrote about what they called the “cycle of bias” that identifies four points in the research process where bias can appear: when the agenda is set and how the question is asked, the method and its internal validity, how the study is actually conducted, and whether the results are published in full (Odierna et al., 2013). What is published, said Bero, influences additional questions and meta-analyses and systematic reviews.

Methods other than meta-research have been used to study industry influence within the cycle of bias. One approach looks at study protocols or full reports, such as those a sponsor submits to the Food and Drug Administration (FDA) or that are part of human ethics research protocols and compares them to what is published. Another approach is to analyze internal industry documents released through legal settlements, which describe companies’ scientific research and publication strategies. This method was used with the tobacco industry and then the pharmaceutical and chemical industries. Examining these documents has provided good insights into what the mechanisms of bias might be, said Bero. Another approach is to interview the researchers and funders to determine how and where bias occurs.

The Research Agenda

One source of bias is when industry sponsorship influences the research agenda, described Bero. She and her colleagues conducted a

___________________

² This section is based on the presentation of Lisa Bero, University of Colorado Anschutz Medical Center.

scoping review of all studies that examined the influence of the research agenda by sponsors and found that the agendas rarely align with public health questions or prevention. For example, research on gambling funded by the gambling industry tends to emphasize psychological or genetic characteristics associated with gambling addiction and the genetic predisposition hypothesis. This is similar to the research the tobacco industry conducted that promoted the idea that tobacco addiction was related to genetic factors, not the product itself, said Bero.

She acknowledged that sponsors can fund whatever study they want, but for meta-research or systematic reviews, it is important for the researcher to know that the research sponsor may provide a skewed body of evidence on the research questions and the topic area as well. For example, meta-research on nutrition studies found that food industry–funded studies were more likely to focus on micronutrients or minor components rather than on whole diets or dietary patterns. “You can see how this would have a commercial interest, because then those elements can be manipulated within the products, whereas the companies do not profit directly from whole dietary patterns,” said Bero.

One analysis of studies funded by Coca-Cola, for instance, found that it was much more likely to fund research on exercise than on sugar (Fabbri et al., 2018). Another study of research funded by five tobacco companies through the Center for Indoor Air Research identified both peer-reviewed and non-peer-reviewed projects (Barnes and Bero, 1996). The latter, which were not published in the scientific literature, were used in court cases and focused on secondhand smoke. None of the much larger body of peer-reviewed research funded by the center focused on secondhand smoke; rather, it addressed subjects such as whether carpet off-gassing was harmful or having a plant in the office improved indoor air quality. These two examples, said Bero, “show how industry sponsors can drive a research agenda toward what we call ‘distracting research,’ or research that distracts from harm of their product.”

Research Methods

Bero then discussed whether the observed bias identified by meta-research studies can be explained by differences in the methods used by industry and non-industry sponsors. In meta-research studies of drug studies, for example, the methods assessed include whether studies were appropriately blinded or randomized and usually do not differ by sponsor, largely because they are often regulated. However, Bero explained, it is important to examine whether standards (including statistical information) differ by sponsors, given that sponsors have become involved in setting standards. For example, the tobacco industry promoted standards

that would favorably evaluate and apply to secondhand smoke (Baba et al., 2005). Its approach was to argue for a larger effect estimates (e.g., odds ratio) for harm than for efficacy. “Of course, that ignores all sorts of things about population risk and the level of exposure that is happening,” said Bero. She noted that none of this planning was public until litigation led to the tobacco industry releasing internal company documents.

Industry was also heavily involved in crafting the Brussels declaration on ethics and principles for science and society policy making (Kazatchkine et al., 2017). Of the 165 names on the declaration, 26 were affiliated with the tobacco or alcohol industries, and the declaration lined up with an action plan for influencing science policy developed by the tobacco industry 20 years earlier. Release of the Brussels declaration prompted Bero to write an editorial listing 10 tips for spotting industry involvement in science policy (Bero, 2019). The lesson, she said, is to be aware of where the standards are coming from and who develops them.

Conduct of Research

Bias can arise when investigators do not conduct their studies according to the research protocol. Internal pharmaceutical industry documents, said Bero, show that research and scientific publication are part of its marketing strategy, and the goal is to use research to disseminate information widely through the medical literature (Steinman et al., 2006). This strategy makes her worry about whether what is getting published is really research. To control for this possibility, many journals have introduced statements regarding sponsor involvement, but Bero questions whether this guarantees that nothing is going on behind the scenes.

When Bero and her colleagues interviewed the lead academic investigators for 200 industry-funded drug trials, all of which had statements to the effect that the sponsor had no role in study design or conduct, 92, 73, and 87 percent of investigators said the sponsor was involved in study design, data analysis, and reporting the findings, respectively (Rasmussen et al., 2018). In addition, only 33 percent of the 80 authors commented that the author had the final say on what appeared in the publication. “This is a little alarm bell about these statements that we see in papers,” said Bero. “It really should not assure us that there is nothing going on behind the scenes in terms of industry involvement in the conduct of these studies.”

Publication

Despite many approaches to determine whether studies are published in full, the most stunning information comes from internal company documents, said Bero. One set, for example, showed that a sponsoring

pharmaceutical company would publish the results only if they showed a statistically significant finding for drug efficacy. When Bero and her collaborators reviewed internal documents and the medical literature, they found that a cluster of studies that produced statistically insignificant results were not published, in contrast with those with statistically significant results for the same drug (Vedula et al., 2009). In addition, they found instances where the pre-specified primary outcome was subsequently replaced with a different outcome which was statistically significant. This, said Bero, is “a big no-no.” Of the 19 industry-sponsored trials of gabapentin for neuropathic pain, selective outcomes were published in four papers, selective analysis occurred in 11, and seven were not published at all (publication bias). She noted that detecting this kind of bias when reading a paper is difficult and requires meta-research or comparing publications to protocols.

Though she used drug company examples to illustrate the four elements of the cycle of bias, Bero noted that publication bias is not limited to that industry. A study of internal corporate documents on perfluorinated chemicals that she is conducting with colleagues at the University of California, San Francisco (UCSF) found that industry was late to publish studies documenting the influence of perfluorinated chemicals on human health. Given these chemicals remain in the environment for a long time, that is neglegent, said Bero.

BREAKING THE CYCLE OF BIAS

Bero provided a list of steps that could address the problem of sponsor-associated bias:

• Publicly prioritizing research agendas and funding.
• Recognizing industry funding and COIs as a source of bias and account for it.
• Requiring open data for all published protocols and registered reports.
• Eliminating sponsor-associated bias at a structural level through policy.
• Rethinking funding and COI disclosures.
• Establishing independent publishers of research.

For additional potential solutions, she referred workshop participants to some of her publications on commercial influence in health (Lexchin et al., 2021; Moynihan et al., 2019).

DISCUSSION

When asked about the importance of nonfinancial factors with regard to COIs, Bero explained that many of the science policy or evaluation tools that industry is driving emphasize these factors and ignore the large body of evidence showing that financial COIs lead to systematic biases in research. For her, papers that emphasize nonfinancial factors in COI statements are a red flag that the industry might be involved (Bero and Grundy, 2016). “Some of it has really gotten just quite ridiculous, when we know there is a large body of evidence showing that financial conflicts of interest influence nutrition research, and to argue that whether somebody is a vegetarian or practices yoga has more influence than [financial COI], there is just no evidence for it,” said Bero. She noted that she and a colleague published a paper on managing COIs in guideline development that provided evidence-based guidance on how to rate the level of risk associated with different types of interests (Parker and Bero, 2022).

As for improvements, Bero noted a move to provide more open access to data, particularly on drug trials, and big changes to disclosure policies, but disclosure is not a way to eliminate these biases, only to facilitate studying them. Unfortunately, she said, she has not seen a trend of decreasing industry sponsorship, despite changes in how industry money funnels into a university, which is something to watch.

Session moderator Lonnie King, dean emeritus of the Ohio State University College of Veterinary Medicine, asked Bero if sponsors predetermine that they will have a say in the final publication. Bero replied that what she has seen is sponsors occasionally wanting to look at a paper before it is published and including that in contracts. What researchers are telling her, though, is that funders are coming back and suggesting changes to the contract.

This page intentionally left blank.