3

Example of Funder Influence on Health Research

Six speakers presented examples of how sponsors influence health research. David Michaels, professor at the Milken Institute School of Public Health at George Washington University, discussed the rise of corporate disinformation about harms. Adrian Hernandez, vice dean for clinical research at the Duke University School of Medicine, shared his perspective on the multiple entities that can influence research results. Laura Schmidt, professor of health policy in the UCSF School of Medicine, addressed industry funding bias in nutrition science. Martin McKee, professor of European public health at the London School of Hygiene and Tropical Medicine, provided many cases in which important evidence does not get included in research papers and hence is not part of the systematic reviews and meta-analyses that inform policy. Adam Dunn, associate professor of medicine and health at the University of Sydney, discussed how artificial intelligence (AI) can help fill in the blanks that McKee identified. Finally, Dean Schillinger, professor of medicine at UCSF, told a story about American Beverage Association (ABA) influence on diabetes research. Cary Gross, professor of medicine and public health at Yale University, joined the panelists for a discussion moderated by Ross McKinney, chief scientific officer at the Association of American Medical Colleges (AAMC).

PROTECTING PUBLIC HEALTH IN THE FACE OF CORPORATE DISINFORMATION¹

David Michaels noted that it is now standard operating procedure for corporations to create and disseminate disinformation by hiring “product defense” experts to manufacture scientific uncertainty about potential harms caused by their products or activities. Some call this “doubt science,” as it creates uncertainty. This practice, said Michaels, traces back to the tobacco industry’s strategy to counter the idea that smoking caused illness, which became important once researchers began publishing studies in the early 1950s showing the relationship between cigarette smoking and lung cancer. Hill and Knowlton, a public relations firm, advised the industry on how to manufacture uncertainty to convince people that evidence was insufficient to stop smoking or regulate tobacco. The details of this strategy were revealed in a memo,² part of the trove now housed at the Truth Tobacco Industry Documents library at UCSF, stating that “doubt is our product since it is the best means of competing with the

___________________

¹ This section is based on the presentation of David Michaels, George Washington University.

² Available at https://www.industrydocuments.ucsf.edu/docs/psdw0147 (accessed February 2, 2023).

‘body of fact’ that exists in the minds of the general public. It is also a means of establishing a controversy.”

Michaels explained that creating controversy and confusion is the basic idea behind many examples of corporate disinformation activities. The tobacco industry, said Michaels, did everything it could to say other factors cause lung cancer and that not everyone who smokes will get lung cancer. It created its own newsletter, Reports on Tobacco and Health Research, that it sent to physicians and researchers and featured stories such as “Lung Cancer Rare in Bald Men” and a report on a study that supposedly showed that small babies born to mothers who smoked were less likely to die than those born to nonsmokers.

The fossil fuel industry took the same approach regarding climate change, explained Michaels, and even funded an entire industry of people who look like scientists and claim scientists disagree about climate change. He noted that internal papers from fossil fuel companies revealed they all had a tremendously accurate understanding of what would happen if humanity kept pumping greenhouse gases into the atmosphere. “Yet they also funded these groups that said we do not know enough, there is a lot of controversy, so let us not do anything,” he said.

Product Defense Firms: A Growing Industry

Michaels called this new disinformation industry the “Enronization of science”—phony companies producing paperwork and documents that claim something without any evidence to support it. “There are these scientists, many of whom work for very lucrative scientific consulting firms, who have been hired to defend products or activities in the regulatory or legal arenas, and the value of that work is to influence regulation and litigation, not to produce valid science,” said Michaels. “In fact, their science is really of questionable value.” These firms tell prospective clients they take advantage of the concept of “innocent until proven guilty” that is ingrained in U.S. society. Applying that concept makes people think, without saying so, that the product or activity is presumptively innocent, which is a much higher bar to getting something off the market to protect the public’s health, he added.

As an example, Michaels showed a document in which Hill and Knowlton listed case histories detailing its work on selected environmental and occupation health issues. It provided this document to the beryllium industry when it was about to be regulated over danger to human health. One of those case studies described how DuPont hired the firm to calm fears around the allegations that fluorocarbons were depleting the ozone layer, which would increase the risk of skin cancer. That public relations and disinformation campaign delayed regulations for several years,

which enabled DuPont to sell a new fluorocarbon product and maintain market share. Michaels noted that the scientists who produced the evidence linking fluorocarbons to ozone depletion won the Nobel Prize in Chemistry in 1995 for their work.

In another example, Michaels described how an extensive effort by the Weinberg Group on behalf of two pharmaceutical companies led to a 10-year delay before FDA withdrew its approval of a particular drug as a result of post-approval studies showing the drug caused more harm than benefit. Similarly, diesel engine manufacturers attempted to delay the classification of diesel exhaust as a human carcinogen by impeding epidemiologists at the National Institute of Occupational Safety and Health (NIOSH) and National Cancer Institute (NCI). NIOSH and NCI’s studies of lung cancer among miners who worked deep underground mining materials that do not cause lung cancer but alongside giant diesel engines, conclusively demonstrated the link between diesel exhaust exposure and lung cancer (Attfield et al., 2012; Silverman et al., 2012).

When the World Health Organization (WHO) moved to classify diesel engine exhaust as carcinogenic to humans, Michaels explained that the industry hired product defense firms to conduct a disinformation campaign designed to confuse the public and regulators. It relied on legislation designed by the tobacco industry that required any studies done or paid for by the federal government to release their raw data to anyone who wanted to reanalyze the data. Epidemiologists, said Michaels, can take data from a positive study, change the parameters, and turn a positive result into a negative one, which is what the industry consultants did (Chang et al., 2018; Crump et al., 2015, 2016).

In another example Michaels cited, DuPont hired a product defense firm to muddy the waters regarding perfluorinated compounds PFOS and PFOA. In 2002, following the first relevant lawsuits, West Virginia, based on a recommendation from that product defense firm, set safe levels in drinking water at 150 parts per billion, or 150 times DuPont’s internal safe level. In 2007, DuPont hired ChemRisk, whose scientists had worked for the tobacco industry, to estimate the risk among populations that drink PFAS-contaminated water, and ChemRisk concluded that exposures were about 10,000-fold less than levels considered to be a health risk by an independent panel of scientists who had recently studied PFOA (Paustenbach et al., 2006).

From 2005 to 2013, independent scientists conducted numerous studies on workers and residents exposed to PFOA and found probable links with ulcerative colitis, thyroid disease, testicular and kidney cancer, pregnancy-induced hypertension, and hypercholesterolemia. However, in 2014, 3M, facing a lawsuit brought by the State of Minnesota because it had contaminated the water around its plants, hired product defense firm

Exponent to conduct a strategic literature review of the published data. It concluded “the epidemiologic evidence does not support the hypothesis of a causal association between PFOA or PFOS exposure and cancer in humans” (Paustenbach et al., 2006). Nevertheless, 3M settled the lawsuit for $850 million.

Michaels noted that the U.S. National Toxicology Program (NTP) reviewed the evidence in 2016 and concluded that PFOA and PFOS are presumed to be immune hazards to humans. Gradient, another product defense firm, was hired by 3M and said that the hazard ratings for both should be downgraded (Beck, 2017). However, based on a growing number of quality studies, EPA issued a health advisory for both at levels of 70 parts per trillion, and in 2022, EPA revised that advisory, lowering the safe levels of PFOA and PFOS to 0.004 and 0.02 parts per trillion, respectively. “While industry was saying there is not enough compelling evidence that they cause illness, EPA is saying there is no evidence to say these levels are too low,” said Michaels.

Who Pays the Price?

Michaels said the people sickened by exposures that should have been prevented pay the price for such obfuscation. Occasionally, shareholders pay a price when their corporations are caught manipulating scientific evidence. Johnson & Johnson, for example, recently stopped selling its iconic talcum powder globally after losing several lawsuits based on studies showing an association between ovarian cancer and using talcum powder contaminated with asbestos.

Michaels disclosed that he was an unpaid witness in one lawsuit brought by 22 women in Missouri with ovarian cancer; he had access to documents revealing the disinformation campaign. “The jurors were given documents showing how [Johnson & Johnson] and trade associations tried to convince the U.S. government not to label products containing talc as potentially carcinogenic,” said Michaels. These documents outlined how product defense firms would create a reasonable doubt in the minds of NTP’s Board of Scientific Counselors, which was considering categorizing “asbestiform talc” as a human carcinogen and non-asbestiform talc as reasonably anticipated to be a human carcinogen, and cause more confusion over the link between talcum powder use and ovarian cancer. The jurors, after seeing these documents, awarded the women $25 million each plus $4.14 billion in punitive damages, which was later reduced to $2 billion. According to Michaels, one juror told the press, “We were just trying to find something [Johnson & Johnson] would feel.”

Johnson & Johnson is facing approximately 38,000 ovarian cancer lawsuits and attempting what Michaels called a “Texas two-step.” “They

want to avoid liability by creating a subsidy that will have all that liability and spinning it off into bankruptcy, which means most of those women will never see anything or very little in a lawsuit,”³ he said. Michaels pointed out that if Johnson & Johnson had taken its talcum powder off the market in 2000 when it learned about the asbestos and replaced the talc with corn starch, which it has since done, it would have avoided these lawsuits.

The Threat to Public Health

In a book Michaels wrote that discussed these examples, he stated that corporate disinformation threatens human health (Michaels, 2020). Disinformation campaigns have negatively affected the air and water, driven the opioid and obesity epidemics, increased the number of children poisoned by lead in paint and the prevalence of alcohol-related diseases, and even delayed the National Football League from dealing with chronic traumatic encephalopathy resulting from players’ blows to the head, said Michaels. In addition, the work of these mercenary scientists hurts the credibility of all scientists.

To counter these disinformation campaigns, Michaels said it will be necessary to learn to distinguish between real and manufactured uncertainty. “We have got to build the scientific evidence base from research produced by independent, unconflicted scientists,” he said, citing the Health Effects Institute (HEI),⁴ a public-private partnership the supports research on the health effects of air pollution, where the funding is balanced between EPA and the automobile industry, as a positive example of how utilizing industry funding can balance biases. He also called for polluters and producers of hazardous chemicals to pay for but not be able to control the research. He recommended moving away from regulating toxic chemicals one by one and instead regulating them as a class. For example, data on all 9,000 or so perfluorinated compounds do not exist, but enough is known about some to make the reasonable assumption that they all could be somewhat hazardous and should be regulated as a class. “The presumption of innocence has to end,” said Michaels in closing. “Chemicals are not innocent until proven guilty. We need new solutions, and we have to be bold.”

___________________

³ As of January 30, 2023, a federal court ruled that Johnson & Johnson cannot use bankruptcy to resolve litigation over claims its talc products cause cancer https://www.cnn.com/2023/01/30/business/johnson-and-johnson-talc-bankruptcy/index.html (accessed February 21, 2023).

⁴ https://www.healtheffects.org/about (accessed April 11, 2023).

THE INTENDED AND UNINTENDED CONSEQUENCES OF HEALTH TECHNOLOGY AND OBSERVATIONAL REAL-WORLD EVIDENCE⁵

Hernandez provided a perspective on the multiple entities that can influence research results. First, he asked the workshop participants to consider the broader question of who influences research results. Influencers include funders, investigators, and technologies. In fact, he said, all of these can influence research results, making it important to be aware of explicit and implicit bias and the behavioral economics that occurs at both the individual and organizational levels. As for which funders might influence results the most, Hernandez named the life sciences industries, government agencies, contract research organizations, data aggregators, research technology companies, and social networks. When he was vice dean for overseeing clinical research and research integrity, he saw examples of each of these at play.

For instance, Hernandez discussed a case involving a first-of-its-kind clinical trial completed in 2011. This was the largest clinical trial ever, and the investigators presented the results at a major meeting and published their findings in the New England Journal of Medicine. The results, he said, were “pretty neutral,” with the drug producing mild benefits. As a result, sponsor interest in funding further analyses of the study was low or nonexistent. Still, a colleague advised Hernandez—the young faculty member who was the coordinating center’s PI for the trial—to hold on to all the data because he could use them to conduct many analyses and answer many questions, which could make his career and pave the way for funding via multiple mechanisms.

Because the initial sponsor’s lack of interest in conducting new studies or analyzing completed studies, the risk for sponsor influence might seem unlikely. However, said Hernandez, future sponsor influence is a risk, is much more difficult to track, predict, or anticipate, and can be large. For example, another company might be interested in that study’s clinical area and want to access the unique dataset. Hernandez might have some implicit bias to develop research questions that might align with those of the funder, which could be another biopharmaceutical company or a government agency. He noted that he remembers worrying about doing the right thing, given that many parties were interested in the study’s data.

Next, Hernandez pointed out that many published studies cannot be replicated for various reasons, either unintentionally or sometimes intentionally because of implicit bias. In 2011, for example, researchers at

___________________

⁵ This section is based on the presentation of Adrian Hernandez, Duke University.

Bayer found that its scientists replicated originals results in only 21 percent of the studies they examined (Prinz et al., 2011); in 2012, researchers at Amgen found that only 11 percent of preclinical cancer studies had replications that could confirm their conclusions (Begley and Ellis, 2012). Possible causes of the reproducibility crisis include fierce competition arising from lower funding levels, higher future stakes for creating intellectual property that could be of value to others, structural problems and hierarchies that prevent openness of research results, and the complexity of science and data providence (Harris, 2017). “Influence can come in many flavors,” said Hernandez.

As a second example, he discussed a hypothetical case in which a junior investigator develops a concept to improve functional capacity in cardiopulmonary disease with the potential to turn it into a novel intervention. With intellectual property pending, the investigator develops an experimental plan that includes three series of early-phase studies funded by their academic center via a start-up agreement, with plans for future funding through National Institutes of Health (NIH) K or equivalent or R01 grant mechanisms. Hernandez identified some potential influence to generate positive results and innovative experimental designs, which might come from the institution funding the research, given the potential for commercial interest. He added that a behavioral economics concept called “prospect theory,” which highlights issues around loss aversion of future winning, may influence people’s decisions (Barberis, 2013).

In another hypothetical, researchers design a novel trial platform that will collect data remotely from participants and harvest electronic health records automatically. It could solve many of the world’s problems, by allowing patients anywhere to enroll and enabling researchers to acquire their data seamlessly. Funding for a pilot to evaluate feasibility of going to a decentralized model comes from a nonprofit organization, with a research technology company providing in-kind or highly discounted support. The company is aiming for the trial to generate positive results and has plans for scaling the technology with a series of funding rounds and perhaps a public stock offering. The questions, said Hernandez, are whether the company might influence the results, what the quality of the data will be, and if the data will be available in an open-access venue.

These examples have led Hernandez and his colleagues to formulate a model of the research ecosystem with multiple influencers and stakeholders (see Figure 3-1). “We tried to provide a comprehensive view of this, noting that there are many potential gains from different points of view and there are needs to try to address this in a much more public or open way so that we have trust and transparency,” said Hernandez. “If you are part of any of these groups, you probably have been in a situation where there may be either direct or indirect influences or even potentially politi-

cal concerns that come up, and there have been times where unfortunately the political environment can influence what people do.”

Hernandez identified different organizations that continue to address these different issues. The American Heart Association and American College of Cardiology, for example, have been trying to ensure that the cardiology community is proactive around issues of professionalism and ethics by identifying four essential components of a COI compliance program: disclosure of interests, assessment of interests, management of interests, and oversight and enforcement of a conflict management plan (Benjamin et al., 2021). He noted that all four are essential, mere disclosure or reporting is not enough, and the mere presence of an interest is not enough to create a COI. Most COIs are manageable if the conflict compliance program is implemented and maintained effectively, and the entire process must be conducted and overseen with complete confidence.

To address potential implicit bias, Hernandez said it is important to ensure awareness of associational or intellectual interests, that future gain may be important, and that the intellectual interest or agenda can drive different decisions on study design and how the investigators report their results (Benjamin et al., 2021). Open science can address this concern, he noted, and the biomedical research enterprise has been making progress around open science, but unpublished results remain that can be unlocked both for gaining insights and to establish trust in the data and research results.

In summary, Hernandez said influence on research programs and results comes from many directions. Most of the focus has been on funders, given the risk of explicit bias, but implicit bias may exist. Checks and balances are needed, which regulatory agencies often provide to benefit the public. “But we as a community have to consider how we promote open science, whether that is in academia, health care systems, or with other players, to help address this risk of bias, both explicit and implicit,” said Hernandez.

When asked about any clues or methodology he would recommend exposing where biases may influence research, Hernandez endorsed increasing transparency around data and allowing access to them. “Just the potential for someone to lay eyes on it is a way to make sure that at least some of the implicit biases may go away,” he said. The other aspect is to make sure people understand where other players have influence, such as through in-kind support, and ensure it is disclosed.

INDUSTRY FUNDING BIAS IN NUTRITION SCIENCE ON ULTRAPROCESSED FOODS⁶

When Schmidt began to explore the nutrition science field, she was surprised at how common it is to accept industry funding. Her research focused on ultraprocessed foods, which according to the NOVA classification (Monteiro et al., 2018) are “industrially processed foods that combine refined sugars, fats and salt, and chemical additives.” This classification system, she explained, has become an engine for policy change and the international movement to regulate these foods using front-of-packaging warning labels. Countries throughout Latin America are doing that. Efforts also exist to tax such foods, particularly sodas and sugar-sweetened beverages.

Industry sponsors of ultraprocessed food research are large transnational corporations (see Figure 3-2), said Schmidt. Her argument for considering the connection with COIs in research is that these are the alcohol or tobacco of the food system. “Yes, there is conflicting science around whether broccoli is good for you or if nuts are good for you, but ultraprocessed foods have this footprint of public health harm and that raises the stakes in getting the story right scientifically about how they are affecting health,” said Schmidt.

Ultraprocessed foods are abundant. In the United States, for example, 57.9 percent of calories consumed come from them (Martínez Steele et al., 2016), and observational studies link them to obesity, Type 2 diabe-

___________________

⁶ This section is based on the presentation of Laura Schmidt, University of California, San Francisco.

tes, hypertension, heart disease, and some cancers. Clinical trials have shown that a diet of them increases energy intake by some 500 calories a day (Hall et al., 2019) and can be habit forming, with almost 70 percent of them being “hyperpalatable,” or industrially engineered to trigger the dopaminergic reward system (Fazzino et al., 2021).

Most meta-studies on funding bias in food research focus on ultraprocessed foods. Some of these analyses have identified a funding bias: those with industry sponsors were 32.7 times more likely in one analysis (Schillinger et al., 2016) and 57.3 times more likely in another (Litman et al., 2018) to find no increased risk of diabetes from sugar-sweetened beverage consumption. “The industry funding bias in this particular area of nutrition science is quite stunning,” said Schmidt.

One well-documented case of sponsor influence examined research conducted by the Coca-Cola Company–funded nutritional research between 2008 and 2016. These studies reported that physical inactivity, not food or diet, is causing the obesity pandemic. Despite good evidence showing that physical activity matters for maintaining weight loss, less evidence exists regarding weight gain. A systematic review found 389 Coca-Cola–sponsored studies in 169 journals, most of which concluded that the obesity crises resulted from physical inactivity (Serôdio et al., 2018). When the New York Times published an article with the headline, “Coca-Cola Funds Scientists Who Shift Blame for Obesity Away From Bad Diets” (O’Connor, 2015). To address the critique, Coca-Cola established a “transparency initiative” that listed all the studies it was funding on its website. However, a secondary analysis found that Coca-Cola only reported about 5 percent of the studies it funded (Serôdio et al., 2018).

One reason nutrition science and ultraprocessed food research may be a special case in terms of COI is that historically, most nutrition research is industry funded. One study estimated that the federal government in 2009 funded approximately $1.5 billion compared to $60 billion from the food industry (Mozaffarian and Forouhi, 2018). The synergistic relationship between academics in nutrition and agriculture research and the food and beverage industry traces back to U.S. land grant institutions, said Schmidt. Even to this day, academics and industry funders have a tight relationship, particularly in agricultural research. Another reason is that research priorities vary between NIH and the U.S. Department of Agriculture (USDA) in this country and between the Food and Agriculture Organization and WHO internationally. Much of the U.S. nutrition science research funding comes from USDA, where agricultural and food industry interests are high priorities, said Schmidt. “There is no National Institute on Nutrition, so public funding in this space is much tighter than in other areas of medical research,” she added. In addition, many industries have a stake in nutrition research, including agriculture,

chemical, agrochemical, fossil fuel, pharmaceutical, and even tobacco. She noted that tobacco companies are involved through their food subsidiaries (Nguyen et al., 2019, 2020).

The scientific paradigm for nutrition research informs a root issue in funding, said Schmidt. The dominant paradigm, known as “nutritionism,” focuses research on the health benefits or harms of a single food or nutrient (Scrinis, 2013). This approach is a holdover from a field focused historically on vitamin deficiencies and global undernutrition. The problem, said Schmidt, is that whole diets matter more for health today than single nutrients. It also gives rise to “food fads” that demonize sugar, fat, or salt; this shifting advice confuses the public, which undermines the credibility of nutrition science. However, nutritionism and the single-nutrient idea are critical tools for the food industry (Nestle, 2002). “There is a real concern around these single-nutrient studies that are used as an industry marketing strategy to either tout the health benefits of their products or make the case that they are not harmful,” said Schmidt.

She said that an issue in ultraprocessed food research is that the industry is positioned to influence the research agenda and narrative for nutrition science. One reason for this is that industry sponsors most U.S.-based professional societies and journals in the field (see Table 3-1). For example, the International Life Sciences Institute, which Schmidt said is a well-researched front group for food and beverage industry interests,

TABLE 3-1 Industry Sponsors of Nutrition Science Organizations and Scientific Journals

SOURCE: Derived from Schmidt presentation slide 17.

funds the Oxford Academic Nutrition Reviews. “Here is the case where the entire journal is being driven by an industry front group,” said Schmidt.

Schmidt concluded with recommendations for addressing some of the issues she raised:

• Increasing government and philanthropic funding to better balance the funding arena so that investigators can rely less on industry funding.
• Pushing back on nutritionism by funding research on whole diets and foods based on the level of processing.
• Applying more scrutiny to COIs in ultraprocessed food research given the health harms associated with them.
• Earmarking revenue earned from soda and fat taxes to fund independent research on ultraprocessed foods.
• Creating a central public repository of information on scientists’ COI statements, perhaps following the model of clinicatrials.gov, so that journalists and members of the public can better understand who is giving them nutrition information.

WHAT YOU DO NOT SEE IS WHAT COUNTS: A PEEK BEHIND THE SMOKESCREEN⁷

McKee described work conducted when he was the unpaid editor in chief of the European Journal of Public Health (1998–2003). He noted that much of the evidence policy makers use comes from systematic reviews and meta-analyses and that well-established checklists exist for testing the validity of the studies that go into these. The difficulty with those checklists, he said, is that they only prompt the reviewer to assess what is in a paper and not go beyond that, making it difficult to see the whole picture.

In an ideal world, said McKee, a systematic review will capture all the studies that researchers have published, which will balance out the uncertainty relative to effect size of smaller studies with the results from larger studies. An asymmetry between smaller and larger studies is a clue that studies are missing. The problem arises when this distribution is less random than it may seem because researchers are less likely to publish smaller studies. A good example comes from the history of the tobacco industry suppressing results showing that exposure to secondhand smoke is harmful. He pointed out that despite growing evidence of a demand for smoke-free environments as early as 1979, the public had little confidence that the tobacco industry was interested in its welfare. Industry’s official

___________________

⁷ This section is based on the presentation of Martin McKee, London School of Hygiene and Tropical Medicine.

position at the time was that epidemiological research cannot prove a causal link between exposure to tobacco smoke and disease, particularly for passive smoking, where industry argued the apparent increase in health risks was too small to provide any confidence in the findings.

However, said McKee, industry was secretly funding epidemiologists, who often did not know it, and got them to agree to a code of good practice that advises discounting a relative risk of less than 2. Conveniently, said McKee, the relative risk typically found in studies of passive smoking available at the time was about 1.3. Industry also promoted the idea that even if studies demonstrate an effect, it was most likely attributable to confounding factors, as those people who live with smokers differ from those who do not in many ways, such as diet. Industry also claimed no biological evidence showed that secondhand smoke causes disease: “We within the industry are ignorant of any relationship between smoking and disease. Within our laboratories no work is being conducted on biological systems” (Ciresi et al., 1999).

McKee got involved because, as a journal editor, he had overseen publication of a confounder study on the characteristics of women married to smokers and nonsmokers. Nothing was wrong with the unexciting paper, but after publishing it, McKee was told that Ragnar Rylander, Swedish environmental health scholar, had undisclosed links to the tobacco industry. “He denied it, and a very lengthy correspondence followed. At the same time, he sued two anti-tobacco advocates in Geneva for libel,” said McKee.

When informed of this possible link, McKee did some digging through the trove of tobacco litigation documents at UCSF and found evidence of transferring significant sums of money. McKee found that this researcher acted as a link between a testing plant in Germany and the Philip Morris headquarters in Richmond, Virginia, (Diethelm et al., 2005) and also organized symposiums to convey the message to researchers and to the public that “…the available data on the harmful effects of smoke on nonsmokers was insufficient and inconclusive, notably in view of other factors susceptible of influencing their health.” That quote, said McKee, came from a court hearing for the libel case he mentioned, for which he and others were witnesses.

Dr. Rylander also worked with a Kansas law firm known to be at the center of the campaign to distort the evidence on passive smoking. One memo from it stated, “Dr. Rylander prepared a brief memorandum for internal use only concerning a workshop. His major point was that he did not feel that the workshop could or would be in a position to give environmental tobacco smoke a ‘clean bill of health.’ However, Dr. Rylander did believe that he could bring a healthy skepticism to the conference and some of the claims being made about environmental tobacco smoke.” The Swiss court ruled against Rylander, with the court judgment noting that

on at least one occasion, he had altered his results after conferring with Philip Morris.

McKee referenced a 1968 document in which a Philip Morris vice president expressed concern that industry was depending on monitoring the literature to alert it to research from studies “oriented to seeking out and highlighting the negatives associated with tobacco smoker.” Dr. Rylander argued that the industry needed to obtain its own facts and data to avoid being surprised by information from outside sources and “to be able to interpret and understand the results of such studies.”

Philip Morris was not the first tobacco company to conduct its own biological research. American Tobacco had done so and “relocated [it] under conditions of extreme secrecy… to new research facilities.” Its chief executive officer had reservations about the wisdom of these studies, but he agreed that the research should take place in Europe, which “presents an opportunity that is relatively lacking in risk and unattractive repercussions in this country.”

In 1970, Philip Morris purchased a German testing institute to research other causes of smoking-related diseases to “get the industry off the hook.” However, rather than buy the institute itself, it funneled the purchase through the Fabrique de Tabac Réunies, based in Neuchâtel, Switzerland, to conceal its involvement. Despite no formal connection with the testing institute, Philip Morris would authorize any study proposals, and the connection was so well hidden that hardly anyone at Philip Morris knew about it. According to 1996 testimony from a former Philip Morris employee, “All in all, it seemed as if there was an inner company within Philip Morris that conducted at least some of its investigations behind the scenes on a strict need-to-know basis. Interestingly, many if not all these activities appeared to be related in one way or another to these sensitive topics of smoking and health.”

When McKee and his colleagues gained access to the tobacco industry documents, they found over 800 studies on sidestream smoke—the smoke from the lighted end of a burning tobacco product—conducted between 1981 and 1989. The testing institute had 53 publications, only 16 percent of which mentioned tobacco and related terms even though over 95 percent of the work was for Philip Morris or Fabrique de Tabac Réunies. However, between 1990 and 1998, 63 percent of the publications concerned tobacco; in 1990, Philip Morris was advised that the testing institute’s work could no longer be assumed to be safe from disclosure.

The papers the testing institute published, said McKee, focused on research on other possible causes of lung cancer, such as green tea, to cast doubt on the value of cotinine—a chemical formed when the body metabolizes nicotine—as a marker of exposure to environmental tobacco smoke, and research purporting to show that cigarette additives are harmless. A

few papers came out suggesting that tobacco might be harmful, though McKee felt this might have been an attempt to restore credibility. Over 100 animal studies showing that sidestream smoke was often more toxic that what a smoker would inhale were not published yet served to manipulate experimental conditions to produce more desirable results and commission independent researchers at arm’s length for studies whose results were preordained.

McKee’s point was that it is not just the research one can see that is important. “The importance of this is that often we are told that we should set aside all these other considerations and we should simply look at the quality of the science and the paper in front of us. We should not be concerned about who funded it. We should assess it on the basis of the methodology and the results that are presented. I would argue that this is very naïve because it is what we cannot see that is important,” said McKee.

CAN DATA AND AI MAKE IT EASIER TO MANAGE THE IMPACTS OF COIS?⁸

Dunn focused on a potential solution for managing the influence of research sponsorship and financial COIs in more sophisticated ways that go beyond retrospective analyses that discover problems after the harm has been done. He noted the distinction in meta-research between funding for research and financial COIs that represent money or other financial gain for being an expert. Dunn also reiterated statements by previous speakers that influence happens when studies are funded, designed, and reported and that sponsor influence can also happen in systematic reviews, guidelines, what the media products, and what the public consumes.

One of the earliest studies his team conducted in this space examined systematic reviews regarding two drugs used to treat influenza, Tamiflu and Relenza. When the reviews had financial COIs or received funding from the companies that produced Tamiflu or Relenza, they were more likely to reach favorable conclusions regarding using the drugs in broader populations. “Unsurprisingly but also importantly, systematic review authors are also able to manipulate and change the design of their reviews to reach conclusions that are favorable based on their agendas,” said Dunn. His takeaway is that the research agenda can shape every part of the design, reporting, synthesis, and dissemination of health research.

In a study with Quinn Gundy, who spoke on the final day of the workshop, Dunn went through the disclosure of COI and funding statements for a random sample of articles published in journals that had signed onto

___________________

⁸ This section is based on the presentation of Adam Dunn, University of Sydney.

the International Committee of Medical Journal Editors’ expectation on disclosures. They discovered that 23 and 64 percent, respectively, included positive and negative conflict of interest disclosures, and 14 percent did not have a disclosure statement⁹ (Grundy et al., 2018). and the percentage with disclosures was higher (over 31 percent) for drug studies and commentaries of any type. Another finding was that articles with COIs were more likely to be published in high-impact journals and receive media attention, suggesting that people are disproportionately exposed to research at higher risk of presenting biased results, conclusions, and opinions.

Influence is hard to catch, said Dunn. He explained that when applying the standard tools to measure risk of what is published, the things measured are very similar, but the difference in results and conclusions remain. “Put simply,” said Dunn, “there are hidden factors that lead to favorable conclusions in studies that are sponsored by industry. This means that it is not easy to identify or quantify the influence on design and reporting of studies using standard tools.” Meta-research is one approach to identify and measure influence, but these studies require substantial detective work by experts going through many sources of information, many of which require time or effort to access. As a result, no obvious way exists to investigate the trustworthiness of every single study, review, or media communication associated with health research. “It ends up being just too much work,” said Dunn.

Dunn explained that disclosure is not enough; even with perfect disclosure practices, the reader is left with having to decide whether to ignore it, minimize it, be wary of the results, or trust the research more because they can assume that the authors must be experts to have access to industry funding. He argued for better and more accessible records of sponsorship and financial COIs among the people who produce and report on research. This would enable automatically labeling research protocols, registrations, reports, reviews, and media communications with better indicators of how much weight to give to those results and conclusions or to indicate that the study is likely to be so compromised it should not be considered in syntheses. Dunn said that calls for author-centric public records of COIs go back to at least 2007 (Rubenfeld, 2007), and he made the same argument in 2016 (Dunn, 2016).

Modern AI, and natural language processing in particular, may make meta-analyses easier but also cause the biases discussed at the work-

___________________

⁹ “Disclosures were classified as positive when at least 1 author reported a conflict of interest of any type, excluding current study funding or industry employment; negative if all authors stated they had no conflicts; and missing if there was no disclosure statement.” (Grundy et al., 2018).

shop to manifest in ways that traditional meta-research methods may not detect. For example, depending on how a question is phrased, an AI chatbot might correctly summarize study results or put a marketing spin on it. The problem is that the evidence the AI draws on cannot be determined. “We can be reasonably sure that it is not assessing the evidence for bias or for reliability, it is not looking at financial COI, and it may be assuming that if the majority of studies are favorable, then that must be the truth,” said Dunn.

On the other hand, he added, it may be possible to develop natural language processing methods and tools to automatically extract and compare information to support meta-research studies investigating factors that might indicate bias. These include comparing design factors in protocols and registrations, such as changes in primary outcomes or the choice of comparators or identifying missing links between registrations and the articles reporting their results (Bashir et al., 2019; Liu et al., 2022; Surian et al., 2021). AI tools might be able to automatically extract structured summary data from places such as clinicaltrials.gov to compare with what is reported to detect outcome reporting bias. Ultimately, said Dunn, the goal would be to bypass the reporting in articles altogether and have AI synthesize the results directly from structured results data and flag a paper as risky, adjust the weight given to it in syntheses, or discount it completely. AI may also be able to determine whether summaries prepared for a general audience are fair or biased representations of what the paper reports and concludes (Harrison et al., 2020; Shah et al., 2019).

Dunn called for serious consideration for implementing and properly funding three things that can have a transformative effect on how evidence is used and represented in policy, practice, and the public domain. The first is to establish an author-centric open registry for funding and COI data that is not restricted to physicians in single countries or held within institutions. It should connect the Open Researcher and Contributor IDentifier (ORCID) and CrossRef digital identifiers. Second, the methods used to analyze bias in meta-research should be standardized so meta-research studies can be further aggregated. “This is not to say that meta-researchers are bad at sharing data, because they are quite good at it, but rather that we can do more,” said Dunn. Third, taking both of those actions would enable researchers to do a better job building machine learning methods for estimating the likelihood that a registration, report, review article, or media communication presents a distorted view. This would allow for flagging a paper for further investigation or finding ways to reduce its influence on policy and practice, said Dunn.

SPONSOR INFLUENCE IN DIABETES RESEARCH: AN INDUSTRY CASE STUDY¹⁰

Schillinger explained that the diabetes epidemic affected over 400 million people globally in 2015 and is projected to affect approximately 8 percent of the entire global population by 2040. “The rate of the rise of the diabetes epidemic has been inexorable,” said Schillinger. However, he added, a public health turnaround may be happening in the United States, and it is likely connected to a decline in longer-term trends in consumption of sugar-sweetened beverages, which peaked around 1998, thanks to changes in social norms related to research findings around the negative health impacts. He noted that diabetes prevalence has been following the longer-term trends in that consumption, with a delay of approximately 15 years. “This may represent the beginnings of a very important public health turnaround that would be very critical to harness and leverage to a greater degree in the United States and, of course, in the global context as well,” said Schillinger.

The first part of the case study he presented focused on a court case in which ABA sued the City and County of San Francisco after they passed a 2014 ordinance that would require billboards advertising sugar-sweetened beverages on public grounds—billboards are rented from the city and county—to post this message: “WARNING: Drinking these beverages can contribute to obesity, diabetes and tooth decay.” The ABA was suing on constitutional grounds, claiming that the ordinance infringed on its “commercial free speech” by compelling manufacturers to include warnings that were scientifically controversial, misleading, and untrue. Schillinger was a scientific expert witness; he then wrote a short paper describing how both science and public health had been put on trial (Schillinger and Jacobson, 2016) and ABA attempted to use the tools of science to cast doubt on that causal relationship.

Schillinger reminded participants that science attempts to combine unbiased experimentation with objective observations of the natural world to accumulate knowledge that can help approximate truth. “We can never really determine truth through science; we can only get closer and closer to what we believe is the truth,” he said. “In that regard, this case revolved entirely around science and the nature of truth.”

In the hearing for the case and the expert reports submitted by industry, the focus was on the scientific veracity of the warning. The city, backed by his report, responded that it is factually true and strong science supported these causal relationships. Industry argued that it was uncon-

___________________

¹⁰ This section is based on the presentation of Dean Schillinger, University of California, San Francisco.

stitutional for their commercial free speech to be infringed or chilled by having to include compelled noncommercial speech as a warning, particularly when this speech is “misleading, false or a subject of scientific controversy.” Industry cited numerous scientific studies to support its claims of controversy and that the relationship with disease is false.

The district court judge for the case issued an opinion stating that “compelled disclosure must convey a fact rather than an opinion… generally speaking, it must be accurate” and noted that the factual requirement should not “be so easily manipulated that it would effectively bar any compelled disclosure by the government, particularly where public health and safety are at issue” and “controversy cannot automatically be deemed created any time there is a disagreement about the science behind a warning because science is almost always debatable at some level.” The judge decided that the warning likely passed the factual and accurate requirement. San Francisco won the case, but that decision was overturned on appeal.

The second part of the case study focused on financial and nonfinancial COIs and provides a cautionary tale about the importance of recognizing the difference between the two. Schillinger explained that when conducting research to prepare his expert opinion, he explored the degree to which industry was behind the controversy in the literature regarding whether sugar-sweetened beverages are causal factors in the obesity and diabetes epidemics. He and his colleagues systematically reviewed randomized controlled trials with outcomes related to markers of diabetes and obesity and systematic reviews and meta-analyses. They identified 60 studies over 15 years, 28 experimental and 32 systematic reviews or meta-analyses. “We asked the question to what extent are funding of the studies or financial support for the authors of studies associated with the outcomes of these studies,” said Schillinger.

They discovered that the beverage industry appeared to be heavily influencing scientific findings: 26 articles found no associations between the product and the disease outcomes, and 34 described positive associations (Schillinger et al., 2016). Of the 26 negative studies, 25 had funding ties to the industry; conversely, only one of 34 positive studies had industry ties. The relative risk with respect to finding that the industry had funded a study showing no association, when compared to independently funded studies, was 32.7, similar to what Bero found when she measured the effects of tobacco industry funding on study outcomes. “We concluded that this industry appears to be manipulating the contemporary scientific process to create controversy and advance their business interests at the expense of the public’s health,” said Schillinger.

This study garnered significant press coverage, and the industry responded with a letter to the editor written by the chief science and

regulatory officer of the ABA that the Annals of Internal Medicine published several months later (Jack, 2017): “Dismissing industry-sponsored research on the basis of funding is no more valid than discarding studies funded by private foundations or groups that advocate for particular policy views. Transparent disclosure of financial COIs and of potential biases, as well as objective assessment of the research according to accepted scientific principles, is the proper approach to adequately vet the strengths of a study.” The letter added that, “The authors [Schillinger et al., 2016] should ask themselves whether they are totally committed to their point of view and unwilling to consider other perspectives. Intellectually motivated biases are as important as financial conflicts of interest.”

Schillinger argued that treating intellectual and financial COIs as equal is dangerous and seems calculated to undermine the work of independent clinician investigators whose primary obligation is the health of their patients and communities. Accusing investigators concerned about industry influence of intellectual COIs goes back to the 1970s and 1980s and was a strategy of the tobacco industry, he explained (Brandt, 2012).

The third piece of this case study, said Schillinger, deals with an attempt by the sweetened beverage industry to influence policy and dietary guidelines. In early 2017, a widely read paper in the Annals of Internal Medicine called into question the quality of the evidence used to develop national and global guidelines on dietary sugar intake and cautioned public health officials to be aware of these limitations when considering whether to promulgate such recommendations (Erickson et al., 2017). The primary funding source for this study was the Technical Committee on Dietary Carbohydrates of the North American branch of the International Life Sciences Institute (ILSI), an organization funded by a large number of fast food and junk food industries. As one example, ILSI has been behind studies to make the case that physical inactivity, not food or diet, is the cause of the obesity pandemic.

The Annals of Internal Medicine editors asked Schillinger to write an editorial to accompany that ILSI-funded study (Schillinger and Kearns, 2017). Schillinger and his coauthor mentioned that the study had been funded by a trade group that represented the several major food and beverage companies. In essence, they said, that study suggests that placing limits on junk food is based on junk science, a conclusion favorable to the food and beverage industry. They pointed out, however, that the disclosure of the study’s funder was not enough to critically appraise it, so they examined the methods used in the review of added sugar guidelines and concluded that “concerns about the funding source and methods of the review preclude us from accepting its conclusions that recommendations to limit added sugar consumption to less than 10 percent are not trustworthy. Policymakers, when confronted with claims that sugar

guidelines are based on ‘junk science,’ should consider whether junk food was the source.”

According to Schillinger, the industry-funded review suffered from four fatal methodological flaws. The first was that authors used the inconsistency of international and national recommendations across time and across guidelines as a rationale to raise concern about the quality of these guidelines. “However, these guidelines were issued between 1995 and 2016, and one would expect recommendations spanning more than two decades to evolve,” he explained. In fact, he noted, recent guidelines from Public Health England, WHO, and USDA showed remarkable consistency; only the 2002 Institute of Medicine guidelines, partially funded by ILSI were the outlier.

The second flaw was that the ILSI-funded review stated that the funding sources for the Dietary Guidelines for Americans were unclear, so it questioned their editorial independence and gave it a poor score. This assessment was curious because the review’s appendix acknowledged that the guidelines were developed with federal funding and the advisory committee members were vetted thoroughly for COIs per federal advisory committee rules.

The third problem was that the ILSI-funded report used the Appraisal of Guidelines for Research and Evaluation instrument to assess the quality of the guidelines (Brouwers et al., 2010). Schillinger explained that this was inappropriate and essentially guaranteed that the national and international public health guidelines would be given poor ratings. He said that the tool “is designed to assess clinical practice guidelines in the treatment of diseases at an individual patient level [and not] the quality and appropriateness of dietary guidelines to assess risks of consumption at the population level so as to inform public health policy.” Using this tool, the authors downgraded the trustworthiness of the guidelines because ways to limit sugar intake “were not clearly presented” and “likely barriers to and facilitators of implementation” were not discussed. They also created an overall guideline quality score of 1 to 7, with interrater differences of three points permitted, yet did not report its reliability, said Schillinger.

Schillinger’s final critique of the review’s methodology was that the authors used the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system to evaluate the quality of evidence (Terracciano et al., 2010). The authors, said Schillinger, falsely claimed that the food pattern modeling and national caloric data used to inform the U.S. Dietary Guidelines were not publicly available, which prohibited them from applying GRADE, yielding another poor score. Still, they claimed that “using the GRADE approach, we found that the overall quality of evidence to support recommendations was low to very low.” This was puzzling, Schillinger explained, because the authors ignored that the

methods used to assess dietary patterns were extensively described in an appendix, along with a 500-page supporting report.

This case study took a strange turn, Schillinger said, when the Atlantic published a cover story on the sugar controversy titled “The Limits of Sugar Guidelines: Is There a Danger in Governments Offering Too-Specific Advice on Sugar Consumption?” It described the controversy raised by the original paper and Schillinger’s accompanying editorial and called out Schillinger and his coauthor as part-time advocates against sugar who write articles for Sugar Science,¹¹ a group described as devoted to educating the public about sugar’s health dangers. It also quoted the editor of the Annals of Internal Medicine: “It’s shown me that COIs are not only financial but also intellectual.” In essence, said Schillinger, the editor of Annals of Internal Medicine used the same argument that the ABA representative used to undermine his systematic review and current editorial because he had a so-called intellectual COI equal in importance to a financial COI.

That raised the question of whether writing for Sugar Science represents an intellectual COI. Sugar Science, Schillinger explained, is an educational website founded by Schmidt and sponsored by University of California, San Francisco (USCF); it is a repository of studies that address questions about the relationship between added sugar and disease outcomes. It is an academic and educational resource, with no industry funding. He added that he had written one blog post for the website.

Schillinger ended with a list of closing thoughts and questions:

• Industry has unfavorably influenced science in multiple, insidious ways.
• The beverage industry has demonstrated its ability to manipulate the scientific process to shape what is considered scientific “fact” or scientifically “controversial.”
• Scientific—and policy making—communities must continue to be vigilant, in defense of pursuing truth for public health, about the effects of financial COIs.
• Many prestigious journals require expert reviews by biostatisticians. Should journals require not only COI disclosures but also careful reviews by experts in COIs?
• Should journal editors’ performance be assessed regarding their track record on COIs?
• How can researchers prevent the construct of so-called intellectual COIs from being used to undermine public health? A consortium of

___________________

¹¹ https://sugarscience.ucsf.edu/ (accessed February 3, 2023).

• journals has already made this decision with respect to the tobacco industry.
• When is it appropriate for journals to take a stance against publishing science funded by industries with an established history of manipulating the scientific process to promote their bottom lines while they undermine public health?
• How can researchers educate the media about the potential effects of COIs on science while also promoting public trust in science?

DISCUSSION

Cary Gross joined the session’s speakers and noted that one of his first research endeavors was to conduct an umbrella review of the association between financial ties and research outcomes. That review found that the odds of an industry-funded trial yielding a positive or pro-industry outcome were more than three times the odds of a non-industry trial, something that has become more pronounced over the past 20 years. “The impact of industry funding on the clinical research enterprise has been insidious yet transformational,” said Gross.

In his field, oncology, published randomized trials conducted by industry have increased from approximately 50 percent of those in high-profile medical journals to almost 90 percent by some estimates. Biomedicine, he said, has reached the point where industry sponsorship of clinical research has become the norm and researchers have become overly reliant on it. “It is causing us not only to have a bias in the research that is being done, but our whole research ecosystem is so dominated by industry sponsorship that it also affects not only the selection of research questions but also the scope and objectives of clinical research,” said Gross. He noted that the prevalence of industry sponsorship affects the research not being done, such as comparing two drugs within the same class or two treatment regimens that may be commonly used to determine which one is better. Research in the cancer world is also scant on prevention and on how to identify value-based, equitable, or high-quality care.

As a comment before opening the discussion to the rest of the panel, Gross said that it is important to think through how to best reassess the research ecosystem. “If we were going to design [it] from scratch, we never would have designed it to where we are right now, so how can we think about bold changes to reconfigure our ecosystem so it is meeting the needs of patients and of society?” he asked.

McKinney pointed out that clinical research is expensive, and the federal government has not provided enough funds to counterbalance industry funding of clinical trials. “In fact,” said McKinney, “we count on [FDA] to serve as an ex post facto sorter of what is too biased to use as

evidence, and that is probably not enough.” He noted that intellectual COI can indubitably be real and meaningful. The question is how to interpret it relative to the weight that comes from having someone design a study with a purpose, which happens with much of the sponsored research that is the subject of this workshop.

Bero noted the many influences on research and that it would be naïve to think that research is not influenced and is value free. What is important, she said, is bias, and the important feature of bias is that it results in a systematic deviation in results and potentially in inferences. “The talks today clearly illustrated these biases related to industry funding,” she said.

Bero asked Dunn if an automated tool to identify bias and assess risk of bias in an individual study would include funding source and investigator COI. Dunn replied that as financial COIs are associated with systematic biases, that would be put into a model of risk of bias. Other factors related to personal research agendas would not be included. If he were using machine learning to look for bias, he would capture as much information as he could that would connect to the document, such as the registration, protocol, and text of studies on similar topics, and use examples with bias annotated to train the model.

McKinney noted that the risk of a machine learning model being based on intrinsically biased data is enormous. Dunn agreed and explained that modern machine learning does address factors such as fairness and equity, interpretability, and explainability and other aspects related to generalizability and transportability. A modern model can explain the decision it made, identify the factors it used, and factor in fairness and equity via the technical approach to introduce annotation sampling. This point relates to the need to better share computable data and annotations. “I would love to have a database that has thousands of examples of individual studies that were annotated for bias retrospectively, because I could take those and then apply them to new studies that I had not seen before,” said Dunn.

Michaels remarked that it is a given that disclosure is necessary, but one important area does not require disclosure: comments on federal regulations. He noted that the Administrative Procedures Act requires all federal agencies to ask for public comment on any new proposed regulation, but a stakeholder who comments is not required to provide any information about who funded the comment. When he was the administrator of the Occupational Safety and Health Administration (OSHA), he included a request for COI disclosure for the proposed standards on worker silica and beryllium exposure. “It was incredibly controversial,” he said. “I received a letter signed by 13 Republican senators saying that will discourage people from sending in comments if they have to say who

paid for it.” Fortunately, he said, a strong editorial in Nature supported that proposal, and the disclosure requirement remained in place until the Trump administration ended it. “We should know who paid for these comments,” said Michaels.

Rita Redberg, professor of Medicine at UCSF and chief editor of JAMA Internal Medicine, agreed and pointed out that many patient advocacy groups are industry funded, and when she was chair of the Medicare Coverage Committee, many comments would come from representatives from these groups. However, these individuals rarely disclosed that connection, even when specifically asked about a COI. She also said that industry-funded commenters often note nonfinancial COIs to deflect from their financial COIs.

McKinney, posing a question from a virtual participant, asked if the panel could discuss allegiance bias among researchers. He explained that in his area of nutrition research, he can correctly presuppose the direction of an outcome and whether the study is about health risks from carbohydrates, fats, red meat, or other factors based on the title of a paper and the name of a senior author. McKee said that he has seen a great deal of this during the COVID-19 pandemic, where the authors of systematic reviews had a particular view on how serious the virus was that came out strongly in their papers. “One’s allegiance to a political party might be correlated pretty well with your allegiance to ivermectin as a treatment, for example,” added McKinney, adding that a great deal of evidence shows this was true.

Schillinger said he believes that every paper should be viewed with a concern for bias, but labeling every scientist who has an informed opinion on a scientific matter as guilty of an intellectual COI is a slippery slope. In terms of allegiance bias, he believes that that relationship between an investigator and the public differs greatly from that relationship when it is mediated by a third party whose interests go beyond speaking truth to the public. “This notion of a third party being involved whose interests may supersede those of the public’s interest or the pursuit of the truth is the critical difference between one’s allegiance to one’s identity and identity politics and how that might affect your relationship to the public versus one’s relationship to that third party and that third party’s mission,” Schillinger explained.

Bero cautioned that it is important to distinguish between someone who is an expert in a subject and therefore likely to be able to predict the results of a study from the investigator’s lab—which some might construe as bias, as seen with attitudes toward experts during the pandemic—versus someone whose research is biased because of a sponsor’s influence on the research. This is the beauty of meta-research, Bero said, because it looks across a whole body of findings on a particular topic and not just

from a particular lab. “That is when we can detect bias at the systematic deviation in the results when we look across the whole body,” she said.

Schillinger asked Dunn what he thinks should happen in terms of publication and dissemination to the scientific community and the public when an AI-generated score suggests bias may be possible. Dunn replied that the first step is to be more transparent about the data, the information underneath the data, and how an experiment was designed. “That is why things such as registrations, protocols, and structured reporting of results are so important, because then we can start to deal with some of the mechanisms of introducing bias into the research,” said Dunn.

In addition, with an estimate of the risk of bias in an individual report or media communication, it would be possible to flag it to signal the reader they should be careful about the conclusions they draw and perhaps qualitatively adjust what is known about that study. “I do not have the statistical skills to be able to figure out how we are going to do that,” said Dunn, “but there must be some way in synthesis that we can say we expect across this body of evidence that we are going to see more positive results than the truth, and so we need to adjust that.”

Gross asked the panelists for ideas on how to engage the media so reporters better understand, report, and potentially mitigate the effect of financial conflicts. Schmidt cited a structural issue: reporters are trained to show both sides and feel compelled to do so even when the evidence base is flawed and biased. In her experience, they have talked about being confused by hearing conflicting scientific opinions. A centralized resource on scientific COIs is needed that allows reporters to know whom they are talking to, said Schmidt. Bero noted that journalist Jeanne Lenzer keeps a long list of researchers without financial COIs for which researchers can apply to be included.¹²

___________________

¹² Available at https://jeannelenzer.com/list-independent-experts (accessed February 2, 2023).