6

Conformity, Certification, and Continued Product Performance

Key Messages from Individual Speakers¹

• There are many approaches to conformity assessment, and it is important to have a keen understanding and risk-based process for looking at which kind of conformity assessment is appropriate for what kind of PPE and PPT given its use and risks. (Gillerman)
• There is no single regulatory body, official guidance, or mandating authority for conformity assessment of all PPE. (Szalajda)
• While there are clear procedures to assess conformity for respirators, it gets a little grayer with other PPE products, including face masks, surgical masks, face shields, gloves, gowns, and other apparel that can be used primarily in a health care setting, but also by the general public. (Stull)
• Standards do not exist unless there are standards champions, so if someone or some organization wants a standard, they have to create the standard and work hard to get it accepted. (Stull)
• Other types of outcomes, such as comfort, ease of use, reusability, and cleanability are important to the public and workers required to wear PPE day after day but are not included in current respirator standards. (Brosseau)

___________________

¹ The following list of key messages is the rapporteurs’ summary of points made by the individual speakers identified, and the statements have not been endorsed or verified by the National Academies of Sciences, Engineering, and Medicine. They are not intended to reflect a consensus among workshop participants.

Continuing along the components of the personal protective equipment (PPE) and personal protective technology (PPT) supply chain, the speakers in the eighth session focused on standards gaps related to conformity testing, certification, and maintaining the integrity of the product during storage, decontamination, and reuse. The discussion also focused on how conformity testing, and certification processes differ by product type and use scenario. Session moderator Jill Morgan noted that the Product Standardization Task Force² identified numerous gaps in standards around conformity assessment across PPE types. For example, respirators have more rigorous oversight in some ways around conformity assessments, and while National Institute for Occupational Safety and Health (NIOSH) certification requirements mandate that cleaning, disinfecting, and/or decontaminating procedures are supplied by the manufacturer in their user instructions, there is no standard that manufacturers must meet to show the efficacy of cleaning, disinfecting, or decontaminating respirators. There is also a lack of conformity assessment criteria for gloves and gaps in surveilling conformity to standards once a product is on the market, said Morgan.

CONFORMITY ASSESSMENT IN THE NATIONAL STRATEGY FOR A RESILIENT PUBLIC HEALTH SUPPLY CHAIN

Jon Szalajda, deputy director of the National Personal Protective Technology Laboratory (NPPTL) at NIOSH, explained that conformity assessment is the demonstration that a product meets specified requirements. Conformity assessment, he said, can verify that a product meets a level of quality or safety, and it can provide explicit or implicit information about the product’s characteristics, the consistency of those characteristics, and the performance of the product. Conformity assessment can also increase a buyer’s confidence in a product, furnish useful information to a buyer, and help substantiate advertising and labeling claims. Information on conformance or nonconformance to a particular standard can provide an efficient method of conveying information needed by regulators or buyers on the product’s safety and sustainability.

In addition, said Szalajda, conformity assessment serves as the vital link between product requirements and the products themselves. Although the Occupational Safety and Health Administration (OSHA) and other agencies such as NIOSH publish guidance for effective worker protection using PPE, there is no single regulatory body, official guidance, or mandating authority for conformity assessment of all PPE.

___________________

² Additional information is available at https://aspr.hhs.gov/newsroom/Pages/SupplyChain-9Mar2022.aspx (accessed May 2, 2023).

Szalajda noted that the National Strategy for a Resilient Public Health Supply Chain³ included a recommendation to develop and deploy a durable process for assuring conformity assessment of PPE with relevant standards to ensure the quality of and appropriate use for the supply of products prior to an emergency arising (DoD et al., 2021). The strategy also called for federal agencies, including NIOSH, to participate with other agencies with overlapping authorities to plan for future collaborations to implement this process. He explained that the strategy looked at two primary issues that the supply chain could address: (1) the quality of the products received and (2) the inconsistent coordination across federal, state, and local partners. Impacts on the supply chain, said Szalajda, start with designating the required standards to which products must conform, coupled with a reliable means for demonstrating conformance during a product’s life cycle. These standards inform the supply chain by identifying performance, quality, and reliability of qualified products.

Szalajda noted that the strategy contained a few other points about conformity, including that it should clearly inform potential suppliers about the standards and acceptable conformity assessment approaches to assist in preparing for acquisitions. In addition, he said, the strategy noted that the PPE conformity assessment framework should establish a national process, such as third-party certification of supplier qualifications, and that there should be clear inspection and acceptance criteria. The strategy also called for monitoring suppliers for performance through ongoing audits of supplier-required qualification.

Looking at the realm of standards from a conformity perspective, Szalajda said that respirators have the greatest degree of regulatory oversight, with federal regulations that identify the requirements for these products. NIOSH performs post-market evaluations to determine how a product is performing in the field. Beyond that, few PPE standards have conformity assessment requirements in place, let alone standards for life cycle requirements. He noted that ongoing processes present opportunities to identify specific gaps and limitations in PPE standards and target needed changes. There are also opportunities, he added, to make suggestions for improving existing standards and fostering the development of new standards that address the life cycle of PPE products.

Szalajda said that several years ago, absent national policy and guidance, NIOSH developed a national framework for PPE conformity assessment (NIOSH, 2017). It established a set of principles and provided guidance for conformity assessment of PPE in the United States by addressing

___________________

³ For the full report, see https://www.phe.gov/Preparedness/legal/Documents/National-Strategy-for-Resilient-Public-Health-Supply-Chain.pdf (accessed May 2, 2023).

the risk of worker exposure that results from using a nonconforming product. This framework, he explained, helps advance the PPE industry by meeting the need for a consistent risk-based approach to conformity assessment.

Standards, he said, are a critical part of the conformity assessment process. Standards provide the basis for conformity assessment activities and for many buyer-seller transactions. “Standards used in conformity assessment activities can have tremendous impacts on companies and nations and even the economic fabric of the world market,” said Szalajda.

CERTIFICATION AND CONFORMITY ASSESSMENT

Gordon Gillerman, director of the standards coordination office at the National Institute of Standards and Technology (NIST), said that NIST is driven by risk. “What we are thinking about in conformity assessment is what are the risks associated with nonconformity of these products, how bad it will be, and how often will we see those results,” said Gillerman. He noted there are two elements of risk: (1) the frequency at which the hazard occurs and (2) the consequences of the hazard. When thinking about conformity assessment from a risk-based perspective, NIST looks at how it can bring more confidence to PPE when the risk level is higher, which is when the consequences of nonconformity are significant or when the frequency related to the hazard and nonconformity of the products would be common.

Gillerman said there are several factors to consider in a conformity assessment. One relates to the technical rigor of the conformity assessment program. Much of that is driven by the technical standards for product performance—the more technical requirements there are for product performance, the more tests there are to run. Another factor deals with testing products with normal cleaning and disinfecting and running them through the test methodology to see if they still perform as expected. “These all have to do with the technical rigor of conformity assessment programs, but confidence itself is often determined by who makes an attestation of conformity and who tells the public these things meet the requirements,” said Gillerman.

In general, he said, there are three choices. One is to trust the manufacturer’s statement that its products conform to the requirements. The second is for the product’s user to test a sample to make sure it works before deploying it. This option is challenging because the test equipment is special, and most companies or members of the public do not have access to it. The third choice is for third-party attestation that a product conforms to the requirements. There are options to this latter choice, said

Gillerman. One is for an impartial third party to run a set of tests on a group of samples and the other is certification, where, based on evaluating evidence, the certifier states there is a system in place that provides confidence that the entire population of these products will conform to the requirements.

Conformity assessment exists in this hierarchy, said Gillerman, which raises the question of who is watching the watchers (Figure 6-1). For example, a manufacturer’s laboratory, a laboratory for a user with the right equipment, or a third-party laboratory all have the potential to run the tests. If there is an accreditor, the accreditor’s job is to determine whether the conformity assessment organization or laboratory is competent to conduct the technical work required for the scope of work and have an underlying quality and risk management system that meets international standards for conformity assessment.

Gillerman explained that a certification body, such as the Underwriters Laboratories (UL), is always a third party. Certification tells the user that the product they own conforms to the requirements. What is important there is that the certification body enacts surveillance activities to ensure that the production process continues to make products that conform to the requirements. NPPTL performs its own form of surveillance on NIOSH Approved^®4 respirators, for example. In summary, said Gillerman, there are many approaches to conformity assessment, and it is important to have a keen understanding and risk-based process for looking at which kind of conformity assessment is appropriate for what kind of PPE and PPT given its use and risks.

___________________

⁴ NIOSH Approved is a certification mark of the U.S. Department of Health and Human Services (HHS) and is registered in the United States and several international jurisdictions.

CONFORMITY ASSESSMENT BEYOND RESPIRATORS

Jeffrey Stull, president of International Personnel Protection, Inc., said that while there are clear procedures to assess conformity for respirators, it gets a little grayer with other PPE products, including face masks, medical masks, face shields, gloves, gowns, and other apparel that can be used primarily in a health care setting, but also by the general public. For all these products, said Stull, the Food and Drug Administration (FDA) has some oversight, which means one of two things: (1) either a manufacturer has undertaken a program to demonstrate a product conforms to a standard but need not submit a formal application for review, or (2) a products fits into the more formalized 510(k) clearance process (FDA, 2021a). The latter involves the FDA reviewing a product for what it believes is appropriate based on guidance documents. Stull noted that guidance documents are not often up to date. He added that while a manufacturer might highlight that a product has been cleared by the FDA, clearance does not include on-site audits and it is not a conformity assessment process.

For PPE other than respirators, there is a range of government oversight and standards (Figure 6-2). For surgical masks, ASTM F2100 specifies what materials can be used and includes various sampling requirements and test methods on top of conformity assessment (ASTM, 2023). Stull pointed out that there are three levels of performance requirements in terms of fluid resistance and breathability, but there is no guidance for health care organizations on how to interpret the three levels for the purpose of selecting surgical masks.

The face shield and other eyewear and face wear category includes safety glasses, non-ventilated goggles, disposable face shields, and industrial face shields. Stull said the International Safety Equipment Association’s (ISEA) Z87.1 standard covers the whole spectrum of eyewear for industrial and educational uses, but it does not specifically apply to the health care environment (ANSI and ISEA, 2020). There is no conformity assessment process or requirements in that standard, and while there is a new ISEA standard for health care, it focuses specifically on preventing blood spurts or blood sprays from contacting the face, eyes, or nose, with other requirements for visibility and clarity that are detailed in the recently promulgated ISEA Z87.62 (ANSI and ISEA, 2021). However, the new standard does not address cleaning, fogging, or other issues important to many end users, and it does not provide a set of requirements for conformity assessment.

Stull said there are seven standards for gloves based on their intended use and the material from which they are made. Most gloves, he said, are characterized for waterproofness, strength, aging, and powder levels. Except for the National Fire Protection Association (NFPA) standard that

covers exam gloves, there is no conformity assessment process in these standards.

Stull characterized the standards for surgical and isolation gowns as a bit more robust than for gloves, though the primary standard used for classifying the barrier performance of gowns (AAMI PB70) lacks a conformity assessment process. Only recently, he added, was a conformity assessment requirement added to the supplemental individual ASTM standards for both surgical (ASTM F2407) and isolation gowns (ASTM F3352), including testing by an accredited laboratory and the provision of a supplier declaration.

One voluntary standard from NFPA that requires independent third-party certification applies to coveralls and several other types of emergency medical clothing and equipment for firefighters, emergency medical technicians, and other first responders. However, said Stull, that standard is so robust that most manufacturers ignore it, perhaps because of the cost of going through the certification process and the continuing obligations for maintaining that certification.

Stull said that other than respirators and some PPE for emergency responders, most conformity assessment is based on self-declaration. In fact, much of the PPE industry resists mandatory independent certification. There is confusion, he noted, as to how standards are applied, particularly for new entrants to PPE manufacturing, and end users do not always understand FDA’s role in clearing PPE as medical devices. Stull added that gaps exist in several standards related to expectations pertaining to continued use, with no metrics for cleaning effectiveness on reusable products. Furthermore, surveillance programs often do not exist to establish product shelf life. On a final note, Stull said that standards do not exist unless there are standards champions, so if someone or some organization wants a standard, they will have to work hard to get it accepted.

THE ROLE OF SCIENTIFIC KNOWLEDGE IN DEVELOPING STANDARDS

Lisa Brosseau, retired professor at the University of Illinois at Chicago and research consultant at the University of Minnesota Center for Infectious Disease Research and Policy, said that while science needs to inform standards, it is important to understand the limits of science when developing standards for PPE. “That is why the requirement that a consensus standard be reevaluated every few years is important, because scientific knowledge can change rapidly,” said Brosseau. For example, a standard was developed during the COVID-19 pandemic to evaluate the ability of a barrier face covering to limit outward leakage or emission of particles from inside the facepiece.

At the moment, without a standard for evaluating source control, the best method for determining outward leakage involves measuring particle concentrations in a chamber with a mannequin wearing a mask compared to particle concentrations when the mannequin is not wearing a mask. Brosseau explained that NIOSH studies using the mannequin method demonstrated that inward and outward leakage are similar for things with good filters and good fit, such as respirators. However, inward and outward leakage are not similar for things such as cloth and surgical masks that have poor filters and do not fit well. As a result, using inward leakage as a surrogate for outward leakage is not appropriate for all types of facepieces. Brosseau said ASTM will update its barrier face covering standard with an outward leakage method once the science provides some direction.

Brosseau said she agrees with the speakers who have discussed the need for mask and respirator standards for the public before the next pandemic strikes, as well as to protect the public from other types of hazardous aerosols. It may be, however, that the public does not need something as effective as a respirator, which is designed to be worn for 8 or more hours in a workplace, but masks for public use should be more effective than a cloth or surgical mask. “Do we need the same standards and expectations for public respirators as we have for those used in the workplace?” she asked. “In some respects, I think the answer is yes. The tests used to evaluate filter efficiency and breathing resistance are very appropriate for all types of things worn on the face.”

At the same time, she added, other types of outcomes are important to the public and workers required to wear PPE day after day, such as comfort, ease of use, reusability, and cleanability, for example, that are not included in current respirator standards. Ninety-five percent filter efficiency may not be necessary for a public respirator, said Brosseau, because N95^®5-filters are overengineered for most workplace exposures. A public respirator also needs breathing resistance lower than those specified in the NIOSH standard, and there is a need for some creative solutions to the challenge of conducting a fit test for members of the public, something NIOSH is pursuing.

Brosseau said it is also important to consider standards and performance requirements in the context of a pandemic compared to other types of emergencies and to understand how those differences affect the types of PPE needed and the supply chain that is necessary to provide that PPE. “We do not need to add to the already heavy burden of solid waste with large quantities of disposable PPE, for example. We need standards that ensure long-term storage, interoperability, and reusability, and standards

___________________

⁵ N95 is a certification marks of the U.S. Department of Health and Human Services (HHS) and are registered in the United States and several international jurisdictions.

for rapid manufacturing without sacrificing quality and user safety,” she said.

As a final comment, Brosseau noted that the underlying decisions about which interventions to use during a pandemic are included in the National Pandemic Plan, which focuses on influenza and does not mention the words “airborne” or “aerosol” and never discusses the use of respirators or masks (CDC, 2022). “Hopefully, we will rethink our National Pandemic Plan with a more precautionary approach, which is that the next pandemic is likely to involve an organism that is highly transmissible by aerosol inhalation person to person, and it is also likely to have much higher lethality than SARS-CoV-2,” said Brosseau. A more cautionary approach, she added, will help move toward ensuring a better and adequate supply of PPE for workers and the public in the next pandemic.

EVALUATING SUPPLY CHAIN RESILIENCE

David Simchi-Levi, professor of engineering systems at the Massachusetts Institute of Technology, said that in the 10 to 20 years preceding the COVID-19 pandemic, most companies in a variety of industries focused on supply chain efficiency through strategies such as just-in-time manufacturing, outsourcing, and offshoring, which enabled them to cut costs from their supply chains. These strategies, however, significantly increased the exposure to risk, and events such as the 2011 tsunami in Japan and the 2011 flood in Thailand had enormous impact on companies in a variety of industries.

To address the tradeoff between supply chain efficiency and resilience, Simchi-Levi and his colleagues collaborated with the Ford Motor Company to develop a new way for companies to identify the level of resiliency in their supply chain, find hidden risk, and develop mitigation strategies (Simchi-Levi et al., 2014). He noted that between the time he and his collaborators published their work in 2014 and the start of the COVID-19 pandemic, there was little interest in focusing on supply chain resiliency. However, when the pandemic started and as he collected data on what was happening in China, he realized he could use the model he and his collaborators had developed to understand how what was happening in China would affect supply chains in North America and Europe, predicting that supply chain disruptions would peak in mid-March 2020. Moreover, these disruptions would force companies in North America and Europe to shut down or significantly reduce manufacturing and assembly activities (Haren and Simchi-Levi, 2020).

As for the capabilities, he believes there are two important concepts for the nation to consider when seeking to improve supply chain resiliency and to identify any existing risks. The first is time to recover, which

is how long it takes for a specific manufacturing, distribution, or other facility in the supply chain to recover full functionality after a disruption. Time to recover is scenario dependent, but there are methods to estimate it, such as, by understanding business continuity plans established for each facility. For example, the time to recover for the chemical supply chain would be approximately 2 weeks. The second important concept, said Simchi-Levi, is performance impact, the impact of a disruption during the time to recover on a performance measure, such as profit, revenue, market share, or loss of production. Simchi-Levi added that the relationship in his framework between supply chain performance, business performance, and time to recover can provide insights on the possible effect of conformity assessment.

One important insight from his research is that risk is not directly related to geography. “There is a notion in the public media that if you just move manufacturing activities from, say, Asia closer to market demand, you are going to guarantee resiliency, but this is not consistent with our data and analysis,” said Simchi-Levi. While there may be many reasons to move manufacturing activities closer to market demand, doing so by itself does not ensure supply chain resiliency. For example, when he analyzed Ford Motor Company’s risk, he found that the company had risky suppliers in Asia, Europe, and North America. During the COVID-19 pandemic, this was seen in the meat processing industry. This industry, which is entirely domestic, went through significant consolidation before the pandemic to cut supply chain cost. But during the pandemic, “when one of the manufacturing facilities was affected, there was an enormous impact on the supply of meat in North America,” he said.

While time to recover is scenario dependent, a different measure, called by Simchi-Levi time to survive, is not. This measure identifies the maximum duration that the supply chain can match supply with demand after a disruption. “If this time-to-survive measure is very short, no matter what the scenario is, I am very concerned about the supply chain’s ability to match supply with demand,” he said. Simchi-Levi also noted that intuitive analysis of the supply chain does not necessarily provide insights into where risk is, how to develop mitigation strategies, or how to ensure that the supply chain can continuously match supply with demand, especially in health care. As a final comment, he said that while just-in-time manufacturing increases exposure to risk, quantifying efficiency, resiliency, and the impact of lean approaches on time to survive, time to recover, and business performance enables a company to decide where it wants to be regarding the tradeoffs between efficiency and resiliency. He noted that he has seen a shift from just-in-time to a combination of just-in-time and just-in-case.

DISCUSSION

Morgan opened the discussion by asking the panelists how conformity assessments affect the supply chain and if conformity assessments extend the time to survive. Gillerman said the more third-party intervention that is required to conduct conformity assessment, the longer the time to recover from a disruption. For example, if there is a supply chain disruption that makes a particular material unavailable, substituting another material in a PPE item will require the item to get certified in its new form. If the supplier or manufacturer had in-house technical expertise to run the tests needed to demonstrate compliance, getting the new product to market would occur more quickly than if the manufacturer had to contract with an outside laboratory to certify the modified product. The tradeoff here, said Gillerman, is the level of confidence one gets from a third-party conformity assessment versus a supplier- or manufacturer-conducted assessment. “There might be times during critical issues in the supply chain where we could flex down our confidence needs publicly in order to recover the supply quicker,” said Gillerman.

Another approach, he added, would be to distribute conformity assessment resources across multiple geographic locations. Stull noted that conformity assessments could affect the ability of new companies to enter the market at times of heightened demand. The challenge, said Brosseau, is figuring out how to make the conformity assessment process work and address risk without affecting health and safety. During the pandemic, for example, FDA allowed KN95 respirators to be imported, and NIOSH did some testing, but not enough to avoid the fact that some of the imported KN95 respirators did not work properly and some were counterfeit.

Morgan then asked the panelists for their thoughts on ongoing conformity assessment of stockpiled PPE. Szalajda and Stull both noted that PPE manufacturers could conduct accelerated aging tests on products that would provide some confidence in the viability of stockpiled products. Stull added that this type of testing is limited in the PPE world. Morgan than asked if self-declarations of conformity by PPE producers are trustworthy. Stull replied there are companies that are diligent about doing the right thing and some that may cut corners. An intermediate solution would be to have test results from an independent, accredited laboratory attached to a manufacturer’s self-declaration. Szalajda said that an important consideration is the risk a user takes on by using a nonconforming product.

Responding to a question about whether the qualities in a standard are both relevant and sufficient to the risks people encounter, given today’s dynamic environment of changing workplaces and lifestyles, Stull

said that most standards are reactive rather than proactive. This is why standards rely more on performance requirements than design requirements. Gillerman said the user community needs to be considered in the technical standards for performance requirements. For example, there are defibrillators intended for use by health care providers and there are automated external defibrillators designed to be used by the public. Those intended for use by the public are designed to help individuals determine whether a patient needs defibrillation and where to put the electrodes, which is not needed for devices intended for use by health care professionals.

This page intentionally left blank.