key: cord-0828190-jthc4vau
authors: Andrews, Angela S.; Powers, John R.; Cichowicz, Jaclyn K.; Coffey, Christopher C.; Fries, Marisa L.; Yorio, Patrick L.; D’Alessandro, Maryann M.
title: Respiratory Protection in a Time of Crisis: NIOSH Testing of International Respiratory Protective Devices for Emergency Use
date: 2021-01-11
journal: Health Secur
DOI: 10.1089/hs.2020.0173
sha: d0ff7b7ddf579e7be7d7abf9ec872430c1a4e1f3
doc_id: 828190
cord_uid: jthc4vau

National Institute for Occupational Safety and Health (NIOSH)-approved respirators are required by the Occupational Safety and Health Administration (OSHA) when personal respiratory protection is used in US occupational settings. During the COVID-19 pandemic, the demand for NIOSH-approved N95 filtering facepiece respirators overwhelmed the available supply. To supplement the national inventory of N95 respirators, contingency and crisis capacity strategies were implemented and incorporated a component that endorsed the use of non-NIOSH-approved respiratory protective devices that conformed to select international standards. The development and execution of this strategy required the collaborative effort of numerous agencies. The Food and Drug Administration temporarily authorized non-NIOSH-approved international respiratory protective devices through an emergency use authorization, OSHA relaxed their enforcement guidance concerning their use in US workplaces, and NIOSH initiated a supplemental performance assessment process to verify the quality of international devices. NIOSH testing revealed that many of the non-NIOSH-approved respiratory protective devices had filtration efficiencies below 95% and substantial inconsistencies in filtration performance. This article reports the results of the NIOSH testing to date and discusses how it has contributed to continuous improvement of the crisis strategy of temporarily permitting the use of non-NIOSH-approved respirators in US occupational settings during the COVID-19 pandemic.

The NATIONAL Institute for Occupational Safety and Health (NIOSH) -National Personal Protective Technology Laboratory (NPPTL) of the Centers for Disease Control and Prevention (CDC) is responsible for the conformity assessment of respirators used in occupational settings. Conformity assessment encompasses preapproval testing and quality assurance evaluations and includes postapproval product and site audits. The stringency of this comprehensive approval process assures that once a respirator has earned NIOSH approval the user can be confident that the device will provide the expected level of protection, as long as it fits properly and is selected, maintained, and worn correctly. The Occupational Safety and Health Administration (OSHA) requires that only NIOSHapproved respirators be used in US workplaces. 1 It had been projected that during a public health emergency the supply of N95 respirators may become strained, 2,3 and, indeed, a global shortage of NIOSH-approved N95 filtering facepiece respirators was experienced during the coronavirus disease 2019 (COVID-19) pandemic. 4 Given the high demand for respiratory protective devices (RPDs) by healthcare personnel, the shortage prompted several governmental and nongovernmental organizations to implement strategies to conserve the limited supply of NIOSH-approved respirators. Such strategies included control banding, a technique used to guide the assessment and management of workplace risks 5 ; extended use and limited reuse of N95 respirators 6, 7, 8 ; the use of respirators beyond their shelf life stored in strategic stockpiles 9 ; decontamination of respirators 10 ; and the use of NIOSH-approved respirators designed to be reused, such as elastomeric and powered air-purifying respirators. 11 These strategies have not been enough to meet the demand for respiratory protection during the COVID-19 pandemic due to the increased use of RPDs beyond acute hospital care settings, such as nursing homes and ambulatory care clinics. Additional crisis strategies were, therefore, needed. To increase the availability of RPDs, the CDC 12 and the US Food and Drug Administration (FDA) 13, 14 implemented a crisis strategy to permit the occupational use of non-NIOSH-approved RPDs designed to meet international standards that incorporate requirements similar to NIOSH-approved N95 respirators. 15 To inform this process, NPPTL recommended the use of performance standards that specified protection factors of at least 10* from 7 international organizations associated with Australia, Brazil, China, Europe, Japan, Korea, and Mexico. 12 It further recommended that non-NIOSHapproved RPDs developed by manufacturers that were not NIOSH approval holders should be used only in crisis situations when no other NIOSH-approved N95 respirator or respirator produced by a NIOSH approval holder is available. At the same time, because of the shortages experienced during the COVID-19 pandemic, OSHA relaxed its enforcement guidelines around RPDs used in occupational settings. 16 The primary mechanism used to implement these crisis strategies during the COVID-19 pandemic was a series of emergency use authorizations (EUAs) issued by the FDA. 13 An initial EUA was issued on March 24, 2020, for specified disposable RPDs. A separate EUA for certain disposable RPDs manufactured in China was issued on April 3, 2020. 13 To be eligible, respirators had to meet specified criteria. † Authorized filtering facepiece respirators meeting the specified criteria were included in Appendix A of the EUA. Shortly after the April 3, 2020, EUA was issued, nearly 90 Chinese manufacturers were included in Appendix A. 17

In response to public inquiries and concerns regarding the quality of international RPDs temporarily authorized for occupational use, ‡ NIOSH developed a supplemental filtration performance assessment process. 18 The NIOSH testing incorporated a modified version of NIOSH Standard Test Procedure TEB-APR-STP-0059. § The abridged test included a focused assessment of the particulate filtration (air-purifying) efficiency of RPDs. Inhalation and exhalation resistance were not included.

The test was developed to rapidly evaluate the filtration efficiency of international RPDs temporarily authorized for emergency use in occupational settings for the health and safety benefit of US workers and to transparently communicate the test results. NPPTL received requests directly from federal, state, and local government agencies, healthcare providers, employers in nonhealthcare industries, public safety and first responder organizations, and universities. ** Information such as the manufacturer name, model designation, and performance standard under which the RPDs were reportedly manufactured were required components of the request. * An OSHA assigned protection factor of 10 will reduce exposures by one-tenth. 1 † The criteria were different between the EUAs issued on March 24 and April 3, 2020. They were also revised after the FDA became aware of performance concerns. 13 The strategy of the EUAs was to ensure an adequate supply of personal protective equipment available to the healthcare industry. However, the overall strategy affected every industry in the United States as indicated by OSHA's issued enforcement guidance 16 and as evidenced by the industries represented among those that made request to NPPTL to conduct testing. ‡ NPPTL received a total of 253 inquiries about international respiratory protective devices originating from China during an approximate 7-week time period, from March 22 to May 6, 2020.

§ Each respiratory protective device was tested on a TSI Inc. Automated Filter Tester, model 8130 or 8130A, set to the following parameters: the flow rate was set to 85.0 ± 4.0 liters per minute; the aerosol concentration was l ≤ 200 mg/m 3 ; the particle size distribution was 0.075 ± 0.020 micrometer with a geometric standard deviation not exceeding 1.86. Each RPD was tested for 10 minutes; the maximum penetration was used as the metric to compute the observed filtration efficiency for each device tested. If valves were present on the devices, they were sealed prior to testing. 19 ** Requests received directly from a manufacturer, distributor, importer, or supplier were outside the scope of this evaluation process.

For each valid request received, a minimum of 10 RPDs (of the same model and from the same manufacturer) were tested. The sampling protocol used to select the 10 RPDs was at the discretion of the group making the request; however, requestors reported using both a convenience, nonprobability sampling technique (in which samples were drawn from the population based on their availability) or a random sampling technique (in which each unit in the defined population has an equal probability of being selected for testing).

After NIOSH testing was complete, results were communicated to the requesting group through a formal report and were publicly released online. 19 The reports included basic information about the RPD evaluated (eg, manufacturer name, model information, international standard claim), photographs, and the results for the sample RPDs tested. The results were also openly and rapidly shared with the FDA to aid the collaborative process of continually improving the EUAs.

The results included the maximum and minimum filtration efficiencies observed among the RPDs tested. † † By reporting the maximum and minimum efficiencies, the observed range in filtration efficiency could be computed. The range indicates the consistency of filtration efficiency between the units evaluated. Understanding the consistency in filtration efficiency is important because users must be able to trust that individual units, which are labeled and packaged identically, provide a consistent level of performance when respiratory hazards are encountered in the workplace. Further, as required in 42 CFR 84, 20 NIOSH-approved N95 filtering facepiece respirators consistently filter out at least 95% of particulate matter, and the strategy to supplement the national supply with international RPDs includes an expectation of a similar level and consistency in filtration efficiency.

NPPTL received valid requests for testing nearly immediately after initiating the supplemental performance assessment process. The first NIOSH test was completed on April 10, 2020, 1 week after the EUA was issued. By the end of April 2020, NIOSH tests for approximately 80 distinct sets of international RPDs were completed. The test results revealed quality concerns related to both the level and consistency of observed filtration efficiency. In part due to these findings, on May 7, 2020, the FDA revised the EUA for RPDs manufactured in China by removing many of the manufacturers and models initially included in Appendix A. 17 In addition, the FDA and NPPTL further partnered to heighten the role of NIOSH testing and increase the surveillance of all RPDs imported from China. 13 The testing procedure was revised to increase the sample size and statistical power and to emphasize random sampling, thereby enhancing the ability of each test to uncover distinct pockets of filtration performance within consistently labeled RPDs.

After the April 3, 2020 EUA was revised on May 7, 2020, NIOSH testing results were categorized in 2 phases: Phase 1 included tests completed up to May 6, 2020, and Phase 2 included tests completed between May 7 and August 17, 2020.

By May 6, 2020, NIOSH testing had been completed for 105 international RPDs at the request of state governments (29%), healthcare providers (24%), employers in nonhealthcare industries (21%), public safety and first responder organizations (10%), individuals and organizations not categorized (6%), federal government agencies (5%), universities (3%), and local governments (2%). Of the RPDs tested, approximately 90% used an ear loop design to secure the mask to the wearer's face, and the others used a head strap design.

Through the packaging and labeling, NPPTL verified the manufacturer, model, and performance standard to which the samples claimed conformance. NIOSH testing was completed for international RPDs that were largely distinct in terms of the samples assessed: there were 87 manufacturers, 102 models, and a variety of international standards represented. Some samples received did not readily indicate the international performance standard to which the product conformed. In these cases, NPPTL classified the performance standard as "unknown." Table 1 The average range in filtration efficiency was calculated by subtracting the lowest observed filtration efficiency from the highest observed filtration efficiency, for each of the 105 tests analyzing at least 10 consistently packaged and labeled RPD units. The average and standard deviation of the ranges is shown in Table 2 . There was large variability in filtration efficiency across the tests in which all units were observed below 95% efficiency and those with a mixture of units that tested above and below 95% efficiency. The greatest variability in the range of filtration efficiency results was about 78% (maximum 91%, minimum 14%), for a sample of 10 units that claimed to conform with the Chinese standard GB2626-2006.

NPPTL completed an additional 251 tests of non-NIOSH-approved international RPDs following the FDA May 7, 2020 revision of its April 3, 2020 EUA. During Phase 2, 170 distinct manufacturers and 177 distinct models were tested at the request of federal government agencies (47%), employers in nonhealthcare industries (27%), state governments (12%), healthcare providers (9%), local governments (3%), individuals and organizations not classified (1%), and public safety and first responder organizations (1%). Of the RPDs tested, approximately 95% used an ear loop design to secure the mask to the wearer's face and the remainder used a head strap design (5%). Similar to Phase 1, a majority (n = 175, 70%) of the tests were conducted on samples of RPDs that were reported to conform to the Chinese GB2626-2016 standard ( Table 1) .

As shown in Table 2 , 104 (41%) of the units tested above 95% particulate filtration efficiency, 82 (33%) of the units tested below 95%, and 65 (26%) of the units were a mixture that tested above and below 95%. As in Phase 1, large variability in filtration efficiency was found across the tests in which all units were below 95% efficiency and those in which there was a mixture of units testing above and below 95% efficiency. The highest range in filtration results was 87% (maximum 92%, minimum 5%) for a sample of 10 RPD units that claimed to conform with both the European EN149-2001 and Chinese GB2626-2006 standards. Figure 1 shows a comparison of the results between Phase 1 and Phase 2. In Phase 2, we observed an 8% increase in tests in which all units tested above 95% filtration efficiency and a 7% decrease in tests in which all units tested below 95% filtration efficiency. These changes may be partly due to an increased awareness of the NPPTL testing results being shared with the FDA in a transparent and timely manner. 

This case study represents an interagency collaboration designed to support the health and safety of the US workforce throughout the COVID-19 pandemic. The NIOSH supplemental testing of international RPDs filled a critical gap during the COVID-19 pandemic by communicating evidence-based test results within 1 week (usually within 48 hours of the units being received) to the public and other agencies involved with the response. 21 These results directly contributed to FDA continuously updating the EUAs to add and remove RPDs authorized for use in healthcare settings. A third phase of NIOSH testing is underway, representing a continuation of the supplemental testing of non-NIOSH-approved RPDs that continue to be listed on the FDA's EUA.

Although NPPTL has conducted an extensive number of tests of international RPDs, other entities have conducted similar tests. Across the country, state governments have partnered with universities and third-party laboratories to evaluate the performance of RPDs conforming to international standards to inform purchasing decisions or remove substandard products from the market-all for the benefit of the US workforce in need of RPDs during the pandemic. 22 The Massachusetts Institute of Technology and Harvard University, for example, found that many of the filtering facepiece respirators available for procurement during the COVID-19 pandemic do not provide levels of fit and filtration similar to those of NIOSH-approved N95 respirators. 23, 24 The NPPTL procedure provides a consistent protocol for evaluating these devices and enables third-party laboratories and other organizations to conduct the testing on their own. 25 As the COVID-19 pandemic continues, testing will remain important to the health and safety of the US workforce.

Several limitations should be noted. First, given that a random sampling technique could not be guaranteed, the test results are generalized to the sample of international RPDs tested and may not be representative of a larger sample. Second, the tests were designed to provide a rapid evaluation of the filtration efficiency only. Given that a standard NIOSH approval evaluation consists of additional performance tests, as well as a comprehensive quality assurance review of the quality process and manufacturing site, any type of comparison between the international devices assessed and NIOSH-approved N95 respirators is not possible. In addition, certificates of approval or validation of conformance with an international standard were not provided with the samples. The authenticity of a claim that a product met or did not meet a particular international standard could, therefore, not be made.

Given these limitations, the results suggest that a substantial proportion of RPDs tested did not provide consistent filtration protection above 95% when evaluated using an adapted version of NIOSH's test method. While the international RPDs are not a precise equivalent, the standards considered incorporate filtration efficiency specifications that are nearly equivalent to NIOSH's requirement for N95s to have at least 95% filtration efficiency.

There are several possible reasons why the products assessed did not perform to these expectations. First, companies newly established to meet the global demand for RPDs during the COVID-19 pandemic may not have had the time or opportunity to establish quality control systems needed to consistently produce high-performing RPDs. Second, shortages or an inconsistent supply of the raw materials needed to produce high-performing RPDs may have forced manufacturers to procure lower-performing filtration material. Third, several counterfeit reports were received by NPPTL. In such cases, documents may have been altered to make RPD models appear to comply with a standard when they do not, or manufacturers' names, logos, and model numbers may have been misrepresented or counterfeited. For questions regarding product authenticity and potential counterfeiting, consumers are urged to directly contact the manufacturers and others involved in the supply chain.

Potential purchasers of international RPDs should refer to the CDC and NIOSH websites for guidance on how to evaluate respirators from other countries to determine if they provide adequate protection before making purchasing decisions. Consumers are urged to consult the FDA's current EUAs and the results of NPPTL supplemental performance assessments. 20 Abbreviations: NIOSH, National Institute for Occupational Safety and Health; SD, standard deviation.

Note: NIOSH calculated the average range in filtration efficiency by subtracting the lowest observed filtration efficiency from the highest observed filtration efficiency, for each of the 105 tests analyzing at least 10 consistently packaged and labeled respiratory protective device units. This provides an indication of how consistent the unit-to-unit filtration properties were within each test. For example, if the highest filtration efficiency is 95% and the lowest is 60%, the average range in filtration efficiency is 35% (indicating low confidence in the level of protection).

Respiratory Protection. 29 CFR §1910

Potential demand for respirators and surgical masks during a hypothetical influenza pandemic in the United States

Personal protective equipment supply chain: lessons learned from recent public health emergency responses

Rational Use of Personal Protective Equipment for Coronavirus Disease

Approach to prioritizing respiratory protection when demand exceeds supplies during an influenza pandemic: a call to action

Planning for epidemics and pandemics: assessing the potential impact of extended use and reuse strategies on respirator usage rates to support supply-and-demand planning efforts

Considerations for recommending extended use and limited reuse of filtering facepiece respirators in health care settings

National Institute for Occupational Safety and Health. Recommended guidance for extended use and limited reuse of N95 filtering facepiece respirators in healthcare settings. Page last reviewed

PPE CASE-Personal Protective Equipment Conformity Assessment Studies and Evaluations. Inhalation and Exhalation Resistance and Filtration Performance of Stockpiled Air-Purifying Respirators: Overall Performance of Nearly

US Centers for Disease Control and Prevention

Can N95 respirators be reused after disinfection? How many times?

Reusable Elastomeric Respirators in Health Care: Considerations for Routine and Surge Use

Strategies for optimizing the supply of N95 respirators

Personal protective equipment EUAs

Surgical mask and gown conservation strategies -letter to healthcare providers

Strategies for optimizing the supply of N95 filtering facepiece respirators during the coronavirus disease 2019 (COVID-19) pandemic. Disaster Med Public Health Prep

Enforcement guidance for respiratory protection and the N95 shortage due to the coronavirus disease 2019 (COVID-19) pandemic. Enforcement memo, interim guidance

Umbrella EUA for non-NIOSH-approved disposable filtering facepiece respirators (FFRs) manufactured in China

National Institute for Occupational Safety and Health (NIOSH). Determination of Particulate Filter Efficiency Level for N95 Series Filters Against Solid Particulates for Non-Powered, Air-Purifying Respirators Standard Test Procedure (STP)

Respiratory Protective Devices. 42 CFR §84

National Institute for Occupational Safety and Health NPPTL respiratory assessments to support the COVID-19 response

Gregory Rutledge on initial testing of KN95 respirators for public health officials. MIT News

Assessing the quality of nontraditional N95 filtering face-piece respirators available during the COVID-19 pandemic

Low-quality masks infiltrate U.S. coronavirus supply

Assessment of Filter Penetration Performance for Non-NIOSH Approved Respirators: NPPTL Assessment to Support the COVID-19 Response

The authors thank the following people for their input, support, and expertise in this work: Matt Duling, Brenda Boutin, Megan Casey, and Jim Harris; all from the National Personal Protective Technology Laboratory, National Institute for Occupational Safety and Health, CDC.

The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of NIOSH or CDC. Product and company names are provided for identification purposes only and do not imply endorsement by the CDC.