5

Looking to the Future: Multiplex PCR and Next Generation Sequencing

The changing technology landscape will affect the BioWatch program. This chapter comments on additional assay performance considerations that arise from the use of multiplexed real-time polymerase chain reaction (PCR) assays and potential implications for the program of continued advances in genetic sequencing technology.

MULTIPLEXING PCR ASSAYS FOR USE IN BIOWATCH

It is possible to “multiplex” a nucleic acid–based detection assay in various ways. Several examples of possible approaches include

• Parallel readout of multiple, singleplex reactions (not discussed further);
• Multiplex PCR using multiple, specific primer sets in a single reaction (see Box 5-1);
• Multiplex PCR using conserved primers in a single reaction to amplify a large number of related species for subsequent analysis (e.g., using primers directed toward 16S ribosomal DNA, which is capable of amplifying a wide range of bacteria, followed by sequencing of the amplification products to identify the organisms); • PCR amplification using random, universal or whole genome amplification followed by another, high-throughput method of analysis and readout, such as via microarrays; and
• PCR amplicon, targeted probe enrichment, or metagenomic sequencing of various types.

With several potential technologies, the signal obtained can suffer from nonspecific hybridization (e.g., microarrays), nonspecific protein signals (e.g., protein mass spectrometry), or limited resolution provided by information on only the relative percentages of A,C,G,T DNA nucleotides (e.g., nucleotide mass spectrometry). PCR with conserved primers is most useful for cases where it is desirable to detect a wide range of targets with broad sensitivity (e.g., the use of primers to detect 16S or rpoB sequences in bacteria [Case et al. 2007]). This method may be less useful for viruses, which are highly diverse, or specific organism identification to the species or strain level required by BioWatch for most targets. Sequencing can be prone to multiple potential sources of errors, some systematic (e.g., runs of homopolymers) and some random, and each sequencing technology has its own error model. Recent reports have examined potential alternative technologies for BioWatch detection (IOM and NRC 2014), and these are not explored in detail here.

The type of multiplex PCR technology that the committee focuses on in the report is the inclusion of multiple sets of target-specific primers in a single reaction (the second item in the list above and described in Box 5-1). All of the real-time PCR assays currently used in BioWatch are performed as singleplex assays, in which a single target DNA signature is amplified for each assay. The secondary assay for any of the tested BioWatch agents, which looks for the presence of three or more signatures, also is run in a singleplex manner, meaning that the “secondary assay” consists of three or more individual singleplex assays. However, if BioWatch assays were combined in a multiplex fashion, multiple targets could be tested at one time, potentially saving time and labor but with a potential loss of detection sensitivity. Multiplexing BioWatch assays has potential advantages and disadvantages for the program, some of which are listed below.

Characteristics of Singleplex PCR:

• Has good sensitivity and specificity as a result of carefully designed primer and probe sets. However, assay performance is dependent on availability of nucleotide information for primer design.
• Narrowly focuses on only one region of the genome. However, it may not provide sufficient discrimination between a biothreat agent and nonpathogenic flora because only a fragment of the genome is detected.
• Relies on conservation of primer binding sites used for amplification. Sequence specificity is required for primer and probe hybridization to the target. However, viruses have high diversity, which must be considered in designing primers and probes that

• can detect the virus or that may need to anneal to a primer binding region that can exhibit variability. In bacteria, the potential dropout of targets on plasmids or in certain genes (such as the antibiotic resistance gene mecA in Staphylococcus aureus) may need to be addressed. As a result, the use of more than one target in an initial screen or in a secondary assay is beneficial.

• For serial testing, uses up a lot of sample, but there is potentially less input per target assay.
• Typically provides limited information on strain genotype and on additional pathogen characteristics such as antibiotic resistance.

Characteristics of Multiplex PCR:

• Targeted PCR has the same types of limitations as singleplex PCR (conservation of primer binding sites, narrow focus on specific regions of genome).
• “Random” PCR methods amplify everything in the sample ideally without biased representation of the targets. As a result, these methods can detect a broad range of targets, although a method to further type the targets is required.
• Multiplex methods must deal with interactions between multiple sets of primers and probes that may have different annealing temperatures and different performance in the assay, and that may cross-react and generate primer dimers, for example. An “all-in-one” approach may not be useful in the detection of many agents with large variations in diversity, genome sizes, AT/GC ratios, or other parameters that affect assay performance.
• Selection and use of proper controls can be challenging where many targets are being detected simultaneously.
• Rather than designing one multiplex to cover all desired targets, it is also possible to design several multiplex assays that each covers a set of targets whose performance under PCR conditions can be better optimized; such multiplex assays can be used in combination.
• Detection of a multiplex PCR typically requires more sophisticated instrumentation than a singleplex, because multiple colors need to be detected when using fluorescent probes (see Box 5-1). The excitation and emission spectra start to overlap when more than six or so probes are used in a reaction. Distinguishing multiple distinct colors can be a challenge to sensitivity and specificity of the assay. Options include use of hybridization to beads, or splitting up the assays into a combinatorial approach into many parallel reaction tubes for real-time PCR.

For nucleic acid–based multiplex options, such as the multiplex PCR focused on by the committee, the core technology includes a PCR amplification step. Thus, the considerations that guide the development of PCR assay performance standards for singleplex real-time PCR analysis, the focus of the rest of the report, remain relevant to the development of appropriate standards for multiplex assays. Multiplex assays will need to be characterized and validated for the limit of detection (LOD) of each target in the multiplex, and assay specificities will need to be determined by testing against appropriate inclusivity, exclusivity, and environmental panels. However, several additional considerations arise in the development and validation of multiplex PCR assays.

Just as in singleplex assays but on a larger scale, the inclusion of multiple pairs of primers and multiple probes into a single reaction mix can result in primer–primer interactions that interfere with target amplification or lead to the production of nonspecific amplification products and reduce assay sensitivity. The composition of DNA sequences also affects optimal thermal cycling conditions, and optimizing the overall reaction conditions becomes more difficult with an increased number of DNA targets. For example, some microorganisms have GC-rich genomes, and GC base pairs in DNA generally require a higher melting temperature because of the greater number of hydrogen bonds compared with AT base pairs. In a multiplex assay, reaction conditions need to be suitable for targets that may vary in composition. The nature of the target DNA and reaction conditions also can result in one target sequence preferentially amplifying over others; strategies such as reducing (or increasing) concentration of certain primers may be needed to enable sufficient amplification of all desired sequences. Development of a multiplex assay becomes a fairly complex matrix of variables, including cross-reactivity, the potential for dye quenching and spectral overlap, and the incorporation of appropriate positive and negative control references, and conditions will likely need to be adjusted and checked as probes and primers are added to the multiplex. In the case of hybridization-based detection assays with primers and probes on beads, the capture sequences must also be verified and adjusted as needed. As a result, it has been suggested that it may be easier to start from scratch in the design of a multiplex assay rather than to adapt several existing singleplex targets, primers, and probes to be run together in a multiplex manner (Naraghi-Arani 2014). This approach may be hampered if there is only one known signature to detect and discriminate a target. Despite these challenges, assay developers have successfully designed PCR assays that multiplex large numbers of targets. The BioWatch program indicated to the committee a particular interest in multiplexing the routine initial screening assays to save time and labor. Because the screening assays identify one signature per pathogen, the

required BioWatch multiplex assay for this purpose would include fewer than 10 signatures in the sample.

Multiplex assays are becoming increasingly common and assay developers and regulators have substantial experience with optimizing and evaluating them. One of the first Food and Drug Administration (FDA)-approved multiplex assays was the Luminex xTAG Respiratory Viral Panel, which tested for 12 viruses and was approved for marketing in 2008 (FDA 2008). The field has continued to expand, and FDA now has guidance available on performance characterization for highly multiplexed nucleic acid–based diagnostics designed to detect 20 or more targets per multiplex (FDA 2014a). As noted, many core aspects of performance characterization and validation are similar for singleplex and multiplex assay development, but additional guidance provided by the FDA for highly multiplex reactions includes

• Demonstrating through in silico analysis and validation data that negative interactions among assay components such as primers, probes, and amplified targets are not interfering with performance.
• Providing information not only on the individual LODs for each target in the assay, but also on the assay cutoff values for each target and how these values were determined and validated.
• Using representative panels of organisms targeted by the multiplex assay to demonstrate system repeatability and reproducibility, in order to reduce the overall size of the required studies. Similarly, one or more representative targets of the multiplex assay can be used to demonstrate that carryover and cross-contamination do not occur in the device between samples, rather than testing all targets.

Finally, the current FDA guidance indicates that a change to an approved multiplexed assay, such as the addition of a new target, requires the manufacturer to submit a 510(k) premarket notification submission because it represents a new intended use of the assay. However, the guidance acknowledges that only a subset of performance studies would be required because much of the assay’s previously reviewed performance is expected to remain similar. New data to be submitted would focus on the additional target (characterization of LOD and inclusivity and exclusivity detection, for example). Tailored studies on a representative subset of the existing targets also would be conducted, such as a limited series bracketing the LOD to confirm that it is performing consistent with the original assay version (FDA 2014a). FDA is concerned with clinical use of devices and reportedly considers the impact of assay changes on the potential false positive and false negative rates as they would affect making appropriate patient treatment decisions; thus performance data required by the

FDA can depend on the tested pathogen. The overall goal, however, is to ask developers to provide information necessary to evaluate performance without putting an undue burden on them (Hobson 2014). Standards approaches such as the Stakeholder Panel on Agent Detection Assays, Public Health Actionable Assays, and Federal Standards for Assay Performance and Equivalency also recognize that assays may be developed in a multiplex fashion; the documents that describe validation under these approaches generally indicate that multiplex validation studies should be conducted so as to test the individual PCR targets together in a single sample. These approaches in concert with the FDA guidance documents should provide a suitable basis for validation testing of multiplexed PCR assays for BioWatch.

The BioWatch program utilized actionable multiplex assays from late 2007 before withdrawing them from use in mid-2009. The proposed acquisition of Gen-3 autonomous detection systems (canceled in 2013) also was envisioned as operating with multiplex assays. When multiplex PCR assays were field-tested by the BioWatch program, concerns were expressed over loss of assay sensitivity (U.S. House of Representatives 2012b,c). The evaluation of BioWatch Gen-3 vendor technology undertaken by Los Alamos National Laboratory (LANL) used the multiplex PCR assays developed by the vendors. In the experience of LANL scientists, PCR assays run in a multiplex fashion did show reduced sensitivity compared with singleplex assays, but the effect was reportedly closer to a factor of 2 than to an order of magnitude. For example, if the LOD with an appropriate probability of detection (e.g., 95 percent) of a target in a singleplex assay was 20 copies per reaction, the LOD of the target in multiplex might be closer to 40-50 copies per reaction, a difference that falls within the potential range of experimental error (Kristin Omberg, LANL, personal communication, October 24, 2014). Given the relatively low numbers of copies detectable by the PCR assays, the statistical error bars that surround copy number quantification at low levels, and the uncertainty in moving from analytical characterization of an assay using purified nucleic acid to the real world use in the system with DNA extracted from aerosol filter samples, it is unclear whether this degree of difference in assay performance would have a significant operational effect.

As with other aspects that the committee examined, the primary challenge to the use of multiplexed assays by BioWatch appears to be a communication issue more than a technical one. It appears likely that suitable multiplex assays can be designed for use in BioWatch, particularly to replace the routine screening assays. However, data on multiplex assay performance and validation, compared with the existing singleplex assays as references, need to be shared with the laboratory experts in each jurisdiction in order to give them a basis from which to use the tests. The

committee’s understanding is that most jurisdictions did not have access to such data during the limited prior deployment of multiplex assays by the program, which contributed to uncertainly around use of the assays. Without providing performance data to relevant jurisdictional experts, discussing differences such as possible, but likely limited, reductions in sensitivity, and explicitly considering how any performance reductions would be likely to translate to system operational performance, it is not surprising that resistance to use of the assays occurred.

TAKING ACCOUNT OF STEP CHANGES IN TECHNOLOGY

The BioWatch program currently operates under several assumptions and limitations. These include the use of a two-stage screening and secondary assay process based on real-time PCR of a nominal number of signature sequences, which places the emphasis on making these PCR assays as reliable and robust (as sensitive and as specific) as possible. These assumptions are not necessarily unreasonable and the current system is functioning. Real-time PCR as the detection methodology makes sense for BioWatch’s mission, although the committee has suggested that in certain cases additional rule-out assays could be performed.

Technology in the life sciences continues to advance rapidly, driven largely by medical applications. As BioWatch looks to the future, one particular technology to watch is next-generation sequencing (NGS). The advantage of NGS technology is that it can provide more rapid, less expensive, and higher-throughput sequence data from a sample. NGS protocols in current use still largely depend on amplification of the sample materials prior to sequencing, although amplification-free methods such as nanopore-based single molecule sequencing are being developed (Quick et al. 2014). The implementation of NGS technology by metagenomics (random shotgun) or whole-genome sequencing has the potential to provide more precise information on genotype or strain and opens the possibility of detecting engineered or novel biothreat agents with high sensitivity and specificity. For example, genetic profiles associated with antibiotic resistance or with virulence may be identifiable. In this way, the technology would enable BioWatch to expand beyond its current portfolio of defined inclusivity strains for a small number of pathogen species. In clinical diagnostic use, these types of sequencing analyses to identify the root cause of hard-to-identify infections are starting to be successfully employed (Wilson et al. 2014).

However, it also is important to be realistic about the challenges associated with this technology and how it might or might not be appropriate for BioWatch in the near term. The technology will take time to mature. For BioWatch, in particular, samples are extracted from a complex envi-

ronmental background on collected air filters. Accurate taxonomic and strain identification (requiring sufficient sequencing coverage and sufficient length of assembled sequence reads) and separation of sequences of interest from the enormous unknown background would be both critical and computationally challenging. In fact, a useful assembly from unbiased metagenomic sequencing of BioWatch filter samples may not be possible. The curated databases and informatics required to undertake the necessary sequence assembly or mapping at this scale are developing rapidly but can still be considered immature. Furthermore, the BioWatch program is designed to be an early warning system; thus information needs to be obtainable with a relatively rapid turnaround time. Because much of the technology and informatics required are still under development and are not routine, the required performance validation (involving both the assay and the informatics pipeline) also would be complex.

How Sequencing Can Be Useful Now

One key way in which sequencing currently can be incorporated into BioWatch is as a tertiary, follow-up analysis method for cases in which jurisdictions experience assay-positive results that do not appear to be true biothreat detections. In particular, sequencing should be undertaken where jurisdictions experience repeated apparent detections of certain pathogens, in an effort to truly understand what is causing the results. The committee understands that sequencing by Centers for Disease Control and Prevention (CDC) laboratories from collected filters has been attempted in at least some such cases, but that an answer was not always obtainable, or the details may not have always been conveyed effectively back to the jurisdictional laboratories and officials. Metagenomic sequencing could also be used now as part of efforts to better understand background genomic diversity on BioWatch filters from various collectors in various jurisdictions and at various time points. As discussed in Chapter 3, certain microbial strains may produce positive assay detections as a result of time, weather, or location-specific environmental sources and improving the ability to predict these circumstances would aid users in interpreting results. The information obtained through sufficiently deep metagenomic sequencing would also serve as an especially valuable resource for the in silico design of assay target sequences, primers, and probes, which currently rely on available databases.

How Sequencing Can Be Useful in the Future

The committee agrees with the findings of a 2013 BioWatch workshop that NGS is not yet sufficiently mature to be considered as a replacement

technology for the current PCR-based BioWatch operations (NRC and IOM 2014). However, the committee believes that NGS could play valuable roles for BioWatch in the near and far term.

Sequencing as Confirmation of a PCR Screening Positive

The primary issue is whether it is feasible to utilize metagenomic (“random” shotgun) NGS on a portion of a BioWatch filter to provide confirmation of a PCR screening positive result and to provide additional characterization of the material present in the sample. In this way, NGS could essentially serve as a replacement for the current real-time PCR secondary assays. Multiple factors are involved in calculating the feasibility of this approach, including

1. The amount of DNA that can be extracted from a BioWatch filter quadrant, because different NGS technologies require different starting amounts;
2. Whether deposition on the filter is homogeneous (an issue for the current assays as well);
3. Whether random amplification of the DNA is required to obtain sufficient starting material, which may introduce biases because some DNA regions may be amplified more efficiently than other regions;
4. The likely unknown extent of the genomic background on the filter and how this background relates to the genomic material of the pathogen(s) indicated as present by PCR;
5. The sequencing depth of the NGS technology used, in particular how many random reads and what read length can be obtained from complex mixtures;
6. The amount of pathogen present that NGS will need to be able to detect (e.g., the LOD for the NGS);
7. The cost of the NGS run;
8. The sample-in to sequence-out time;
9. The time needed to analyze the sequence data, which ranges from hours to days depending on the bioinformatics algorithms used and additional factors; and
10. Technology transfer requirements, especially bioinformatics.

Items 3, 4, and 5 on the list define the tradeoff between pathogen LOD and the sequencing depth needed. Because of the enormous genomic background present on aerosol filters, standard desktop NGS on an instrument such as the Illumina MiSeq (which currently produces approximately 15-24 million reads of up to ~250 base pairs in length) may

support a LOD that is too many orders of magnitude worse than PCR for it to be useful for confirmation and further characterization. Deeper sequencing may take too long a time or may be too costly. Thus, while NGS is not feasible for routine BioWatch characterization currently, it may be acceptable as confirmatory follow-up to near-BAR events as discussed above. In the future, continued technology and informatics developments may make NGS technology suitable as a replacement for the BioWatch secondary assay.

A Note About Targeted Versus Metagenomic NGS

The above discussion assumed that metagenomic sequencing was performed on nucleic acids extracted from BioWatch filters. Commercial technologies exist that permit targeted sequencing, which can be accomplished via targeted-PCR methods (e.g., pools of PCR primers that amplify genomic regions that are highly discriminating) or via bait-capture methods (e.g., array probes that capture highly discriminating regions, which can be released for library preparation and sequencing once the background is washed away). Targeted sequencing is an option that BioWatch could utilize to look for a specific set of agent targets. Hundreds of regions could be targeted, to provide confident species and strain identification, as well as to identify key genes related to virulence or resistance. The use of a targeted amplification approach could hasten the potential applicability of NGS for BioWatch for confirmation and characterization of a screening PCR result. Deep metagenomic sequencing would still need to be employed in order to detect novel or engineered organisms.

Sequencing as a Replacement for PCR for BioWatch

Given the factors listed above, the committee also asked what requirements would be needed for sequencing to be feasible as a full replacement for PCR for BioWatch. Clearly, the issues of sequencing depth and LOD, total time-to-answer, cost, software to facilitate data analysis, personnel training in NGS, and required information technology and laboratory infrastructure would dominate the calculation. Necessary requirements for BioWatch might include, but not be limited to

• Achieving a pathogen LOD at a level equivalent to current PCR assays;
• Delivering an answer in < 24 hours from sample receipt;
• Costing no more than $1,000/sample (or even a much lower cost
• that might be roughly comparable to PCR assay costs); and
• Being able to process in batch approximately 36 samples/day (or

To our knowledge, no existing or announced sequencing technology is able to meet these specifications yet. The DNA sequencing industry currently is driven by the need to lower the cost of sequencing a pure human genome or exome, and the industry has succeeded tremendously in doing so. However, BioWatch faces the need to confidently identify a pathogen at a variable and unknown concentration against a background that can also vary enormously over time (e.g., on high pollen versus low pollen days). This challenge is very different, especially when the required time, cost, and throughput are taken into consideration. Although the technology continues to improve, it is not clear when an appropriate sequencing platform(s) with the necessary performance characteristics will become available. However, the deep analysis of microbiomes, such as those found in the human gut, by NGS is drawing increasing attention from multiple agencies because of the potential health benefits that may be derived. Microbiome analysis may serve as a market driver to produce sequencing platforms optimized in ways that BioWatch can ultimately find useful and leverage. The FDA, CDC, state public health laboratories, and other partners also are exploring the role of high-throughput pathogen genome sequencing using NGS to identify microorganisms in food and environmental samples and compare them with clinical samples from ill patients, in order to identify causative pathogens in outbreaks of foodborne disease. Indeed, approximately 500 isolates of bacteria such as Salmonella spp. and Listeria spp. are being sequenced each month.¹ These whole-genome sequencing approaches using NGS can also be applied by BioWatch in the future to recover additional genomes of biothreat agents and thereby establish more complete reference databases.

Conclusion About Sequencing Technology for BioWatch

Table 5-1 summarizes ways that three types of next-generation sequencing could be applicable to the BioWatch program, along with current challenges to implementation. In addition to technical challenges, the experience, training and infrastructure within jurisdictional laboratories conducting BioWatch assays will need to be considered.

Ultimately, the committee recommends that the BioWatch program should continue to monitor and evaluate NGS technologies as they develop. Sequencing (through NGS or older Sanger sequencing technol-

___________________

¹ Information on the FDA’s Whole Genome Sequencing Program is available at http://www.fda.gov/Food/FoodScienceResearch/WholeGenomeSequencingProgramWGS/.

TABLE 5-1 Applications of Next-Generation Sequencing for BioWatch

ogy) currently could be used to follow up on unexpected assay results from the jurisdictions. In the nearer term, targeted approaches coupled with NGS may be useful to the program as a replacement for the current real-time PCR secondary assays because of their ability to analyze many more genomic regions for identification and characterization. The applicability of metagenomic NGS to the program would be a longer-term prospect.