Journal:Comparative performance of SARS-CoV-2 lateral flow antigen tests and association with detection of infectious virus in clinical specimens: A single-centre laboratory evaluation study

Full article title	Comparative performance of SARS-CoV-2 lateral flow antigen tests and association with detection of infectious virus in clinical specimens: A single-centre laboratory evaluation study
Journal	The Lancet Microbe
Author(s)	Pickering, Suzanne; Batra, Rahul; Merrick, Blair; Snell, Luke B.; Nebbia, Gaia; Douthwaite, Sam; Reid, Fiona; Reid, Fiona; Patel, Amita; Kia Ik, Mark T.; Patel, Bindi; Charalampous, Themoula; Alcolea-Medina, Adela; Lista, Maria J.; Cliff, Penelope R.; Cunningham, Emma; Mullen, Jane; Doores, Katie J.; Edgeworth, Jonathan D.; Malim, Michael H.; Neil, Stuart J.D.; Galão, Rui P.
Author affiliation(s)	King's College London, Guy's and St Thomas' NHS Foundation Trust
Primary contact	Email: rui_pedro dot galao at kcl dot ac dot uk
Year published	2021
Volume and issue	2(9)
Page(s)	e461-e471
DOI	10.1016/S2666-5247(21)00143-9
ISSN	2666-5247
Distribution license	Creative Commons Attribution 4.0 International
Website	https://www.sciencedirect.com/science/article/pii/S2666524721001439
Download	https://www.sciencedirect.com/science/article/pii/S2666524721001439/pdfft (PDF)

Abstract

Background: Lateral flow devices (LFDs) for rapid antigen testing are set to become a cornerstone of SARS-CoV-2 mass community testing, although their reduced sensitivity compared with polymerase chain reaction (PCR) methods has raised questions of how well they identify infectious cases. Understanding their capabilities and limitations is, therefore, essential for successful implementation. We evaluated six commercial LFDs and assessed their correlation with infectious virus culture and PCR cycle threshold (Ct) values.

Methods: In a single-center, laboratory evaluation study, we did a head-to-head comparison of six LFDs commercially available in the U.K.: Innova Rapid SARS-CoV-2 Antigen Test, Spring Healthcare SARS-CoV-2 Antigen Rapid Test Cassette, E25Bio Rapid Diagnostic Test, Encode SARS-CoV-2 Antigen Rapid Test Device, SureScreen COVID-19 Rapid Antigen Test Cassette, and SureScreen COVID-19 Rapid Fluorescence Antigen Test. We estimated the specificities and sensitivities of the LFDs using stored nasopharyngeal swabs collected at St. Thomas' Hospital (London, U.K.) for routine diagnostic SARS-CoV-2 testing by real-time RT-PCR (qRT-PCR). Swabs were from inpatients and outpatients from all departments of St. Thomas' Hospital, and from healthcare staff (all departments) and their household contacts. SARS-CoV-2-negative swabs from the same population (confirmed by qRT-PCR) were used for comparative specificity determinations. All samples were collected between March 23 and Oct 27, 2020. We determined the limit of detection (LOD) for each test using viral plaque-forming units (PFUs) and viral RNA copy numbers of laboratory-grown SARS-CoV-2. Additionally, LFDs were selected to assess the correlation of antigen test result with qRT-PCR Ct values and positive viral culture in Vero E6 cells. This analysis included longitudinal swabs from five infected inpatients with varying disease severities. Furthermore, the sensitivities of available LFDs were assessed in swabs (n = 23; collected from Dec 4, 2020, to Jan 12, 2021) confirmed to be positive (qRT-PCR and whole genome sequencing) for the B.1.1.7 variant, which was the dominant genotype in the U.K. at the time of study completion.

Findings: All LFDs showed high specificity (≥98.0%), except for the E25Bio test (86.0% [95% CI 77.9–99.9]), and most tests reliably detected 50 PFU/test (equivalent SARS-CoV-2 N gene Ct value of 23.7, or RNA copy number of 3 × 106/mL). Sensitivities of the LFDs on clinical samples ranged from 65.0% (55.2–73.6) to 89.0% (81.4–93.8). These sensitivities increased to greater than 90% for samples with Ct values of lower than 25 for all tests except the SureScreen fluorescence (SureScreen-F) test. Positive virus culture was identified in 57 (40.4%) of 141 samples; 54 (94.7%) of the positive cultures were from swabs with Ct values lower than 25. Among the three LFDs selected for detailed comparisons (the tests with highest sensitivity [Innova], highest specificity [Encode], and alternative technology [SureScreen-F]), sensitivity of the LFDs increased to at least 94.7% when only including samples with detected viral growth. Longitudinal studies of qRT-PCR-positive samples (tested with Innova, Encode, and both SureScreen-F and the SureScreen visual [SureScreen-V] test) showed that most of the tests identified all infectious samples as positive. Test performance (assessed for Innova and SureScreen-V) was not affected when reassessed on swabs positive for the U.K. variant B.1.1.7.

Interpretation: In this comprehensive comparison of antigen LFDs and virus infectivity, we found a clear relationship between Ct values, quantitative culture of infectious virus, and antigen LFD positivity in clinical samples. Our data support regular testing of target groups with LFDs to supplement the current PCR testing capacity, which would help to rapidly identify infected individuals in situations in which they would otherwise go undetected.

Research in context

Evidence before this study

We searched PubMed on April 22, 2021, with no date or language restrictions, using the terms (“SARS-CoV-2” OR “COVID-19”) AND (“antigen”) AND (“infectivity” OR “virus isolation”). Our search revealed 28 research publications, among which only two specifically addressed the characterization of rapid antigen tests in the context of a correlation between their performance and sample infectivity in vitro. As evidence of a rapidly moving field, the same search in medRxiv identified several manuscripts showing either evaluations of different Lateral flow devices (LFDs) for rapid antigen testing according to reverse transcription polymerase chain reaction (RT-PCR) cycle threshold (Ct) values, or relationships between Ct values and virus infectivity. A common limitation of these papers was the scarcity of clear evidence of a relationship between antigen test positivity and the existence of infectious virus in the same clinical specimen. In addition, none of the studies reassessed the performance of rapid antigen tests in the context of longitudinal panels, or against the variant that was becoming dominant in the U.K. at the time of the study, B.1.1.7.

Added value of this study

This study is, to our knowledge, the largest to date assessing the correlation between Ct values, quantitative culture of infectious virus, and antigen test positivity, alongside an unbiased head-to-head comparison of six commercial antigen tests. We found that most rapid antigen tests performed to a high standard in clinical samples. Among three LFDs selected for detailed comparisons, we found a sensitivity of at least 94.7% when compared with samples that were infectious in vitro, with absolute viral titer in the specimens correlating with Ct values. Longitudinal studies of real-time RT-PCR (qRT-PCR) -positive samples provided evidence that differences in test sensitivities can lead to missed cases in the absence of repeated testing, which is particularly relevant in the context of asymptomatic or presymptomatic individuals. We also showed that despite amino acid changes in the SARS-CoV-2 nucleocapsid antigen, detection of the B.1.1.7 variant by selected LFD tests was not affected. This study provides clear evidence of the relationship between Ct values, cultivable virus, and antigen LFD positivity, with tests delivering reliable identification of infectious clinical samples.

Implications of all the available evidence

In a time when LFDs for rapid antigen testing are expected to have a major role in SARS-CoV-2 mass community and healthcare testing, we believe that this study will inform the ongoing debate about how these tests should be deployed. Our data support regular testing of target groups with LFDs, not as standalone one-off tests, but rather to supplement current polymerase chain reaction (PCR) testing capacity, and thus rapidly identify infectious individuals in situations in which they would otherwise go undetected.

Supplementary material

• Supplementary appendix (PDF)

Acknowledgements

We are thankful to Spring Healthcare, SureScreen Diagnostics, Zhuhai Encode Medical Engineering, Innova Medical Group, and E25Bio for donating test kits. We are also thankful to the Biostatistics and Data Management Platform of the National Institute for Health Research (NIHR) Guy's and St. Thomas' Biomedical Research Centre for providing guidance on statistical analyses. We are extremely grateful to all patients and staff at St. Thomas' Hospital who participated in this study. This research was supported by the U.K. Department of Health via an NIHR Comprehensive Biomedical Research Centre award to Guy's and St. Thomas' NHS Foundation Trust in partnership with King's College London and King's College Hospital NHS Foundation Trust. SP was supported by a Huo Family Foundation award and Wellcome Trust Senior Fellowship (WT098049AIA) granted to SJDN. MHM, KJD, SJDN, and RPG were supported by King's Together Rapid COVID-19 call awards. MHM was supported by a Wellcome Trust award (106223/Z/14/Z). SJDN, KJD, and MHM were supported by a UK Medical Research Council Discovery award (MC/PC/15068). MHM, KJD, and SJDN were supported by a Huo Family Foundation award.

Contributions

SP, RB, LBS, BM, GN, SD, MHM, JDE, SJDN, and RPG conceived and designed the study. SP, RB, BM, LBS, MTKI, TC, AA-M, and RPG collated the data. LBS, BM, TC, AA-M, GN, and SD supervised specimen collection. AP, BP, TC, PRC, EC, and JM supervised data collection. MJL and KJD generated reagents. SP, RPG, and FR analysed the data. SP, SJDN, and RPG wrote the first draft of the manuscript. All authors had full access to all the data in the study and had final responsibility for the decision to submit for publication. SP and RPG accessed and verified the data. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Funding

King's Together Rapid COVID-19, Medical Research Council, Wellcome Trust, Huo Family Foundation, UK Department of Health, National Institute for Health Research Comprehensive Biomedical Research Centre.

Data sharing

All relevant data are within the manuscript and supplementary appendix.

Competing interests

We declare no competing interests.

Notes

This presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added.