View source for Journal:Comparative performance of SARS-CoV-2 lateral flow antigen tests and association with detection of infectious virus in clinical specimens: A single-centre laboratory evaluation study

{{Infobox journal article
|name         = 
|image        = 
|alt          = <!-- Alternative text for images -->
|caption      = 
|title_full   = Comparative performance of SARS-CoV-2 lateral flow antigen tests and association with detection of infectious virus in clinical specimens: A single-centre laboratory evaluation study
|journal      = ''The Lancet Microbe''
|authors      = Pickering, Suzanne; Batra, Rahul; Merrick, Blair; Snell, Luke B.; Nebbia, Gaia; Douthwaite, Sam; Reid, Fiona; Reid, Fiona; Patel, Amita; Kia Ik, Mark T.; Patel, Bindi; Charalampous, Themoula; Alcolea-Medina, Adela; Lista, Maria J.; Cliff, Penelope R.; Cunningham, Emma; Mullen, Jane; Doores, Katie J.; Edgeworth, Jonathan D.; Malim, Michael H.; Neil, Stuart J.D.; Galão, Rui P.
|affiliations = King's College London, Guy's and St Thomas' NHS Foundation Trust
|contact      = Email: rui_pedro dot galao at kcl dot ac dot uk
|editors      = 
|pub_year     = 2021
|vol_iss      = '''2'''(9)
|pages        = e461-e471
|doi          = [http://dx.doi.org/10.1016/S2666-5247(21)00143-9 10.1016/S2666-5247(21)00143-9]
|issn         = 2666-5247
|license      = [http://creativecommons.org/licenses/by/4.0/ Creative Commons Attribution 4.0 International]
|website      = [https://www.sciencedirect.com/science/article/pii/S2666524721001439 https://www.sciencedirect.com/science/article/pii/S2666524721001439]
|download     = [https://www.sciencedirect.com/science/article/pii/S2666524721001439/pdfft?md5=1cf2dd534de88542bc9eaf37d73aeed9&pid=1-s2.0-S2666524721001439-main.pdf https://www.sciencedirect.com/science/article/pii/S2666524721001439/pdfft] (PDF)
}}

==Abstract==
'''Background''': [[Lateral flow test|Lateral flow devices]] (LFDs) for rapid antigen testing are set to become a cornerstone of [[SARS-CoV-2]] mass community testing, although their reduced sensitivity compared with [[polymerase chain reaction]] (PCR) methods has raised questions of how well they identify infectious cases. Understanding their capabilities and limitations is, therefore, essential for successful implementation. We evaluated six commercial LFDs and assessed their correlation with infectious virus culture and PCR cycle threshold (Ct) values.

'''Methods''': In a single-center, [[laboratory]] evaluation study, we did a head-to-head comparison of six LFDs commercially available in the U.K.: Innova Rapid SARS-CoV-2 Antigen Test, Spring Healthcare SARS-CoV-2 Antigen Rapid Test Cassette, E25Bio Rapid Diagnostic Test, Encode SARS-CoV-2 Antigen Rapid Test Device, SureScreen COVID-19 Rapid Antigen Test Cassette, and SureScreen COVID-19 Rapid Fluorescence Antigen Test. We estimated the specificities and sensitivities of the LFDs using stored [[nasopharyngeal swab]]s collected at St. Thomas' Hospital (London, U.K.) for routine diagnostic SARS-CoV-2 testing by [[Reverse transcription polymerase chain reaction|real-time RT-PCR]] (qRT-PCR). Swabs were from inpatients and outpatients from all departments of St. Thomas' Hospital, and from healthcare staff (all departments) and their household contacts. SARS-CoV-2-negative swabs from the same population (confirmed by qRT-PCR) were used for comparative specificity determinations. All [[Sample (material)|samples]] were collected between March 23 and Oct 27, 2020. We determined the limit of detection (LOD) for each test using viral plaque-forming units (PFUs) and viral RNA copy numbers of laboratory-grown SARS-CoV-2. Additionally, LFDs were selected to assess the correlation of antigen test result with qRT-PCR Ct values and positive viral culture in Vero E6 cells. This analysis included longitudinal swabs from five infected inpatients with varying disease severities. Furthermore, the sensitivities of available LFDs were assessed in swabs (''n'' = 23; collected from Dec 4, 2020, to Jan 12, 2021) confirmed to be positive (qRT-PCR and [[whole genome sequencing]]) for the B.1.1.7 variant, which was the dominant genotype in the U.K. at the time of study completion.

'''Findings''': All LFDs showed high specificity (≥98.0%), except for the E25Bio test (86.0% [95% CI 77.9–99.9]), and most tests reliably detected 50 PFU/test (equivalent SARS-CoV-2 N gene Ct value of 23.7, or RNA copy number of 3 × 106/mL). Sensitivities of the LFDs on clinical samples ranged from 65.0% (55.2–73.6) to 89.0% (81.4–93.8). These sensitivities increased to greater than 90% for samples with Ct values of lower than 25 for all tests except the SureScreen fluorescence (SureScreen-F) test. Positive virus culture was identified in 57 (40.4%) of 141 samples; 54 (94.7%) of the positive cultures were from swabs with Ct values lower than 25. Among the three LFDs selected for detailed comparisons (the tests with highest sensitivity [Innova], highest specificity [Encode], and alternative technology [SureScreen-F]), sensitivity of the LFDs increased to at least 94.7% when only including samples with detected viral growth. Longitudinal studies of qRT-PCR-positive samples (tested with Innova, Encode, and both SureScreen-F and the SureScreen visual [SureScreen-V] test) showed that most of the tests identified all infectious samples as positive. Test performance (assessed for Innova and SureScreen-V) was not affected when reassessed on swabs positive for the U.K. variant B.1.1.7.

'''Interpretation''': In this comprehensive comparison of antigen LFDs and virus infectivity, we found a clear relationship between Ct values, quantitative culture of infectious virus, and antigen LFD positivity in clinical samples. Our data support regular testing of target groups with LFDs to supplement the current PCR testing capacity, which would help to rapidly identify infected individuals in situations in which they would otherwise go undetected.

==Research in context==
===Evidence before this study===
We searched PubMed on April 22, 2021, with no date or language restrictions, using the terms (“[[SARS-CoV-2]]” OR “[[COVID-19]]”) AND (“antigen”) AND (“infectivity” OR “virus isolation”). Our search revealed 28 research publications, among which only two specifically addressed the characterization of rapid antigen tests in the context of a correlation between their performance and sample infectivity ''in vitro''. As evidence of a rapidly moving field, the same search in ''medRxiv'' identified several manuscripts showing either evaluations of different [[lateral flow test|Lateral flow devices]] (LFDs) for rapid antigen testing according to [[reverse transcription polymerase chain reaction]] (RT-PCR) cycle threshold (Ct) values, or relationships between Ct values and virus infectivity. A common limitation of these papers was the scarcity of clear evidence of a relationship between antigen test positivity and the existence of infectious virus in the same clinical specimen. In addition, none of the studies reassessed the performance of rapid antigen tests in the context of longitudinal panels, or against the variant that was becoming dominant in the U.K. at the time of the study, B.1.1.7.

===Added value of this study===
This study is, to our knowledge, the largest to date assessing the correlation between Ct values, quantitative culture of infectious virus, and antigen test positivity, alongside an unbiased head-to-head comparison of six commercial antigen tests. We found that most rapid antigen tests performed to a high standard in clinical samples. Among three LFDs selected for detailed comparisons, we found a sensitivity of at least 94.7% when compared with samples that were infectious ''in vitro'', with absolute viral titer in the specimens correlating with Ct values. Longitudinal studies of [[Reverse transcription polymerase chain reaction|real-time RT-PCR]] (qRT-PCR) -positive samples provided evidence that differences in test sensitivities can lead to missed cases in the absence of repeated testing, which is particularly relevant in the context of asymptomatic or presymptomatic individuals. We also showed that despite amino acid changes in the SARS-CoV-2 nucleocapsid antigen, detection of the B.1.1.7 variant by selected LFD tests was not affected. This study provides clear evidence of the relationship between Ct values, cultivable virus, and antigen LFD positivity, with tests delivering reliable identification of infectious clinical samples.

===Implications of all the available evidence===
In a time when LFDs for rapid antigen testing are expected to have a major role in SARS-CoV-2 mass community and healthcare testing, we believe that this study will inform the ongoing debate about how these tests should be deployed. Our data support regular testing of target groups with LFDs, not as standalone one-off tests, but rather to supplement current [[polymerase chain reaction]] (PCR) testing capacity, and thus rapidly identify infectious individuals in situations in which they would otherwise go undetected.

==Introduction==
[[COVID-19]] continues to have a profound impact on global health, with many countries resorting to economically and socially damaging restrictions to minimize the spread of SARS-CoV-2 and protect healthcare systems from being overwhelmed. Pathways out of national lockdowns, and strategies to mitigate the need for such measures in the future, depend on the successful implementation of mass vaccination programs, effective [[contact tracing]] systems, and mass community testing. In addition to the existing PCR-based testing systems, mass community testing might take the form of targeted intensive testing in areas of increasing incidence, alongside regular routine screening in healthcare, education, workplace, and leisure settings. Realistically, the expansion of regular testing relies on an element of low-infrastructure testing or self-testing, such as that offered by LFDs for rapid antigen detection.<ref name=":12">{{Cite journal |last=Crozier |first=Alex |last2=Rajan |first2=Selina |last3=Buchan |first3=Iain |last4=McKee |first4=Martin |date=2021-02-03 |title=Put to the test: use of rapid testing technologies for covid-19 |url=https://www.bmj.com/content/372/bmj.n208 |journal=BMJ |language=en |volume=372 |pages=n208 |doi=10.1136/bmj.n208 |issn=1756-1833 |pmid=33536228}}</ref><ref name=":13">{{Cite journal |last=Pavelka |first=Martin |last2=Van-Zandvoort |first2=Kevin |last3=Abbott |first3=Sam |last4=Sherratt |first4=Katharine |last5=Majdan |first5=Marek |last6=group† |first6=CMMID COVID-19 working |last7=Analýz† |first7=Inštitút Zdravotných |last8=Jarčuška |first8=Pavol |last9=Krajčí |first9=Marek |last10=Flasche |first10=Stefan |last11=Funk |first11=Sebastian |date=2021-05-07 |title=The impact of population-wide rapid antigen testing on SARS-CoV-2 prevalence in Slovakia |url=https://www.science.org/doi/abs/10.1126/science.abf9648 |journal=Science |language=EN |doi=10.1126/science.abf9648 |pmc=PMC8139426 |pmid=33758017}}</ref>

Thoroughly understanding the advantages and limitations of LFDs is, therefore, a priority, and will help to inform decisions about settings in which these tests will have the most utility and, conversely, those in which they could be contraindicated. There are concerns about the reduced sensitivity of LFDs in comparison with PCR, and controversies have emerged over the suitability of their implementation.<ref>{{Cite web |last=Deeks, J.; Raffle, A.; Gill, M. |date=12 January 2021 |title=Covid-19: Government must urgently rethink lateral flow test roll out |work=The BMJ Opinion |url=https://blogs.bmj.com/bmj/2021/01/12/covid-19-government-must-urgently-rethink-lateral-flow-test-roll-out/ |publisher=BMJ Publishing Group Limited |accessdate=23 February 2021}}</ref><ref name=":4">{{Cite journal |last=Wise |first=Jacqui |date=2020-12-15 |title=Covid-19: Lateral flow tests miss over half of cases, Liverpool pilot data show |url=https://www.bmj.com/content/371/bmj.m4848 |journal=BMJ |language=en |volume=371 |pages=m4848 |doi=10.1136/bmj.m4848 |issn=1756-1833 |pmid=33323368}}</ref><ref name=":0">{{Cite journal |last=Guglielmi |first=Giorgia |date=2021-02-09 |title=Rapid coronavirus tests: a guide for the perplexed |url=https://www.nature.com/articles/d41586-021-00332-4 |journal=Nature |language=en |volume=590 |issue=7845 |pages=202–205 |doi=10.1038/d41586-021-00332-4}}</ref> Problems with comparing cycle threshold (Ct) values from RT-PCR between different protocols, and even between the same protocols at different locations, combined with uncertainty about the range of viral loads that constitute a transmission risk, have been the cause of many of the controversies.<ref name=":0" /><ref name=":5">{{Cite journal |last=Mina |first=Michael J |last2=Peto |first2=Tim E |last3=García-Fiñana |first3=Marta |last4=Semple |first4=Malcolm G |last5=Buchan |first5=Iain E |date=2021-04 |title=Clarifying the evidence on SARS-CoV-2 antigen rapid tests in public health responses to COVID-19 |url=https://doi.org/10.1016/S0140-6736(21)00425-6 |journal=The Lancet |volume=397 |issue=10283 |pages=1425–1427 |doi=10.1016/s0140-6736(21)00425-6 |issn=0140-6736 |pmc=PMC8049601 |pmid=33609444}}</ref> Individuals are most infectious around the time of symptom onset, when viral load and [[Viral shedding|shedding]] in the upper respiratory tract is highest<ref>{{Cite journal |last=He |first=Xi |last2=Lau |first2=Eric H. Y. |last3=Wu |first3=Peng |last4=Deng |first4=Xilong |last5=Wang |first5=Jian |last6=Hao |first6=Xinxin |last7=Lau |first7=Yiu Chung |last8=Wong |first8=Jessica Y. |last9=Guan |first9=Yujuan |last10=Tan |first10=Xinghua |last11=Mo |first11=Xiaoneng |date=2020-05 |title=Temporal dynamics in viral shedding and transmissibility of COVID-19 |url=https://www.nature.com/articles/s41591-020-0869-5 |journal=Nature Medicine |language=en |volume=26 |issue=5 |pages=672–675 |doi=10.1038/s41591-020-0869-5 |issn=1546-170X}}</ref><ref name=":1">{{Cite journal |last=Singanayagam |first=Anika |last2=Patel |first2=Monika |last3=Charlett |first3=Andre |last4=Lopez Bernal |first4=Jamie |last5=Saliba |first5=Vanessa |last6=Ellis |first6=Joanna |last7=Ladhani |first7=Shamez |last8=Zambon |first8=Maria |last9=Gopal |first9=Robin |date=2020-08-13 |title=Duration of infectiousness and correlation with RT-PCR cycle threshold values in cases of COVID-19, England, January to May 2020 |url=https://www.eurosurveillance.org/content/10.2807/1560-7917.ES.2020.25.32.2001483 |journal=Eurosurveillance |language=en |volume=25 |issue=32 |doi=10.2807/1560-7917.ES.2020.25.32.2001483 |issn=1560-7917 |pmc=PMC7427302 |pmid=32794447}}</ref>, with recent studies confirming an association between viral load and increased transmission of SARS-CoV-2.<ref name=":6">{{Cite journal |last=Marks |first=Michael |last2=Millat-Martinez |first2=Pere |last3=Ouchi |first3=Dan |last4=Roberts |first4=Chrissy h |last5=Alemany |first5=Andrea |last6=Corbacho-Monné |first6=Marc |last7=Ubals |first7=Maria |last8=Tobias |first8=Aurelio |last9=Tebé |first9=Cristian |last10=Ballana |first10=Ester |last11=Bassat |first11=Quique |date=2021-05 |title=Transmission of COVID-19 in 282 clusters in Catalonia, Spain: a cohort study |url=https://linkinghub.elsevier.com/retrieve/pii/S1473309920309853 |journal=The Lancet Infectious Diseases |language=en |volume=21 |issue=5 |pages=629–636 |doi=10.1016/S1473-3099(20)30985-3 |pmc=PMC7906723 |pmid=33545090}}</ref> For asymptomatic individuals, infectivity and viral load dynamics involve a similar, limited period of infectivity. Asymptomatic and presymptomatic contributions to viral spread in the community remain problematic, accounting for a notable proportion of transmissions while often going undetected.<ref name=":14">{{Cite journal |last=Lee |first=Seungjae |last2=Kim |first2=Tark |last3=Lee |first3=Eunjung |last4=Lee |first4=Cheolgu |last5=Kim |first5=Hojung |last6=Rhee |first6=Heejeong |last7=Park |first7=Se Yoon |last8=Son |first8=Hyo-Ju |last9=Yu |first9=Shinae |last10=Park |first10=Jung Wan |last11=Choo |first11=Eun Ju |date=2020-11-01 |title=Clinical Course and Molecular Viral Shedding Among Asymptomatic and Symptomatic Patients With SARS-CoV-2 Infection in a Community Treatment Center in the Republic of Korea |url=https://doi.org/10.1001/jamainternmed.2020.3862 |journal=JAMA Internal Medicine |language=en |volume=180 |issue=11 |pages=1447 |doi=10.1001/jamainternmed.2020.3862 |issn=2168-6106 |pmc=PMC7411944 |pmid=32780793}}</ref><ref>{{Cite journal |last=Cevik |first=Muge |last2=Kuppalli |first2=Krutika |last3=Kindrachuk |first3=Jason |last4=Peiris |first4=Malik |date=2020-10-23 |title=Virology, transmission, and pathogenesis of SARS-CoV-2 |url=https://www.bmj.com/content/371/bmj.m3862 |journal=BMJ |language=en |volume=371 |pages=m3862 |doi=10.1136/bmj.m3862 |issn=1756-1833 |pmid=33097561}}</ref>

Several studies have shown a relationship between Ct value and culture of infectious virus<ref name=":1" /><ref name=":7">{{Cite journal |last=Bullard |first=Jared |last2=Dust |first2=Kerry |last3=Funk |first3=Duane |last4=Strong |first4=James E |last5=Alexander |first5=David |last6=Garnett |first6=Lauren |last7=Boodman |first7=Carl |last8=Bello |first8=Alexander |last9=Hedley |first9=Adam |last10=Schiffman |first10=Zachary |last11=Doan |first11=Kaylie |date=2020-11-15 |title=Predicting Infectious Severe Acute Respiratory Syndrome Coronavirus 2 From Diagnostic Samples |url=https://doi.org/10.1093/cid/ciaa638 |journal=Clinical Infectious Diseases |volume=71 |issue=10 |pages=2663–2666 |doi=10.1093/cid/ciaa638 |issn=1058-4838 |pmc=PMC7314198 |pmid=32442256}}</ref><ref name=":8">{{Cite journal |last=Kim |first=Min-Chul |last2=Cui |first2=Chunguang |last3=Shin |first3=Kyeong-Ryeol |last4=Bae |first4=Joon-Yong |last5=Kweon |first5=Oh-Joo |last6=Lee |first6=Mi-Kyung |last7=Choi |first7=Seong-Ho |last8=Jung |first8=Sun-Young |last9=Park |first9=Man-Seong |last10=Chung |first10=Jin-Won |date=2021-02-18 |title=Duration of Culturable SARS-CoV-2 in Hospitalized Patients with Covid-19 |url=http://www.nejm.org/doi/10.1056/NEJMc2027040 |journal=New England Journal of Medicine |language=en |volume=384 |issue=7 |pages=671–673 |doi=10.1056/NEJMc2027040 |issn=0028-4793 |pmc=PMC7934323 |pmid=33503337}}</ref><ref name=":9">{{Cite journal |last=Jefferson |first=T |last2=Spencer |first2=E A |last3=Brassey |first3=J |last4=Heneghan |first4=C |date=2020-12-03 |title=Viral cultures for COVID-19 infectious potential assessment – a systematic review |url=https://academic.oup.com/cid/advance-article/doi/10.1093/cid/ciaa1764/6018217 |journal=Clinical Infectious Diseases |language=en |pages=ciaa1764 |doi=10.1093/cid/ciaa1764 |issn=1058-4838 |pmc=PMC7799320 |pmid=33270107}}</ref>, and manufacturers of LFDs have implied a link between antigen test positivity and infectious potential. The aim of this study was to assess in detail the relationship between Ct value, viral load, quantitative culture of infectious virus, and antigen test positivity, and provide an independent and unbiased head-to-head comparison of six widely available commercial antigen tests. Tests were also reassessed in consideration of the emergence and spread of the SARS-CoV-2 B.1.1.7 genotype in the U.K., first detected in November 2020.

==Methods==
===Study samples===
In a single-center, [[laboratory]] evaluation study, we evaluated the performance of six rapid antigen tests commercially available in the U.K. by head-to-head comparison. Combined [[nasopharyngeal swab]]s were submitted for routine diagnostic SARS-CoV-2 testing by qRT-PCR to the Viapath Infection Sciences Laboratory (St. Thomas' Hospital, London, U.K.) in 1 mL of viral transport medium (VTM; Sigma Virocult, Medical Wire & Equipment, Corsham, UK). Surplus VTM was stored immediately at –80°C after determination of a diagnostic result, with no additional freeze-thaw cycles before inclusion in the study. Swabs processed by the laboratory were from inpatients and outpatients from all departments of St. Thomas' Hospital, and from health-care staff across all departments and their household contacts (a breakdown of the numbers of inpatients, outpatients, staff, and contacts is not available due to masking of the research team to this [[information]]).

VTM from 100 SARS-CoV-2-negative swabs (confirmed by qRT-PCR) from the same population were used for comparative specificity determinations. Two different SARS-CoV-2-positive sample sets (qRT-PCR-confirmed) were used: one set for head-to-head sensitivity comparisons of six commercial antigen tests (''n'' = 100) and one set for comparative studies of infectivity and antigen test positivity (''n'' = 141). A single sample set could not be used for all analyses because the volume of VTM required exceeded that provided by a single sample; therefore, two collections of samples were used. There was no difference in how these samples were selected. All samples were collected between March 23 and Oct 27, 2020, and they were demographically representative of the typical population providing samples for testing at the diagnostic laboratory during this period. Samples confirmed as positive were selected to cover a wide range of Ct values (12.7 to 40.0), but were not subjected to any further selection criteria. Sampling time from symptom onset ranged from –1 to 37 days. An independent subset of sequential swabs from five inpatients with different disease severities (two asymptomatic, two mild, and one severe; whereby we planned to include different disease severities within the constraints of sample availability, with details of disease scoring reported previously<ref name=":2">{{Cite journal |last=Pickering |first=Suzanne |last2=Betancor |first2=Gilberto |last3=Galão |first3=Rui Pedro |last4=Merrick |first4=Blair |last5=Signell |first5=Adrian W. |last6=Wilson |first6=Harry D. |last7=Ik |first7=Mark Tan Kia |last8=Seow |first8=Jeffrey |last9=Graham |first9=Carl |last10=Acors |first10=Sam |last11=Kouphou |first11=Neophytos |date=2020-09-24 |title=Comparative assessment of multiple COVID-19 serological technologies supports continued evaluation of point-of-care lateral flow assays in hospital and community healthcare settings |url=https://journals.plos.org/plospathogens/article?id=10.1371/journal.ppat.1008817 |journal=PLOS Pathogens |language=en |volume=16 |issue=9 |pages=e1008817 |doi=10.1371/journal.ppat.1008817 |issn=1553-7374 |pmc=PMC7514033 |pmid=32970782}}</ref><ref name=":3">{{Cite journal |last=Seow |first=Jeffrey |last2=Graham |first2=Carl |last3=Merrick |first3=Blair |last4=Acors |first4=Sam |last5=Pickering |first5=Suzanne |last6=Steel |first6=Kathryn J. A. |last7=Hemmings |first7=Oliver |last8=O’Byrne |first8=Aoife |last9=Kouphou |first9=Neophytos |last10=Galao |first10=Rui Pedro |last11=Betancor |first11=Gilberto |date=2020-12 |title=Longitudinal observation and decline of neutralizing antibody responses in the three months following SARS-CoV-2 infection in humans |url=https://www.nature.com/articles/s41564-020-00813-8 |journal=Nature Microbiology |language=en |volume=5 |issue=12 |pages=1598–1607 |doi=10.1038/s41564-020-00813-8 |issn=2058-5276 |pmc=PMC7610833 |pmid=33106674}}</ref>), collected as part of the patient's routine standard of care, were used for longitudinal studies of infectivity and antigen test positivity (between two and five longitudinal samples per individual; 21 samples overall). A further 23 qRT-PCR-confirmed SARS-CoV-2 positive swabs, collected from Dec 4, 2020, to Jan 12, 2021, were shown by on-site [[whole genome sequencing]] (Oxford Nanopore Technologies, Oxford, U.K.) to be from the B.1.1.7 variant and used for comparative evaluation of the sensitivity of available LFDs (based on kit availability). These were compared with samples from April and September 2020 (''n'' = 23 for each month; assumed to be non-B.1.1.7 variant) from the SARS-CoV-2-positive sample sets, which were selected on the basis of approximately equivalent Ct values to the B.1.1.7 samples. To minimise sample deterioration, all VTM samples were thawed once and immediately subjected to RNA extraction (for confirmatory qRT-PCR), LFD assessment, and viral growth assays as appropriate.

===qRT-PCR===
Initial diagnostic laboratory testing was done with the AusDiagnostics multiplexed-tandem PCR assay including SARS-CoV-2 (Chesham, U.K.), and positive and negative swabs were selected on the basis of this diagnostic test. For confirmatory PCR testing and to ensure uniformity of qRT-PCR conditions and Ct determination, RNA was extracted from 100 μL swab with the Qiagen QIAamp Viral RNA Kit (Hilden, Germany) following manufacturer's instructions and eluted in 60 μL water. qRT-PCR reactions (total volume 20 μL) were done with 5 μL eluted RNA, TaqMan Fast Virus 1-Step Master Mix (4X formulation; Applied Biosystems, Waltham, MA, USA), and primer-probes sets targeting SARS-CoV-2 nucleocapsid (SARS-CoV-2-N; N1 set) gene regions or human RNAse P designed by the U.S. [[Centers for Disease Control and Prevention]] (manufactured by Integrated DNA Technologies, Coralville, IA, USA), with a QuantStudio 5 Real-Time PCR System (ThermoFisher Scientific, Waltham, MA, USA). For the calculation of viral loads (RNA copies per mL), RNA standards were extracted as described from serial dilutions of a NATtrol SARS-CoV-2 Stock (ZeptoMetrix, Buffalo, NY, USA), which is formulated with purified, inactivated, intact viral particles of known RNA copies per /mL. A calibration step was used to determine SARS-CoV-2-N Ct value (see the supplemental appendix, p. 3; referred to as N Ct value hereafter).

===Rapid antigen tests===
The following SARS-CoV-2 rapid antigen tests were used for comparative studies: Innova Rapid SARS-CoV-2 Antigen Test (Xiamen Biotime Biotechnology, Fujian, China), Spring Healthcare SARS-CoV-2 Antigen Rapid Test Cassette (Shanghai ZJ Bio-Tech, Shanghai, China), E25Bio Rapid Diagnostic Test (E25Bio, Cambridge, MA, USA), Encode SARS-CoV-2 Antigen Rapid Test Device (Zhuhai Encode Medical Engineering, Zhuhai, China), SureScreen COVID-19 Rapid Antigen Test Cassette, and SureScreen COVID-19 Rapid Fluorescence Antigen Test (both from SureScreen Diagnostics, Derby, UK). We refer to the test kits as Innova, Spring Healthcare, E25Bio, Encode, SureScreen visual (SureScreen-V), and SureScreen fluorescent (SureScreen-F) hereafter. 

To allow extensive comparative studies alongside the determination of infectious virus in clinical samples, studies were done on swabs stored in VTM, rather than direct swabs taken immediately before tests. 50 μL of stored VTM was mixed with 100 μL buffer supplied by each test kit, and 100 μL of this solution was applied to the test cassette, as per manufacturer instructions. To identify the limit of detection (LOD) for each test, known plaque-forming units (PFUs) of SARS-CoV-2 (England 02/2020 strain; Public Health England, Porton Down, U.K.) were propagated in Vero E6 cells and diluted in phosphate-buffered saline, and 50 μL of each dilution was mixed with 100 μL kit-supplied buffer. 100 μL of this solution was applied to each test, equating to 1–10 000 PFUs/test or 30–300 000 PFU/mL, and each quantity was tested in triplicate for all tests. NATrol SARS-CoV-2 Stock (ZeptoMetrix, Buffalo, NY, USA) was tested as a ''post-hoc'' control on tests with the best LOD. Results of all tests were read at the time instructed by the manufacturer (between 10 and 30 min); results were recorded independently by two readers (SP and RPG) and compared, and, in the event of a discordant reading, a third individual (RB or MTKI) was referred to. For purposes of comparison, the [[Chromatography|chromatographic]] tests were scored according to whether the test band was strongly positive, unequivocally positive, weakly positive, or negative, with the exception of the SureScreen-F test, which is machine-read and delivers a binary result (positive or negative). Test band scoring was used to provide detailed information on the nature of the test result, but all sensitivity and specificity calculations were based on the binary results of the tests. Details of the tests and examples of the scoring of tests are given in the supplementary appendix (pp. 4, 7). Although the tests are qualitative and all results were treated as binary, results are also displayed as a heatmap (see supplementary appendix, p. 7) to convey the magnitude of the result, allowing more detailed comparisons between the tests and potentially informing future use.

===Viral growth assays===
For the comparative studies of infectivity and antigen positivity, each swab was subjected to the following procedures: RNA extraction for subsequent qRT-PCR and sequencing; titration and viral titer measurement by plaque assay; titration and infectivity determination by intracellular anti-SARS-CoV-2-N staining (in samples with sufficient volume remaining); viral propagation for isolation of virus; and LFD testing (see supplementary appendix, p. 3). LFDs were selected for the correlative study of infectivity and antigen test positivity on the basis of highest sensitivity and highest specificity (in the initial comparative studies), and use of an alternative technology (SureScreen-F). However, due to sample volume or test availability, not all samples could be assayed in all selected tests. To avoid bias due to comparing different samples, results were also analyzed for the subset of samples available for all tests. Viral growth assays were done in Vero E6 cells (see supplementary appendix, p. 3).

===Statistical analysis===
Linear regressions and associated R<sup>2</sup> and ''p'' values were determined to test the relationship between observed Ct values for the N gene and either log<sub>10</sub> SARS-CoV-2 viral load (measured as RNA copies per mL) or log<sub>10</sub> SARS-CoV-2 PFUs/ml. Exact binomial 95% CIs for the specificity and sensitivity of all LFDs were determined with the Wilson–Brown method. Analytical sensitivities in clinical samples were also assessed with binomial logistic regressions fitted to a binary dependent Y variable (LFD or viral culture), and an independent X variable (Ct N value or log<sub>10</sub> SARS-CoV-2 viral load measured as RNA copies per mL). These regression models were used to determine 50% detection rates (and corresponding 95% CIs) of the LFDs, defined as the predicted Ct N or RNA viral load concentrations measured by qRT-PCR, at which 50% of results were positive in each LFD or for viral culture. A receiver operating characteristic curve and corresponding area under the curve were determined for each logistic regression. Cumulative sensitivity was calculated for each LFD across ascending Ct values in single-sample increments. Sensitivities and specificities were compared for paired samples across test kits with McNemar's test (head-to-head comparisons). Fisher's exact test was used to compare sensitivities for B.1.1.7 variant swabs with earlier variant specimens collected in 2020. All tests were two-sided, and ''p'' values lower than 0.05 were considered statistically significant. All statistical analyses were calculated in GraphPad Prism 9.

===Role of the funding source===
The funder of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report, or in the decision to submit this manuscript for publication.

==Results==
Six commercial rapid antigen tests (Innova, E25Bio, Spring Healthcare, Encode, SureScreen-V, and SureScreen-F; see supplementary appendix, p. 4) were compared for specificity, LODs, and sensitivity.

Specificity was determined for each test with a panel of 100 qRT-PCR-confirmed SARS-CoV-2-negative swabs (Table 1). All tests showed high (≥98.0%) specificity, with the exception of E25Bio (86.0% [95% CI 77.9–99.9]). SureScreen-V and Encode both achieved a specificity of 100.0% (96.3–100.0). None of the negative samples gave a false-positive result for more than one test kit, suggesting that false positives appear stochastically and are not a particular feature of the samples. All false positives were only weakly positive, with the exception of SureScreen-F, for which this information was not available as the electronic reader delivers a binary result.

{| 
 | style="vertical-align:top;" |
{| class="wikitable" border="1" cellpadding="5" cellspacing="0" width="80%"
 |-
  | colspan="8" style="background-color:white; padding-left:10px; padding-right:10px;" |'''Table 1.''' Specificity and sensitivity of six commercial SARS-CoV-2 rapid antigen tests compared with real-time RT-PCR. ROC curves (not shown) and corresponding AUCs were determined in logistic regressions. A Ct value of 28 (corresponding to 50 PFUs/ml) is the approximate value at which all the tests failed to detect our viral stock dilutions (see Figure 1A); a Ct value of 25 provided a 10-times interval from Ct=28, and was chosen to facilitate the data analysis and highlight the observation that test sensitivities are higher at lower Ct values. SureScreen-F = SureScreen fluorescent; SureScreen-V = SureScreen visual; N = SARS-CoV-2 nucleocapsid; Ct = cycle threshold; ROC = receiver operating characteristic; AUC = area under the curve.
 |-
  ! style="padding-left:10px; padding-right:10px;" |
  ! style="padding-left:10px; padding-right:10px;" |Number of samples
  ! style="padding-left:10px; padding-right:10px;" |Specificity (95% CI)
  ! style="padding-left:10px; padding-right:10px;" |Number of samples
  ! style="padding-left:10px; padding-right:10px;" |Sensitivity (95% CI)
  ! style="padding-left:10px; padding-right:10px;" |RNA copies per mL at 50% detection rate (95% CI)
  ! style="padding-left:10px; padding-right:10px;" |N Ct value at 50% detection rate (95% CI)
  ! style="padding-left:10px; padding-right:10px;" |ROC AUC (95% CI)
 |-
  | colspan="8" style="background-color:white; padding-left:10px; padding-right:10px;" |'''Innova'''
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Overall
  | style="background-color:white; padding-left:10px; padding-right:10px;" |100
  | style="background-color:white; padding-left:10px; padding-right:10px;" |99.0% (94.6–99.9)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |100
  | style="background-color:white; padding-left:10px; padding-right:10px;" |89.0% (81.4–93.8)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |2.29 × 10<sup>4</sup> (1.70 × 10<sup>3</sup>–9.55 × 10<sup>4</sup>)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |30.7 (28.8–34.3)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |0.948 (0.896–0.999)
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <28
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |88
  | style="background-color:white; padding-left:10px; padding-right:10px;" |95.5% (88.9–98.2)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <25
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |69
  | style="background-color:white; padding-left:10px; padding-right:10px;" |98.6% (92.2–99.9)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | colspan="8" style="background-color:white; padding-left:10px; padding-right:10px;" |'''Spring Healthcare'''
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Overall
  | style="background-color:white; padding-left:10px; padding-right:10px;" |100
  | style="background-color:white; padding-left:10px; padding-right:10px;" |98.0% (93.0–99.6)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |100
  | style="background-color:white; padding-left:10px; padding-right:10px;" |77.0% (67.8–84.2)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |3.8 × 10<sup>5</sup> (1.35 × 10<sup>5</sup>–8.51 × 10<sup>5</sup>)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |26.8 (25.8–28.3)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |0.949 (0.908–0.989)
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <28
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |88
  | style="background-color:white; padding-left:10px; padding-right:10px;" |85.2% (76.4–91.2)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <25
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |69
  | style="background-color:white; padding-left:10px; padding-right:10px;" |95.7% (88.0–98.8)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | colspan="8" style="background-color:white; padding-left:10px; padding-right:10px;" |'''E25Bio'''
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Overall
  | style="background-color:white; padding-left:10px; padding-right:10px;" |100
  | style="background-color:white; padding-left:10px; padding-right:10px;" |86.0% (77.9–99.9)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |100
  | style="background-color:white; padding-left:10px; padding-right:10px;" |75.0% (65.7–82.4)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |4.27 × 10<sup>5</sup> (1.26 × 10<sup>5</sup>–1.07 × 10<sup>6</sup>)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |26.7 (25.5–28.4)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |0.915 (0.854–0.975)
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <28
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |88
  | style="background-color:white; padding-left:10px; padding-right:10px;" |83·0% (73·8–89·4)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <25
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |69
  | style="background-color:white; padding-left:10px; padding-right:10px;" |94·2% (86·0–97·7)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | colspan="8" style="background-color:white; padding-left:10px; padding-right:10px;" |'''Encode'''
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Overall
  | style="background-color:white; padding-left:10px; padding-right:10px;" |100
  | style="background-color:white; padding-left:10px; padding-right:10px;" |100.0% (96.3–100.0)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |100
  | style="background-color:white; padding-left:10px; padding-right:10px;" |74.0% (64.6–81.6)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |6.31 × 10<sup>5</sup> (2.69 × 10<sup>5</sup>–1.29 × 10<sup>6</sup>)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |26.2 (25.2–27.4)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |0.956 (0.920–0.992)
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <28
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |88
  | style="background-color:white; padding-left:10px; padding-right:10px;" |83.0% (73.8–89.4)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <25
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |69
  | style="background-color:white; padding-left:10px; padding-right:10px;" |94.2% (86.0–97.7)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | colspan="8" style="background-color:white; padding-left:10px; padding-right:10px;" |'''SureScreen-F'''
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Overall
  | style="background-color:white; padding-left:10px; padding-right:10px;" |100
  | style="background-color:white; padding-left:10px; padding-right:10px;" |98.0% (93.0–99.6)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |100
  | style="background-color:white; padding-left:10px; padding-right:10px;" |69.0% (59.4–77.2)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |8.91 × 10<sup>5</sup> (2.57 × 10<sup>5</sup>–2.24 × 10<sup>6</sup>)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |25.7 (24.4–27.4)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |0.891 (0.819–0.962)
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <28
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |88
  | style="background-color:white; padding-left:10px; padding-right:10px;" |75.0% (65.0–82.9)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <25
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |69
  | style="background-color:white; padding-left:10px; padding-right:10px;" |88.4% (78.8–94.0)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | colspan="8" style="background-color:white; padding-left:10px; padding-right:10px;" |'''SureScreen-V'''
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Overall
  | style="background-color:white; padding-left:10px; padding-right:10px;" |100
  | style="background-color:white; padding-left:10px; padding-right:10px;" |100·.0% (96.3–100.0)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |100
  | style="background-color:white; padding-left:10px; padding-right:10px;" |65.0% (55.2–73.6)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |2.09 × 10<sup>6</sup> (1.12 × 10<sup>6</sup>–3.80 × 10<sup>6</sup>)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |24.5 (23.7–25.4)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |0.968 (0.936–1.000)
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <28
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |88
  | style="background-color:white; padding-left:10px; padding-right:10px;" |73.9% (63.8–81.9)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <25
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |69
  | style="background-color:white; padding-left:10px; padding-right:10px;" |91.3% (82.3–96.0)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
|}
|}

LODs were determined with specified PFUs of SARS-CoV-2 propagated in Vero E6 cells, applied to each test in triplicate. Informed by the specificity determinations, in which we observed very few false positives, any visible band was considered positive regardless of intensity or relationship to the control band. Most tests reliably detected 50 PFUs/test (1500 PFUs/mL), with the exception of Encode and SureScreen-F (Figure 1A). SureScreen-V and Innova had the lowest consistent LOD (excluding E25Bio due to poor specificity), and on further testing of SureScreen-V, this test also consistently detected 20 PFUs/test (600 PFUs/mL). Calibration experiments with SARS-CoV-2 laboratory stock (Figure 1B) and standardized RNA control reagents (Figure 1C) delivered the equivalent N gene Ct value of 23.7 or RNA copy number of 3x106/mL for the LOD of 1500 PFUs/mL. As particle-to-infectious unit ratios can vary between viral variants or according to growth or assaying conditions, as a ''post-hoc'' control we applied the Zeptometrix NATrol inactivated viral particle standard to Innova and SureScreen-V, as the two tests with the best LOD. This standard showed as weakly positive on both tests at 1.2 × 106 RNA copies per mL, or projected Ct value of 25, in agreement with the results shown in Figure 1.


[[File:Fig1 Pickering LancetMicrobe2021 2-9.jpg|900px]]
{{clear}}
{| 
 | style="vertical-align:top;" |
{| border="0" cellpadding="5" cellspacing="0" width="900px"
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |<blockquote>'''Figure 1.''' Comparative sensitivity of six commercial SARS-CoV-2 rapid antigen tests. (''A'') Heatmap comparison of lower limit of detection. (''B'') Association between PFUs/mL and Ct result from qRT-PCR for the SARS-CoV-2 N gene (N Ct value). Error bars represent SD of three independent experiments. (''C'') Association between RNA copy number and N Ct values. Copy number per mL was derived from N qRT-PCR on the Zeptometrix RNA standard (consisting of purified, inactivated viral particles of known RNA copy number per mL). In (B) and (C), points were fitted with a linear regression, with the equation of the line and the R<sup>2</sup> value shown. Horizontal dashed lines denote Ct values of 25 and 28, as the threshold cutoffs used for sensitivity determinations. (''D'') Tests were evaluated in head-to-head comparisons on a panel of 100 SARS-CoV-2-positive nasopharyngeal swabs. Bars denote the N Ct result for each swab, in ascending order, with the antigen test results for each sample directly below each bar presented as a heatmap. Sensitivity determinations from this sample set are shown on the right. Ct value cutoffs of 25 and 28, corresponding to 1.49 × 106 and 1.65 × 105 RNA copies per mL, or 400 and 50 PFUs/mL, respectively, are indicated, with corresponding sensitivity values for each test at each threshold on the right. PFU = plaque-forming unit; SureScreen-F = SureScreen fluorescent; SureScreen-V = SureScreen visual; N = SARS-CoV-2 nucleocapsid; Ct = cycle threshold; qRT-PCR = real-time RT-PCR.</blockquote>
 |- 
|}
|}

Sensitivity comparisons on clinical samples were done as head-to-head evaluations on 100 SARS-CoV-2-positive combined nasopharyngeal swabs, with Ct values ranging from 14.0 to 39.0 (see Figure 1D and supplementary appendix, p. 8). The 50% detection rates and cumulative sensitivities across all Ct values are shown in the supplementary appendix (p. 8). Innova had the highest overall sensitivity (89.0% [95% CI 81.4–93.8]) for the clinical samples, with this increasing to 95.5% (88.9–98.2) when applied to samples with Ct values lower than 28, and 98.6% (92.2–99.9) when applied to samples with Ct values lower than 25 (see Table 1 and supplementary appendix, p. 8). All other tests had overall sensitivities of between 65.0% and 77.0%, increasing to greater than 90% for samples with Ct values lower than 25 for all tests except SureScreen-F. Thus, we found good sensitivity and specificity for all tests on swabs within a defined Ct value window.

Three of the rapid antigen tests from the first phase of comparisons were selected for more detailed comparisons: the tests with highest sensitivity (Innova), highest specificity (Encode), and alternative technology (fluorescent machine-read result; SureScreen-F). 141 combined nasopharyngeal swabs were compared for N Ct value, antigen test result, and positive viral culture (Figure 2A). Samples covered a range of Ct values (12.7 to 40.0). The direct viral titer of the swabs was determined by plaque assay of serially diluted swabs, with additional confirmatory intracellular anti-SARS-CoV-2 nucleocapsid staining performed on viral culture samples for 110 samples of sufficient volume. 57 (40.4%) of the 141 samples were positive for viral growth. 54 (94.7%) of the 57 cultures positive for viral growth had Ct values lower than 25, and the highest Ct value from a sample with positive viral culture was 26.3. The latest timepoint that virus was isolated was 15 days after symptom onset. Titers of infectious virus in the samples showed a moderate inverse linear relationship with N Ct values (R<sup>2</sup> = 0.47, ''p''<0.0001; Figure 2B). Both viral culture and antigen test positivity were associated with Ct value, rather than the timing of the sample relative to symptom onset (see Figure 2C and supplementary appendix, p. 9).


[[File:Fig2 Pickering LancetMicrobe2021 2-9.jpg|900px]]
{{clear}}
{| 
 | style="vertical-align:top;" |
{| border="0" cellpadding="5" cellspacing="0" width="900px"
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |<blockquote>'''Figure 2.''' Comparison of Ct value, infectivity, and rapid antigen test result. (''A'') Infectivity, N Ct value, and antigen test results for three commercial tests. Bars show the N Ct result for each swab in ascending order, colored according to whether virus was cultured from the sample. Antigen test results for each sample are shown below each bar. Sensitivity determinations for the sample set are shown on the right. (''B'') Direct viral titers of swabs determined by plaque assay and compared with N Ct value. The linear relationship between Ct value and log<sub>10</sub> concentration of infectious virus in 57 samples is shown. (''C'') Ct results from (A) and RNA copies per mL plotted against days from symptom onset. Points are colored according to virus growth (left graph; ''n'' = 141) or the score derived from the Innova antigen test (right graph; ''n'' = 110). (''D'') Longitudinal examples of infectivity, antigen test positivity, and Ct values for five infected individuals with varying COVID-19 severities.<ref name=":2" /><ref name=":3" /> Bars show the N Ct value for each sample, shaded according to virus culture result. Antigen test results for each sample are shown below the bars. N = SARS-CoV-2 nucleocapsid; Ct = cycle threshold; SureScreen-F = SureScreen fluorescent; SureScreen-V = SureScreen visual. <sup>*</sup> = Less than total sample due to limited sample volume or number of test kits available. <sup>†</sup> = Time post-onset of symptoms for symptomatic individuals; time from the first positive PCR test for asymptomatic individuals. <sup>‡</sup> = Sufficient sample volume and test availability for inclusion in the longitudinal study for completeness of the dataset.</blockquote>
 |- 
|}
|}

Among the three selected antigen tests, overall test sensitivities and rankings were similar to those observed in initial assessments (Figure 1), with Innova providing the highest sensitivity at 78.2% (95% CI 69.6–84.9), followed by Encode (74.4% [64.6–82.3]) and SureScreen-F (60.3% [52.0–68.0]; see Table 2 and supplementary appendix, p. 8). When compared only with the samples from which virus was cultured, all tests achieved a sensitivity of at least 94.7% (See Figure 2A and Table 2). For the subset of available samples tested on all three LFDs (''n'' = 90 for sensitivity, ''n'' = 34 for infectious samples) to give a head-to-head comparison, we observed similar results to the overall comparison (see Table 2 and supplementary appendix, p. 5).

{| 
 | style="vertical-align:top;" |
{| class="wikitable" border="1" cellpadding="5" cellspacing="0" width="80%"
 |-
  | colspan="8" style="background-color:white; padding-left:10px; padding-right:10px;" |'''Table 2.''' Sensitivity of three commercial SARS-CoV-2 rapid antigen tests compared with qRT-PCR and virus isolation. SureScreen-F = SureScreen fluorescent; qRT-PCR = real-time RT-PCR; N = SARS-CoV-2 nucleocapsid; Ct = cycle threshold; ROC = receiver operating characteristic; AUC = area under the curve. <sup>*</sup> = Test results were re-analysed due to differences in sample number between Encode, Innova, and SureScreen-F (for the subset of samples available for all three tests: ''n'' = 90 for sensitivity compared with qRT-PCR, ''n'' = 34 for sensitivity compared with virus isolation); two-tailed ''p'' values for these subsets, calculated with McNemar's test for all possible permutations, are shown in the supplementary appendix (p. 5). <sup>†</sup> = Due to insufficient sample volume or test unavailability, not all samples could be assayed on Innova and Encode.
 |-
  ! style="padding-left:10px; padding-right:10px;" |
  ! style="padding-left:10px; padding-right:10px;" |Number of samples
  ! style="padding-left:10px; padding-right:10px;" |Sensitivity vs. qRT-PCR (95% CI)
  ! style="padding-left:10px; padding-right:10px;" |RNA copies per mL at 50% detection rate (95% CI)
  ! style="padding-left:10px; padding-right:10px;" |N Ct value at 50% detection rate (95% CI)
  ! style="padding-left:10px; padding-right:10px;" |ROC AUC (95% CI)
  ! style="padding-left:10px; padding-right:10px;" |Number of samples
  ! style="padding-left:10px; padding-right:10px;" |Sensitivity vs. virus isolation (95% CI)
 |-
  | colspan="8" style="background-color:white; padding-left:10px; padding-right:10px;" |'''Encode'''
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Overall
  | style="background-color:white; padding-left:10px; padding-right:10px;" |90<sup>†</sup>
  | style="background-color:white; padding-left:10px; padding-right:10px;" |74.4% (64.6–82.3)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |1.31 × 10<sup>5</sup> (3.47 × 10<sup>4</sup>–3.71 × 10<sup>5</sup>)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |28.3 (26.9– 30.1)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |0.931 (0.879–0.984)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |34
  | style="background-color:white; padding-left:10px; padding-right:10px;" |100.0% (89.8–100.0)
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <28
  | style="background-color:white; padding-left:10px; padding-right:10px;" |70
  | style="background-color:white; padding-left:10px; padding-right:10px;" |88.6% (79.0–94.1)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <25
  | style="background-color:white; padding-left:10px; padding-right:10px;" |52
  | style="background-color:white; padding-left:10px; padding-right:10px;" |96.2% (87.0–99.3)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Subset<sup>*</sup>
  | style="background-color:white; padding-left:10px; padding-right:10px;" |90
  | style="background-color:white; padding-left:10px; padding-right:10px;" |74.4% (64.6–82.3)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <28
  | style="background-color:white; padding-left:10px; padding-right:10px;" |70
  | style="background-color:white; padding-left:10px; padding-right:10px;" |96.2% (87.0–99.3)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <25
  | style="background-color:white; padding-left:10px; padding-right:10px;" |52
  | style="background-color:white; padding-left:10px; padding-right:10px;" |88.6% (79.0–94.1)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | colspan="8" style="background-color:white; padding-left:10px; padding-right:10px;" |'''Innova'''
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Overall
  | style="background-color:white; padding-left:10px; padding-right:10px;" |110<sup>†</sup>
  | style="background-color:white; padding-left:10px; padding-right:10px;" |78.2% (69.6–84.9)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |1.31 × 10<sup>5</sup> (3.47 × 10<sup>4</sup>–3.71 × 10<sup>5</sup>)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |28.3 (26.9– 30.1)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |0.931 (0.879–0.984)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |46
  | style="background-color:white; padding-left:10px; padding-right:10px;" |97.8% (88.7–99.9)
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <28
  | style="background-color:white; padding-left:10px; padding-right:10px;" |88
  | style="background-color:white; padding-left:10px; padding-right:10px;" |92.0% (84.3–96.0)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <25
  | style="background-color:white; padding-left:10px; padding-right:10px;" |66
  | style="background-color:white; padding-left:10px; padding-right:10px;" |97.0% (89.6–99.5)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Subset<sup>*</sup>
  | style="background-color:white; padding-left:10px; padding-right:10px;" |90
  | style="background-color:white; padding-left:10px; padding-right:10px;" |76.7% (67.0–84.2)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |1.10 × 10<sup>5</sup> (3.31 × 10<sup>4</sup>–2.82 × 10<sup>5</sup>)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |28.6 (27.3–30.2)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |0.957 (0.919–0.996)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |34
  | style="background-color:white; padding-left:10px; padding-right:10px;" |100.0% (89.8–100.0)
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <28
  | style="background-color:white; padding-left:10px; padding-right:10px;" |70
  | style="background-color:white; padding-left:10px; padding-right:10px;" |91.4% (82.5–96.0)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <25
  | style="background-color:white; padding-left:10px; padding-right:10px;" |52
  | style="background-color:white; padding-left:10px; padding-right:10px;" |98.1% (89.9–99.9)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | colspan="8" style="background-color:white; padding-left:10px; padding-right:10px;" |'''SureScreen-F'''
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Overall
  | style="background-color:white; padding-left:10px; padding-right:10px;" |141
  | style="background-color:white; padding-left:10px; padding-right:10px;" |60.3% (52.0–68.0)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |1.38 × 10<sup>6</sup> (6.31 × 10<sup>5</sup>–2.88 × 10<sup>6</sup>)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |25.1 (24.1–26.2)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |0.918 (0.875–0.962)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |57
  | style="background-color:white; padding-left:10px; padding-right:10px;" |94.7% (85.6–98.6)
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <28
  | style="background-color:white; padding-left:10px; padding-right:10px;" |111
  | style="background-color:white; padding-left:10px; padding-right:10px;" |74.8% (66.0–81.9)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <25
  | style="background-color:white; padding-left:10px; padding-right:10px;" |82
  | style="background-color:white; padding-left:10px; padding-right:10px;" |84.2% (74.7–90.5)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Subset<sup>*</sup>
  | style="background-color:white; padding-left:10px; padding-right:10px;" |90
  | style="background-color:white; padding-left:10px; padding-right:10px;" |55.6% (45.3–65.4)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |2.04 × 10<sup>6</sup> (8.12 × 10<sup>5</sup>–4.68 × 10<sup>6</sup>)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |24.6 (23.4–25.8)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |0.918 (0.864–0.971)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |34
  | style="background-color:white; padding-left:10px; padding-right:10px;" |100.0% (89.8–100.0)
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <28
  | style="background-color:white; padding-left:10px; padding-right:10px;" |70
  | style="background-color:white; padding-left:10px; padding-right:10px;" |71.4% (60.0–80.7)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <25
  | style="background-color:white; padding-left:10px; padding-right:10px;" |52
  | style="background-color:white; padding-left:10px; padding-right:10px;" |78.8% (66.0–87.8)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
|}
|}

To investigate how antigen test result changed with time, preliminary investigations were done on sequential nasopharyngeal swabs from five infected inpatients with varying disease severities (see supplementary appendix, p. 6). Samples were compared for N Ct value, antigen test result (SureScreen-F, SureScreen-V, Innova, and Encode), and positive viral culture (Figure 2D). All tests identified infectious samples as positive (with the exception of one sample tested by SureScreen-F), and most tests in most patients continued to deliver positive results for several days after peak infectivity. In patients 2, 3, and 4, Innova (and when available, Encode) tested positive for several days longer than SureScreen-F, although the exact length of this extended positivity cannot be stated as intermediate samples were not obtained. In two patients, qRT-PCR testing identified pre-infectious individuals (patients 2 and 4), who were negative in all antigen tests at the time of initial qRT-PCR testing. Two days later, a drop in Ct value in patient 4 coincided with antigen test positivity for all tests and the isolation of infectious virus. The longitudinal results in five individuals (Figure 2D), together with our plots of N Ct result and viral culture against days from symptom onset in the wider sample (56 individuals with a determinate culture result across a range of timepoints; see Gigure 2C), highlight the importance of repeat testing, rather than one-off testing, with LFD rapid antigen tests.

Given that the rapid antigen tests rely on antibody detection of SARS-CoV-2-N, even single amino acid mutations have the potential to affect test sensitivity. As such, test performance should be reassessed for new emerging variants of SARS-CoV-2, such as B.1.1.7, which by the end of our study had become the dominant genotype in the U.K., and contains four mutations in SARS-CoV-2-N, compared with the England 02/2020 strain (Asp3Leu, Arg203Lys, Gly204Arg, and Ser235Phe).<ref>{{Cite journal |last=Loconsole |first=Daniela |last2=Sallustio |first2=Anna |last3=Accogli |first3=Marisa |last4=Centrone |first4=Francesca |last5=Capozzi |first5=Loredana |last6=Del Sambro |first6=Laura |last7=Parisi |first7=Antonio |last8=Chironna |first8=Maria |date=2021-01-28 |editor-last=Roux |editor-first=Simon |title=Genome Sequence of a SARS-CoV-2 VUI 202012/01 Strain Identified from a Patient Returning from London, England, to the Apulia Region of Italy |url=https://journals.asm.org/doi/10.1128/MRA.01487-20 |journal=Microbiology Resource Announcements |language=en |volume=10 |issue=4 |doi=10.1128/MRA.01487-20 |issn=2576-098X |pmc=PMC7844082 |pmid=33509997}}</ref> We did small-scale evaluations using swab samples spanning a range of Ct values (12.7 to 31.8), from April and September 2020, and B.1.1.7-positive swabs (confirmed by viral sequencing) between December 2020 and January 2021 (Figure 3). Both Innova and SureScreen-V tests showed variations in sensitivities between the three batches of samples tested, as would be expected for biological samples, but test results were qualitatively similar, with no evidence of altered sensitivity for the B.1.1.7 swabs (Table 3).


[[File:Fig3 Pickering LancetMicrobe2021 2-9.jpg|600px]]
{{clear}}
{| 
 | style="vertical-align:top;" |
{| border="0" cellpadding="5" cellspacing="0" width="600px"
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |<blockquote>'''Figure 3.''' Comparative evaluation of antigen test sensitivity for the B.1.1.7 variant versus non-B.1.1.7 variant. Combined nasopharyngeal swabs were obtained from 23 individuals with confirmed SARS-CoV-2 B.1.1.7 infection (December 2020 through January 2021) and compared with samples from before the variant was widely circulating in the U.K. population (April and September 2020). All swabs were matched for N Ct values, shown in ascending order in the graph, and tested on Innova and SureScreen-V rapid antigen tests. N = SARS-CoV-2 nucleocapsid; Ct = cycle threshold; SureScreen-V = SureScreen visual.</blockquote>
 |- 
|}
|}

{| 
 | style="vertical-align:top;" |
{| class="wikitable" border="1" cellpadding="5" cellspacing="0" width="80%"
 |-
  | colspan="7" style="background-color:white; padding-left:10px; padding-right:10px;" |'''Table 3.''' Sensitivity comparisons of two commercial SARS-CoV-2 rapid antigen tests on B.1.1.7 samples vs. samples from April and September 2020. qRT-PCR = quantitative RT-PCR; N = SARS-CoV-2 nucleocapsid; Ct = cycle threshold; SureScreen-V = SureScreen visual. <sup>*</sup> = Two-tailed ''p'' values were determined for the comparison of sensitivities between the B.1.1.7 samples (December 2020 through January 2021) and earlier samples (assumed to be non-B.1.1.7) with Fischer's exact test.
 |-
  ! style="padding-left:10px; padding-right:10px;" |
  ! colspan="3" style="padding-left:10px; padding-right:10px;" |Innova
  ! colspan="3" style="padding-left:10px; padding-right:10px;" |SureScreen-V
 |-
  ! style="padding-left:10px; padding-right:10px;" |
  ! style="padding-left:10px; padding-right:10px;" |Number of samples
  ! style="padding-left:10px; padding-right:10px;" |Sensitivity vs. qRT-PCR (95% CI)
  ! style="padding-left:10px; padding-right:10px;" |''p'' value<sup>*</sup>
  ! style="padding-left:10px; padding-right:10px;" |Number of samples
  ! style="padding-left:10px; padding-right:10px;" |Sensitivity vs. qRT-PCR (95% CI)
  ! style="padding-left:10px; padding-right:10px;" |''p'' value<sup>*</sup>
 |-
  | colspan="8" style="background-color:white; padding-left:10px; padding-right:10px;" |'''B.1.1.7'''
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Overall
  | style="background-color:white; padding-left:10px; padding-right:10px;" |23
  | style="background-color:white; padding-left:10px; padding-right:10px;" |78.3% (58.1–90.3)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |23
  | style="background-color:white; padding-left:10px; padding-right:10px;" |65·2% (44.9–81.2)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <25
  | style="background-color:white; padding-left:10px; padding-right:10px;" |16
  | style="background-color:white; padding-left:10px; padding-right:10px;" |100.0% (80.6–100)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
  | style="background-color:white; padding-left:10px; padding-right:10px;" |16
  | style="background-color:white; padding-left:10px; padding-right:10px;" |87.5% (64.0–97.8)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |..
 |-
  | colspan="8" style="background-color:white; padding-left:10px; padding-right:10px;" |'''April 2020'''
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Overall
  | style="background-color:white; padding-left:10px; padding-right:10px;" |23
  | style="background-color:white; padding-left:10px; padding-right:10px;" |82.6% (62.9–93.0)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |1.0
  | style="background-color:white; padding-left:10px; padding-right:10px;" |23
  | style="background-color:white; padding-left:10px; padding-right:10px;" |60.9% (40.8–77.8)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |1.0
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <25
  | style="background-color:white; padding-left:10px; padding-right:10px;" |16
  | style="background-color:white; padding-left:10px; padding-right:10px;" |100.0% (80.6–100)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |1.0
  | style="background-color:white; padding-left:10px; padding-right:10px;" |16
  | style="background-color:white; padding-left:10px; padding-right:10px;" |81.2% (57.0–93.4)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |1.0
 |-
  | colspan="8" style="background-color:white; padding-left:10px; padding-right:10px;" |'''September 2020'''
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Overall
  | style="background-color:white; padding-left:10px; padding-right:10px;" |23
  | style="background-color:white; padding-left:10px; padding-right:10px;" |82.6% (62.9–93.0)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |1.0
  | style="background-color:white; padding-left:10px; padding-right:10px;" |23
  | style="background-color:white; padding-left:10px; padding-right:10px;" |73.9% (53.5–87.4)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |0.75
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |N Ct value <25
  | style="background-color:white; padding-left:10px; padding-right:10px;" |16
  | style="background-color:white; padding-left:10px; padding-right:10px;" |100.0% (80.6–100)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |1.0
  | style="background-color:white; padding-left:10px; padding-right:10px;" |16
  | style="background-color:white; padding-left:10px; padding-right:10px;" |100.0% (80.6–100.0)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |0.48
 |-
|}
|}

==Discussion==
By extensive head-to-head comparison, we found that most rapid antigen tests performed to a high standard, with good sensitivity and excellent specificity. Consistent with previous reports, the tests delivered an overall sensitivity of 65.0–89.0% in comparison with qRT-PCR<ref>{{Cite journal |last=Drevinek |first=Pavel |last2=Hurych |first2=Jakub |last3=Kepka |first3=Zdenek |last4=Briksi |first4=Ales |last5=Kulich |first5=Michal |last6=Zajac |first6=Miroslav |last7=Hubacek |first7=Petr |date=2020-11-24 |title=The sensitivity of SARS-CoV-2 antigen tests in the view of large-scale testing |url=http://medrxiv.org/lookup/doi/10.1101/2020.11.23.20237198 |language=en |doi=10.1101/2020.11.23.20237198}}</ref><ref name=":10">{{Cite journal |last=Peto |first=Tim |last2=Affron |first2=Dominic |last3=Afrough |first3=Babak |last4=Agasu |first4=Anita |last5=Ainsworth |first5=Mark |last6=Allanson |first6=Alison |last7=Allen |first7=Katherine |last8=Allen |first8=Collette |last9=Archer |first9=Lorraine |last10=Ashbridge |first10=Natasha |last11=Aurfan |first11=Iman |date=2021-06 |title=COVID-19: Rapid antigen detection for SARS-CoV-2 by lateral flow assay: A national systematic evaluation of sensitivity and specificity for mass-testing |url=https://linkinghub.elsevier.com/retrieve/pii/S2589537021002042 |journal=EClinicalMedicine |language=en |volume=36 |pages=100924 |doi=10.1016/j.eclinm.2021.100924 |pmc=PMC8164528 |pmid=34101770}}</ref><ref>{{Cite journal |last=Corman |first=Victor M |last2=Haage |first2=Verena Claudia |last3=Bleicker |first3=Tobias |last4=Schmidt |first4=Marie Luisa |last5=Mühlemann |first5=Barbara |last6=Zuchowski |first6=Marta |last7=Jo |first7=Wendy K |last8=Tscheak |first8=Patricia |last9=Möncke-Buchner |first9=Elisabeth |last10=Müller |first10=Marcel A |last11=Krumbholz |first11=Andi |date=2021-07 |title=Comparison of seven commercial SARS-CoV-2 rapid point-of-care antigen tests: a single-centre laboratory evaluation study |url=https://linkinghub.elsevier.com/retrieve/pii/S2666524721000562 |journal=The Lancet Microbe |language=en |volume=2 |issue=7 |pages=e311–e319 |doi=10.1016/S2666-5247(21)00056-2 |pmc=PMC8026170 |pmid=33846704}}</ref><ref>{{Cite journal |last=Jääskeläinen |first=A.E. |last2=Ahava |first2=M.J. |last3=Jokela |first3=P. |last4=Szirovicza |first4=L. |last5=Pohjala |first5=S. |last6=Vapalahti |first6=O. |last7=Lappalainen |first7=M. |last8=Hepojoki |first8=J. |last9=Kurkela |first9=S. |date=2021-04 |title=Evaluation of three rapid lateral flow antigen detection tests for the diagnosis of SARS-CoV-2 infection |url=https://linkinghub.elsevier.com/retrieve/pii/S1386653221000524 |journal=Journal of Clinical Virology |language=en |volume=137 |pages=104785 |doi=10.1016/j.jcv.2021.104785 |pmc=PMC7934791 |pmid=33711694}}</ref>, increasing to greater than 90% for most tests for samples with Ct values of less than 25.<ref name=":10" /><ref>{{Cite journal |last=Kohmer |first=Niko |last2=Toptan |first2=Tuna |last3=Pallas |first3=Christiane |last4=Karaca |first4=Onur |last5=Pfeiffer |first5=Annika |last6=Westhaus |first6=Sandra |last7=Widera |first7=Marek |last8=Berger |first8=Annemarie |last9=Hoehl |first9=Sebastian |last10=Kammel |first10=Martin |last11=Ciesek |first11=Sandra |date=2021/1 |title=The Comparative Clinical Performance of Four SARS-CoV-2 Rapid Antigen Tests and Their Correlation to Infectivity In Vitro |url=https://www.mdpi.com/2077-0383/10/2/328 |journal=Journal of Clinical Medicine |language=en |volume=10 |issue=2 |pages=328 |doi=10.3390/jcm10020328 |pmc=PMC7830733 |pmid=33477365}}</ref><ref>{{Cite journal |last=Young |first=Stephen |last2=Taylor |first2=Stephanie N. |last3=Cammarata |first3=Catherine L. |last4=Varnado |first4=Katey G. |last5=Roger-Dalbert |first5=Celine |last6=Montano |first6=Amanda |last7=Griego-Fullbright |first7=Christen |last8=Burgard |first8=Cameron |last9=Fernandez |first9=Catherine |last10=Eckert |first10=Karen |last11=Andrews |first11=Jeffrey C. |date=2020-12-17 |editor-last=Miller |editor-first=Melissa B. |title=Clinical Evaluation of BD Veritor SARS-CoV-2 Point-of-Care Test Performance Compared to PCR-Based Testing and versus the Sofia 2 SARS Antigen Point-of-Care Test |url=https://journals.asm.org/doi/10.1128/JCM.02338-20 |journal=Journal of Clinical Microbiology |language=en |volume=59 |issue=1 |pages=e2338-e2420 |doi=10.1128/JCM.02338-20 |issn=0095-1137 |pmc=PMC7771450 |pmid=33023911}}</ref> Among the Innova, Encode, and SureScreen-F tests, sensitivity increased to at least 95% when compared with samples that were infectious ''in vitro'', with viral titer in the specimens correlating with Ct values. 

This study, to our knowledge, provides the most comprehensive comparison of antigen LFDs and infectivity to date, showing a clear relationship between Ct values, quantitative culture of infectious virus, and antigen LFD positivity, with the tests used in each analysis delivering reliable identification of infectious clinical samples. The positive samples included in this study were representative of those encountered by the diagnostic laboratory at St. Thomas' Hospital, including from healthcare workers and their household contacts, asymptomatic individuals undergoing unrelated hospital treatment, and individuals with mild to severe COVID-19. Given the breadth of samples, we expect the findings from this study to apply to a wide range of infection scenarios.

In agreement with previous studies, we cultured virus from upper respiratory tract specimens with Ct values of up to 26 (an equivalent viral load of 7 × 10<sup>5</sup> RNA copies per mL), with most of the culturable samples taken in the first week after symptom onset.<ref name=":7" /><ref name=":8" /><ref name=":9" /><ref>{{Cite journal |last=Wölfel |first=Roman |last2=Corman |first2=Victor M. |last3=Guggemos |first3=Wolfgang |last4=Seilmaier |first4=Michael |last5=Zange |first5=Sabine |last6=Müller |first6=Marcel A. |last7=Niemeyer |first7=Daniela |last8=Jones |first8=Terry C. |last9=Vollmar |first9=Patrick |last10=Rothe |first10=Camilla |last11=Hoelscher |first11=Michael |date=2020-05 |title=Virological assessment of hospitalized patients with COVID-2019 |url=https://www.nature.com/articles/s41586-020-2196-x |journal=Nature |language=en |volume=581 |issue=7809 |pages=465–469 |doi=10.1038/s41586-020-2196-x |issn=1476-4687}}</ref><ref name=":11">{{Cite journal |last=Cevik |first=Muge |last2=Tate |first2=Matthew |last3=Lloyd |first3=Ollie |last4=Maraolo |first4=Alberto Enrico |last5=Schafers |first5=Jenna |last6=Ho |first6=Antonia |date=2021-01 |title=SARS-CoV-2, SARS-CoV, and MERS-CoV viral load dynamics, duration of viral shedding, and infectiousness: a systematic review and meta-analysis |url=https://linkinghub.elsevier.com/retrieve/pii/S2666524720301725 |journal=The Lancet Microbe |language=en |volume=2 |issue=1 |pages=e13–e22 |doi=10.1016/S2666-5247(20)30172-5 |pmc=PMC7837230 |pmid=33521734}}</ref> The minimum viral titer required for transmission is unclear<ref name=":11" /> and will depend in part on the proximity and duration of contact. Nevertheless, higher viral loads, as measured by lower Ct, have been strongly associated with transmission<ref name=":6" />, and, therefore, a reasonable assumption is that the quantity of cultured virus ''in vitro'' has a similar correlation with infectivity. However, problems arise when attempting to standardize Ct values as surrogate measures of transmission potential, due to differences in RNA extraction and RT-PCR methods. This issue was indicated in a recent study<ref name=":4" />, in which differences in Ct value of greater than 5 between RT-PCR systems prompted concerns that LFDs were missing up to 50% of potentially infectious cases.<ref name=":4" /><ref name=":0" /><ref name=":5" /> There have been frequent suggestions for SARS-CoV-2 results to be presented as viral load (RNA copies per mL) due to difficulties in comparing Ct values between studies. However, the absence of an agreed upon standard for determining viral load is itself a problem, with reported viral loads often appearing even more disparate than Ct values.<ref>{{Cite journal |last=Han |first=Mi Seon |last2=Byun |first2=Jung-Hyun |last3=Cho |first3=Yonggeun |last4=Rim |first4=John Hoon |date=2021-02 |title=RT-PCR for SARS-CoV-2: quantitative versus qualitative |url=https://linkinghub.elsevier.com/retrieve/pii/S1473309920304242 |journal=The Lancet Infectious Diseases |language=en |volume=21 |issue=2 |pages=165 |doi=10.1016/S1473-3099(20)30424-2 |pmc=PMC7239624 |pmid=32445709}}</ref>

Although the reduced sensitivity of LFDs relative to PCR is less of a concern late in the infection course when Ct values are increasing and the risk of onward transmission is negligible<ref name=":6" />, reduced sensitivity can be problematic during the presymptomatic or early asymptomatic phase of infection.<ref name=":14" /> As shown by our longitudinal studies and our data suggesting early timepoints with higher Ct values give negative results with LFD, an individual can be positive by PCR but negative according to antigen testing for one or two days before testing positive. A negative result delivered at this stage in the course of infection could offer false security to someone who is about to become highly infectious. Furthermore, with the time window of positive results narrower than for PCR testing, relatively small differences in test sensitivities could translate to capturing or missing potentially infectious cases. We, therefore, recommend that regular testing be emphasized, and that tests are deployed in populations in which the limitations of these tests are understood and manageable.

In particular inpatient situations, LFDs can also be used to make early, rapid decisions about patient management, with appropriate isolation pending confirmatory PCR testing. This approach has been successful in hospital LFD pilot studies, with the use of such devices preventing the cohorting of asymptomatic and infectious individuals with uninfected patients while awaiting PCR results (unpublished). Towards the end of the disease course, LFDs could also be useful for determining if persistently PCR-positive individuals pose a transmission risk, potentially in tandem with rapid antibody testing.<ref>{{Cite journal |last=Agarwal |first=Vineet |last2=Venkatakrishnan |first2=A. J. |last3=Puranik |first3=Arjun |last4=Kirkup |first4=Christian |last5=Lopez-Marquez |first5=Agustin |last6=Challener |first6=Douglas W. |last7=Theel |first7=Elitza S. |last8=O’Horo |first8=John C. |last9=Binnicker |first9=Matthew J. |last10=Kremers |first10=Walter K. |last11=Faubion |first11=William A. |date=2020-12 |title=Long-term SARS-CoV-2 RNA shedding and its temporal association to IgG seropositivity |url=http://www.nature.com/articles/s41420-020-00375-y |journal=Cell Death Discovery |language=en |volume=6 |issue=1 |pages=138 |doi=10.1038/s41420-020-00375-y |issn=2058-7716 |pmc=PMC7709096 |pmid=33298894}}</ref>

Although the LFDs are easy to use, the correct sampling, reading, and interpretation of the result are essential to their success in mass screening situations<ref name=":12" /> In particular, mass screening programs need to consider training and familiarity with swabbing when deploying devices to the general public as compared with a trained healthcare worker in a hospital or clinical setting. It is also easy to underestimate the importance of correctly recognizing a positive band. We found that some tests gave clearer results than others, which was the rationale for presenting our results as heatmaps, and is illustrated by the number of weak positive results we observed. Removing this element of subjectivity, for example with use of a smartphone application to read or capture the LFD result, could improve success rates.

Our comparative studies were done with VTM-stored swabs used for routine SARS-CoV-2 diagnostic testing, to accurately reflect the broad range of typical cases encountered and to conduct the detailed viral infectivity studies that were central to this study. However, the LFDs are intended to be used with direct swabs and, therefore, our approach introduced a pre-dilution step, and potentially underestimated the sensitivity of the LFDs. Conversely, self-administration of swabs in community testing could also affect sensitivity. Future studies would benefit from a direct comparison between self-administered dry swabs, VTM-stored swabs, and RT-PCR.

A further point for consideration is the need for continued reassessment of LFDs in the context of emerging new variants of SARS-CoV-2. Most LFDs use monoclonal antibodies targeting the SARS-CoV-2-N protein, and amino acid mutations in the N protein have been documented for notable variants, such as those first identified in the U.K. (B.1.1.7), Brazil (P1), South Africa (B.1.351), and India (B.1.617).<ref>{{Cite journal |last=Plante |first=Jessica A. |last2=Mitchell |first2=Brooke M. |last3=Plante |first3=Kenneth S. |last4=Debbink |first4=Kari |last5=Weaver |first5=Scott C. |last6=Menachery |first6=Vineet D. |date=2021-04 |title=The variant gambit: COVID-19’s next move |url=https://linkinghub.elsevier.com/retrieve/pii/S1931312821000998 |journal=Cell Host & Microbe |language=en |volume=29 |issue=4 |pages=508–515 |doi=10.1016/j.chom.2021.02.020 |pmc=PMC7919536 |pmid=33789086}}</ref> With the potential for accumulating mutations to occur in emerging variants, alongside possible differences in infectivity and tranmissibility, consistency in sensitivity for a given test should not be assumed.

Our data support the judicious use of LFDs for rapid antigen detection: not to replace PCR testing, but to supplement current testing capacity and rapidly identify infected individuals in situations in which they would otherwise go undetected. Although sensitivity is lower than with PCR-based testing, the rapid turnaround of these LFD tests, their versatility in terms of cost and portability, and their utility in disrupting transmission chains<ref name=":13" /><ref>{{Cite journal |last=Larremore |first=Daniel B. |last2=Wilder |first2=Bryan |last3=Lester |first3=Evan |last4=Shehata |first4=Soraya |last5=Burke |first5=James M. |last6=Hay |first6=James A. |last7=Tambe |first7=Milind |last8=Mina |first8=Michael J. |last9=Parker |first9=Roy |date=2021-01 |title=Test sensitivity is secondary to frequency and turnaround time for COVID-19 screening |url=https://www.science.org/doi/10.1126/sciadv.abd5393 |journal=Science Advances |language=en |volume=7 |issue=1 |at=eabd5393 |doi=10.1126/sciadv.abd5393 |issn=2375-2548}}</ref> originating from infectious asymptomatic individuals could outweigh the risk of missing positive cases.

==Supplementary material==

*[https://ars.els-cdn.com/content/image/1-s2.0-S2666524721001439-mmc1.pdf Supplementary appendix] (PDF)

==Acknowledgements==
We are thankful to Spring Healthcare, SureScreen Diagnostics, Zhuhai Encode Medical Engineering, Innova Medical Group, and E25Bio for donating test kits. We are also thankful to the Biostatistics and Data Management Platform of the National Institute for Health Research (NIHR) Guy's and St. Thomas' Biomedical Research Centre for providing guidance on statistical analyses. We are extremely grateful to all patients and staff at St. Thomas' Hospital who participated in this study. This research was supported by the U.K. Department of Health via an NIHR Comprehensive Biomedical Research Centre award to Guy's and St. Thomas' NHS Foundation Trust in partnership with King's College London and King's College Hospital NHS Foundation Trust. SP was supported by a Huo Family Foundation award and Wellcome Trust Senior Fellowship (WT098049AIA) granted to SJDN. MHM, KJD, SJDN, and RPG were supported by King's Together Rapid COVID-19 call awards. MHM was supported by a Wellcome Trust award (106223/Z/14/Z). SJDN, KJD, and MHM were supported by a UK Medical Research Council Discovery award (MC/PC/15068). MHM, KJD, and SJDN were supported by a Huo Family Foundation award.

===Contributions===
SP, RB, LBS, BM, GN, SD, MHM, JDE, SJDN, and RPG conceived and designed the study. SP, RB, BM, LBS, MTKI, TC, AA-M, and RPG collated the data. LBS, BM, TC, AA-M, GN, and SD supervised specimen collection. AP, BP, TC, PRC, EC, and JM supervised data collection. MJL and KJD generated reagents. SP, RPG, and FR analysed the data. SP, SJDN, and RPG wrote the first draft of the manuscript. All authors had full access to all the data in the study and had final responsibility for the decision to submit for publication. SP and RPG accessed and verified the data. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

===Funding===
King's Together Rapid COVID-19, Medical Research Council, Wellcome Trust, Huo Family Foundation, UK Department of Health, National Institute for Health Research Comprehensive Biomedical Research Centre.

===Data sharing===
All relevant data are within the manuscript and supplementary appendix.

===Competing interests===
We declare no competing interests.

==References==
{{Reflist|colwidth=30em}}

==Notes==
This presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added.

<!--Place all category tags here-->
[[Category:LIMSwiki journal articles (added in 2021)]]
[[Category:LIMSwiki journal articles (all)]]
[[Category:LIMSwiki journal articles on clinical research]]
[[Category:LIMSwiki journal articles on epidemiology]]