Performance of Lateral Flow Assays for SARS-CoV-2 compared to RT-qPCR

Published:January 13, 2022DOI:https://doi.org/10.1016/j.jinf.2022.01.013

      Keywords

      Dear Editor
      We have read with interest the study by Lamb et al. on the real-world performance of a lateral flow immunoassay antigen device (LFD) for regular COVID-19 testing of hospital workers
      • Lamb G.
      Real-world evaluation of COVID-19 lateral flow device (LFD) mass-testing in healthcare workers at a London hospital; a prospective cohort analysis.
      . LFDs have found widespread application for broad-range screening, with subsequent confirmation of LFA-positive cases by RT-qPCR. While, in practice, an acceptably low number of positive LFD results turned out as false positives when validated with RT-qPCR, there is accumulating evidence that the number of positive cases missed, i.e., false negative results obtained with LFDs is unacceptably high
      • Armstrong S.
      Covid-19: Tests on students are highly inaccurate, early findings show.
      . The LFD used in the study by Lamb et al. (Innova Medical Group, USA) had a high positive predictive value (PPV) when frequently used during periods of high prevalence of COVID-19. As the authors did not use the same sample for confirmation by RT-qPCR, however, it was not possible to directly link the test outcome to a certain cT value, or to evaluate the specificity
      • Lamb G.
      Real-world evaluation of COVID-19 lateral flow device (LFD) mass-testing in healthcare workers at a London hospital; a prospective cohort analysis.
      .
      We selected 10 common LFDs [Abbott (Panbio, Abbott Rapid Diagnostics Jena GmbH, Jena, Germany), Acon (Flowflex, Acon Laboratories Inc., CA, USA), Clungene (Hangzhou Clungene Biotech Co., Ltd., Tianjin, China), Joysbio (Joysbio (Tianjin) Biotechnology Co., Ltd., Tianjin, China), Lepu Medical (Beijing Lepu Medical Technology Co., Ltd., Peking, China), Nadal (nal von minden GmbH, Moers, Germany), Orient (Zhejiang Orient Gene Biotech Co., Ltd., Zhejiang, China), Realy Tech (Hangzhou Realy Tech Co., Ltd., Zhejiang, China), Roche (SD Biosensors, Gyeonggi-do, Republic of Korea), Siemens (Clinitest, Healgen Scientific Ltd., TX, USA)] and evaluated their performance using samples assigned for routine RT-qPCR screening of patients at the local university hospital between May and November 2021.
      Figure 1
      Figure 1
      Figure 1illustrates the predicted LFD results based on cT values by RT-qPCR. The grey lines and areas indicate the predicted probability of a positive outcome and the 95% confidence derived from the logistic regression of all samples (imputed cT-value for PCR-negative samples = 40.0). The red line indicates the predicted probabilities of PCR-positive samples only. Bars on the top and on the bottom indicate positive and negative LFD test results, respectively. Red bars indicate false results, while correct results are indicated in blue.
      Table 1
      Table 1shows the cT thresholds at which 50% and 95% of the LFD test results become positive; in case of a50 and a95 all samples (imputed cT-value for PCR-negative samples = 40), in case of p50 and p95 only positive samples were considered. Brackets indicate the limit of the lower and upper 95% confidence interval.
      LFATPTNFPFNa50a95p50p95
      Abott7638154332.1 [30.0; 34.3]14.4 [14.0; NA]30.2 [28.3; 32.5]18.4 [14.7; NA]
      Acon395508023.5 [22.6; 24.4]19.9 [18.4; 21.7]23.5 [22.6; 24.4]19.9 [18.4; 21.7]
      Clungene575405826.4 [25.4; 27.5]21.0 [19.3; 23.1]26.4 [25.4; 27.5]20.9 [19.1; 23.0]
      Joysbio355418422.8 [21.6; 23.8]16.2 [14.1; NA]22.8 [21.8; 23.8]18.0 [15.4; 20.4]
      Lepu534876625.7 [23.2; 27.8]NA25.4 [23.4; 27.6]NA
      Nadal9424312540.0 [NA; NA]NA40.0 [34.5; NA]NA
      Orient695415028.3 [26.6; 30.0]16.3 [14.1; NA]29.0 [26.9; 31.7]14.0 [13.9; NA]
      Realy.Tech594966027.4 [25.7; 29.0]14.3 [13.9; NA]26.7 [25.4; 28.3]17.6 [14.5; NA]
      Roche624965727.9 [26.3; 29.6]15.6 [14.0; NA]27.2 [25.8; 28.7]18.6 [15.4; 21.7]
      Siemens535506625.7 [24.5; 26.9]19.0 [16.6; 21.6]25.7 [24.5; 26.9]18.9 [16.2; 21.5]
      All clinical specimens were anonymized nasopharyngeal swabs in universal transport medium (119 positive and 55 negative) performed by trained personnel. All tests were performed in parallel using aliquots of the same samples. Samples were stored for a maximum of 48 h at 4°C. The distribution of the cT-values of our samples was as follows: cT < 20, n = 16; 20-25, n = 31; 25-30, n = 31; 30-35; n = 29; >35; n = 12. Highest sensitivity was achieved by Nadal (79 %), which however yielded the highest number of false positive results (31) and hence the lowest specificity (44 %). Comparatively high sensitivity was achieved by Abbott (62 %) and Orient (56 %). Lowest sensitivity was achieved by Joysbio (33%) and Acon (33%). In the subset of samples with cT-values of < 25, sensitivity ranged from 68% to 91% (Nadal 94%). Siemens, Clungene and Acon showed no false positive results.
      Binomial logistic regression analysis was applied to model the probability of obtaining a positive LFD result depending on the cT-value assuming 40 for PCR-negative tests or neglecting negative test results. We estimated the confidence intervals (CI) via nonparametric bootstrapping with 100.000 iterations.
      In accordance with Lamb et al. and similar other studies, our study confirms that LFDs are suitable to identify potential super-spreaders with low cT values and to prevent large cluster formation, e.g., in hospitals
      • Lamb G.
      Real-world evaluation of COVID-19 lateral flow device (LFD) mass-testing in healthcare workers at a London hospital; a prospective cohort analysis.
      ,
      • Avadhanula V.
      Viral Load of Severe Acute Respiratory Syndrome Coronavirus 2 in Adults During the First and Second Wave of Coronavirus Disease 2019 Pandemic in Houston, Texas: The Potential of the Superspreader.
      ,

      Krone, M., Noffz, A., Richter, E., Vogel, U. & Schwab, M. Control of a COVID-19 outbreak in a nursing home by general screening and cohort isolation in Germany, March to May 2020. Eurosurveillance26, 2001365 (2021).

      , however, regardless of the manufacturer, they are not likely to contribute significantly to control the infection dynamics caused by carriers of low viral load or by asymptomatic individuals, including those at early stages of infection.
      It should be noted that cT values can vary among clinical laboratories, depending on the method for RNA extraction and the PCR kits used. As an example, in a study involving 123 participating certified laboratories in Austria including our lab, a variation in cT values of ± 4.7 cycles was observed
      • Buchta C.
      A look at the precision, sensitivity and specificity of SARS-CoV-2 RT-PCR assays through a dedicated external quality assessment round.
      .
      Samples in our study were exclusively obtained via nasopharyngeal swabs by professionals, whereas other techniques, such as the common anterior nasal swab or incorrect swabbing by non-trained personal may significantly decrease in vitro virus concentration, and thus can increase the cT-values by several orders of magnitude
      • Hanson K.E.
      Self-Collected Anterior Nasal and Saliva Specimens versus Health Care Worker-Collected Nasopharyngeal Swabs for the Molecular Detection of SARS-CoV-2.
      • Richard-Greenblatt M.
      Quantifying the Impact of Nasopharyngeal Specimen Quality on Severe Acute Respiratory Syndrome Coronavirus 2 Test Performance.
      • Zhou Y.
      • O'Leary T.J
      Relative sensitivity of anterior nares and nasopharyngeal swabs for initial detection of SARS-CoV-2 in ambulatory patients: Rapid review and meta-analysis.
      leading to a further loss of sensitivity. In the light of the repeated emergence of SARS-CoV-2 variants, one may also be worried about decreased sensitivity due to altered surface proteins, which are recognized be the LFD antibodies
      • Karim S.S.A.
      • Karim Q.A.
      Omicron SARS-CoV-2 variant: a new chapter in the COVID-19 pandemic.
      .
      Based on our data, we conclude that LFDs cannot be recommended for general broad-range screening for SARS-CoV-2 infection in an asymptomatic population. At high infection dynamics however, insufficient PCR test capacities lead to logistic difficulties, an increased time-to-result, as well as decreased sensitivity related to sample pooling. In such settings, the short time-to-result of LFDs may support the timely identification and isolation of individuals with high viral load to dampen the dynamics of spread, as suggested by Lamb et al.
      • Lamb G.
      Real-world evaluation of COVID-19 lateral flow device (LFD) mass-testing in healthcare workers at a London hospital; a prospective cohort analysis.
      .

      Declaration of Competing Interest

      The authors have no conflict of interest to declare.

      Author contributions

      MP, SH and JH conceived the study and designed methodology; ME and MW provided clinical supervision of the study. CB, JZ, TE and SH collected the data; MP, RE and JH analysed the data; MP, SH and VW led the writing of the manuscript. All authors contributed critically to the drafts and gave final approval for publication.

      Funding

      This study was funded by the Danube University Krems and the Regional Government of Lower Austria.

      Ethical statement

      The study was approved by the Ethics Committee of the Danube University Krems on May 28th, 2021.

      Appendix. Supplementary materials

      References

        • Lamb G.
        Real-world evaluation of COVID-19 lateral flow device (LFD) mass-testing in healthcare workers at a London hospital; a prospective cohort analysis.
        J. Infect. 2021; 83: 452-457
        • Armstrong S.
        Covid-19: Tests on students are highly inaccurate, early findings show.
        BMJ. 2020; 371: m4941
        • Avadhanula V.
        Viral Load of Severe Acute Respiratory Syndrome Coronavirus 2 in Adults During the First and Second Wave of Coronavirus Disease 2019 Pandemic in Houston, Texas: The Potential of the Superspreader.
        J. Infect. Dis. 2021; 223: 1528-1537
      1. Krone, M., Noffz, A., Richter, E., Vogel, U. & Schwab, M. Control of a COVID-19 outbreak in a nursing home by general screening and cohort isolation in Germany, March to May 2020. Eurosurveillance26, 2001365 (2021).

        • Buchta C.
        A look at the precision, sensitivity and specificity of SARS-CoV-2 RT-PCR assays through a dedicated external quality assessment round.
        Clin. Chem. Lab. Med. 2021; https://doi.org/10.1515/cclm-2021-1004
        • Hanson K.E.
        Self-Collected Anterior Nasal and Saliva Specimens versus Health Care Worker-Collected Nasopharyngeal Swabs for the Molecular Detection of SARS-CoV-2.
        J. Clin. Microbiol. 2020; 58 (e01824-20)
        • Richard-Greenblatt M.
        Quantifying the Impact of Nasopharyngeal Specimen Quality on Severe Acute Respiratory Syndrome Coronavirus 2 Test Performance.
        Open Forum Infect. Dis. 2021; 8 (ofab235)
        • Zhou Y.
        • O'Leary T.J
        Relative sensitivity of anterior nares and nasopharyngeal swabs for initial detection of SARS-CoV-2 in ambulatory patients: Rapid review and meta-analysis.
        PLOS ONE. 2021; 16e0254559
        • Karim S.S.A.
        • Karim Q.A.
        Omicron SARS-CoV-2 variant: a new chapter in the COVID-19 pandemic.
        The Lancet. 2021; 398: 2126-2128