We note the recent press speculation around the UK-RTC AbC-19TM assay and its performance. We are grateful to the authors for providing access to their data in the BMJ paper which we have reviewed.
We believe the press articles and related BMJ report puts a spotlight on the different types of antibody testing available and what their applications are, and we very much welcome this.
Our AbC-19TM IgG lateral flow assay is targeted at IgG antibodies to the full spike protein of SARS-CoV-2 (COVID19) to indicate a measure of immune response (aligned with neutralizing antibodies) and differs from other antibody tests, some of which our assay has been compared against (incorrectly in our view). Assays measuring IgM and IgG to the nucleocapsid (as used in this study) may be useful to show previous infection but have no place in understanding the levels of immunity producing antibodies (this requires measurement of IgG to Spike) as generated naturally or via vaccination.
We do have some concerns with the BMJ research report and modelling methodology, and we provide details below. However, the Department for Health & Social Care (DHSC) reviewed in detail the Public Health England (PHE) report on the UK-RTC AbC-19TM assay prior to ordering kits and has provided the following statement:
“This report shows these tests are approved for use in surveillance studies, which is what they were purchased for. “They were never intended for, and have never been issued for widespread public use and it is misleading and unnecessarily inflammatory to purposefully ignore this fact in the report. “This robust evaluation was carried out by PHE at the Department’s request before any purchase was made, and PHE approved the test for use in surveillance studies.”
Our customer, DHSC, is satisfied with the performance of the test, as is the UK-RTC, and will continue to use and roll out the use of the product. Public Health England conducted an evaluation at the DHSC's request, prior to DHSC purchasing the product and will continue to deploy the test for use in surveillance studies.
With regards to details of the report:
1. When using methods as used in previous evaluations by PHE1, results for the AbC-19TM IgG test of 92.5% sensitivity and 97.9%* specificity are in line with those other tests. (This rose to *99.2% specificity when queried samples were repeated. *This increase in specificity relates to 42 positives out of 1995 presumed negative samples (collected in 2016-2017) which drop to 16 positives on repeat testing).
2. An initial study2 performed at Ulster University used methods as defined by Medicines and Healthcare products Regulatory Agency (MHRA)3. Guidance from MHRA stipulated a target product profile should determine the sensitivity and specificity of antibody tests by using at least 200 samples confirmed as positive for antibodies and at least 200 samples confirmed as negative for antibodies to SARS-CoV-2. The Ulster study fulfilled this requirement and undertook a laboratory evaluation of AbC-19TM to detect SARS-CoV-2 IgG in a highly characterised cohort of known antibody positive and known antibody negative samples. This showed performance of the assay, in this well characterised known population, to give sensitivity of 97.9% and specificity of 100%. The study has been expanded so that it now includes additional samples which results in sensitivity of 97.6% and specificity of 99.59%2.
3. We note that PCR positive individuals do not always seroconvert4 to produce IgG, and would therefore give a negative IgG result. This is one reason why a test which measures this immune response is important.
4. The title of the paper illustrates the authors misunderstand the purpose of our test – it is not designed to detect “previous infection” but rather to detect the presence of a particular type of antibody: IgG to the full spike protein of the SARS-CoV-2 virus.
5. There is much reliance on modelling in the pandemic, the authors perform a theoretical statistical modelling exercise with “assumed” incidence of SARS-CoV-2 in key workers of 10% whilst their own data (Table 3 of publication) shows incidence in this population of 2847 key workers tested to be 21.5% (613 positive out of 2847) which results in very different (significantly improved) false positive outcome.
6. With regards to methods of analysis, having noted that there is no reference standard for COVID-19 antibody testing, the authors proceed to use an assay with totally different specifications to the AbC-19TM IgG assay as a reference standard. The AbC-19TM assay is designed to detect IgG antibodies to the SARS-CoV-2 full trimeric spike protein: the type of antibodies which are developed as immunity develops and which vaccines are designed to produce5.6. The reference standard chosen; the Roche pan antibody assay to nucleocapsid protein (NP) (Elecsys Anti-SARS-CoV2), is designed to detect IgM and IgG antibodies to a different immunogenic portion of the virus; not to those binding to the spike protein. Assays for the detection of antibodies to nucleocapsid will not be useful to indicate who has generated immunity or who has produced IgG antibodies to the spike protein following vaccination. This is a very important difference and the assays have very different use cases.
7. Those familiar with mass screening will know that low incidence (prevalence) of a condition will result in a higher probability of false positives. The lower the incidence, the greater the potential for false positive results – it is a simple statistical fact. The standard practice in this situation is to perform additional testing on a positive population, known as orthogonal testing or confirmatory testing7. This standard approach would be suitable here to confirm a presence of an adequate immune response.
1. Public Health England. COVID-19: laboratory evaluations of serological assays. https://www.gov.uk/government/publications/covid-19-laboratory-evaluations-of-serological-assays (last updated 25 August 2020). Accessed 12 November 2020.
2. Robertson et al. SARS-CoV-2 antibody testing in a UK population: detectable IgG for up to 20 weeks post infection. https://www.medrxiv.org/content/10.1101/2020.09.29.20201509v2 doi: https://doi.org/10.1101/2020.09.29.20201509.
3. Target Product Profile: antibody tests to help determine if people have recent infection to SARS-CoV-2: Version 2 (Updated 15 October 2020) https://www.gov.uk/government/publications/how-tests-and-testing-kits-for-coronavirus-covid-19-work/target-product-profile-antibody-tests-to-help-determine-if-people-have-recent-infection-to-sars-cov-2-version-2. Accessed 12 November 2020.
4. Staines, HM et al. Dynamics of IgG seroconversion and pathophysiology of COVID-19 infections. https://www.medrxiv.org/content/10.1101/2020.06.07.20124636v2 Accessed 12 November 2020.
5. Pfizer Vaccines. Behind the Science: What is an mRNA Vaccine. (25 August 2020) https://www.pfizer.co.uk/behind-science-what-mrna-vaccine. Accessed 12 November 2020.
6. University of Oxford. About the Oxford COVID-19 vaccine. https://www.research.ox.ac.uk/Article/2020-07-19-the-oxford-covid-19-vaccine. (19th July 2020). Accessed 12 November 2020.
7. Tyler J. Ripperge. Orthogonal SARS-CoV-2 Serological Assays Enable Surveillance of Low Prevalence Communities and Reveal Durable Humoral Immunity. Immunity (2020), doi: https://doi.org/10.1016/j.immuni.2020.10.004