Skip to main content
  • Research article
  • Open access
  • Published:

High-volume workflow and performance comparisons for Chlamydia trachomatis and Neisseria gonorrhoeae testing using automated molecular platforms

Abstract

Background

The global burden of sexually transmitted infections (STIs) is high and there have been reports of increasing chlamydial and gonorrheal infections. High-volume screening programs for Chlamydia trachomatis (CT) and Neisseria gonorrhoeae (NG) are an important component of STI control. This study evaluated the high-volume workflow and performance of the cobas® CT/NG assay for use on the automated Roche cobas® 6800 system, with the cobas p 480 instrument for pre-analytics, compared with the Aptima Combo 2 assay on the Hologic Panther system.

Methods

High-volume workflow and performance were evaluated using paired female urine specimens. Workflow analysis (n = 376) included hands-on time (HoT), number of manual interventions, and time to first and last results. For performance assessment, paired results from the cobas CT/NG and Aptima Combo 2 assays, for both CT and NG, were compared and two-sided 95% confidence intervals calculated to provide estimates of positive percent agreement (PPA), negative percent agreement (NPA), and overall percent agreement (OPA) between the tests. McNemar’s test was used for significance testing.

Results

Pre-analytical preparations and system start-up on the cobas 6800 system required 00:27:38 (hr:min:sec) HoT whilst the Panther system required 00:30:43. The cobas 6800 system required eight interactions and 00:43:59 HoT to process 376 samples. The Panther system required six interactions and 00:39:10 HoT. Time to first results was 02:53:00 on the cobas c6800 system for 96 samples and 03:28:29 on the Panther system for five samples. The cobas 6800 system delivered all 376 results 3 h faster than the Panther system (07:45:26 and 10:47:30, respectively). The performance correlation between both assays was high (PPA, NPA and OPA > 99% for both CT and NG). McNemar’s test revealed no statistically significant difference between the assays.

Conclusion

For high-volume automated CT/NG testing, both the cobas 6800 system and Panther system provided accurate results. Although less manual intervention steps were needed for the Panther system, improved turnaround time was obtained with the cobas 6800 system with less risk for contamination. The additional testing capacity on the cobas 6800 system would allow a growing service to deliver more results in a single shift.

Peer Review reports

Background

The global burden of sexually transmitted infections (STIs) is high [1]. Recently, data from the USA have identified year-over-year increases in chlamydial and gonorrheal infections [2]. Factors contributing to this include: the targeted expansion of screening practices; wider acceptance of HIV pre-exposure prophylaxis (with the resultant increase in condomless sex); and social networks with increased connectivity and the potential for sexual partners [2,3,4]. In this challenging environment, high-volume screening programs for Chlamydia trachomatis (CT) and Neisseria gonorrhoeae (NG) are an important component of STI control [2, 5,6,7,8].

Nucleic acid amplification tests are highly sensitive and the preferred tests for STI screening and diagnosis [5, 6, 8]. The cobas® CT/NG and Aptima Combo 2 assays are commercially available nucleic acid amplification tests with utility across different specimen types, including female urine [9, 10].

Methods

Study aim and design

The objectives of this study were to compare the cobas CT/NG assay on the cobas® 6800 system, with the Aptima Combo 2 assay on the Panther system, evaluating the high-throughput workflow and assay correlation in female urine specimens. This retrospective study was conducted at a single site (Labor Stein), a laboratory within the Limbach Group, located in Mönchengladbach, Germany, and performed in accordance with relevant local legislation. Remnant de-identified first-catch female urine specimens were collected between March and October 2017. Eligible samples were from symptomatic and asymptomatic individuals (> 18 years of age) presenting for CT/NG testing according to standard practice in North Rhine Westphalia, Germany.

The workflow and analytical performance of two commercially available assays were evaluated on their respective platforms: the cobas CT/NG on the cobas 6800 system with pre-analytics (cobas p 480) for de-capping and re-capping of specimens (Roche Molecular Systems Inc., Pleasanton, California, USA) and the Aptima Combo 2 on the Hologic Panther system (Hologic Inc., Marlborough, Massachusetts, USA). A pre-analytical de-capping, re-capping device was not required for the Panther system, as samples could be directly loaded onto the machine.

Both systems were available at the testing site. All assays were performed in accordance with the manufacturers’ instructions for use [9, 10]. Sample size calculations for the workflow portion of the study were based on the daily throughput of the instruments. Previously tested specimens were used to assess the performance of the cobas CT/NG and Aptima Combo 2 assays. Given that the historical prevalence rate is low at the study site, it was understood that convenience sampling may not be representative of the local test population. Capturing, monitoring, and analyzing of the workflow-related study data were executed by an independent third-party vendor with experience in workflow studies. Operators performing the testing represented the average operator trained to test according to standard procedures and manufacturers’ recommendations.

Clinical specimens

Remnant urine specimens were divided into two aliquots and transferred into the cobas® PCR Urine Sample Kit or the Aptima Urine Specimen Collection Kit. Samples were stored at either 4 °C (for testing with cobas CT/NG) or − 20 °C (for testing with Aptima Combo 2) for a maximum of 8 months.

Workflow (operation and system metrics) and assay agreement

The first 376 specimens collected were used for the workflow evaluation, simulating high-volume throughput in a single day. Testing of the paired cobas and Aptima urine specimens occurred within 1 week of each other. On the first day, specimens were assessed on the cobas 6800 system using the cobas CT/NG assay. On the second day, paired specimens were tested on the Panther system using the Aptima Combo 2 assay. Using standardized data capture forms and detailed interviews with laboratory personnel, the following operation and system metrics were collected: system capabilities (cycle times, throughput, capacity); labor and resourcing requirements (hands-on time [HoT], sample preparation); and total time required to perform all activities for sample testing.

Agreement between the two tests was determined using clinical specimens from both the workflow and additional assay correlation runs (see Additional file 1). Defined positivity cut-offs were pre-specified as per manufacturers’ recommendations. Samples that could not be confidently called positive or negative by either system were excluded from analysis. Results from the routine original Aptima Combo 2 testing were not available to the operators at the time of workflow and assay agreement testing.

Discrepant resolution

Analysis of discrepant specimens, defined as specimens with discordant results between the two tests, included evaluation of the result signal (relative light unit [RLU] for Aptima Combo 2 or cycle threshold [Ct] for cobas CT/NG) and reference to the Aptima Combo 2 result generated during routine testing.

Statistical analysis

No statistical analysis of the workflow data was performed. For performance assessment, paired results from the cobas CT/NG and Aptima Combo 2 assays, for both CT and NG, were compared and two-sided 95% confidence intervals (CIs) calculated to provide estimates of positive percent agreement (PPA), negative percent agreement (NPA), and overall percent agreement (OPA) between the tests. McNemar’s test was used for significance testing. All statistical analyses were performed using SAS software (SAS Institute, Cary, North Carolina, USA).

Results

In total, 606 remnant de-identified female urine specimens were evaluated (n = 376 for workflow studies, n = 606 for method correlation; see Additional file 1). Of these, 50 were excluded from the method correlation analysis because of suspected contamination at the time enrolled specimens were aliquoted for the study (supported by referencing the original Aptima Combo 2 result). Contamination was suspected as high number of initially singularly CT positive samples (tested with the Panther system), collected during the same period, then tested positive for both CT and NG (as tested by both methods). In total 556 specimens were evaluable for method correlation analysis.

Workflow operation and system metrics

Table 1 summarizes the HoT, number of interventions, time to first result, number of specimens at first result, and total turnaround time. The cobas 6800 system required eight manual interactions, comprising the following: 00:08:48 (hr:min:sec) pre-analytics start-up, 00:18:50 molecular system start-up and 00:43:59 ongoing sample processing time (total HoT = 01:11:37). Two manual interactions had the risk for cross-contamination. Once when the de-capped tubes were loaded onto the cobas 6800 system, then once when the de-capped specimens were removed post analysis and placed on the cobas p 480 for re-capping. Cross-contamination at this stage could be problematic if there is a need to re-test samples at a later time, for example to check for additional pathogens.

Table 1 Workflow comparison simulating high-volume CT/NG testing (n = 376)*

For the same workload, the Panther system required six manual interactions, comprising the following: 00:30:43 system start-up and 00:39:10 ongoing sample processing time (total HoT = 01:09:53). Four manual interactions had the risk for cross-contamination. Two of these interactions occurred during reagent preparation and two during the removal of specimens, reagents, and waste.

Assay performance correlation

Paired urine specimens were tested on both the cobas CT/NG and the Aptima Combo 2 assays. Overall agreement between the two assays was > 99% for both CT and NG (Tables 2 and 3). McNemar’s test revealed no statistically significant difference between the assays (p = 1).

Table 2 Assay performance simulating high-volume CT/NG testing (n = 555*), results for Chlamydia trachomatis
Table 3 Assay performance simulating high-volume CT/NG testing (n = 556), results for Neisseria gonorrhoeae*

One specimen was positive by the cobas CT/NG assay and negative by the Aptima Combo 2 for CT (cobas CT/NG Ct = 39.69; Aptima Combo 2 RLU = 12); reference to the original Aptima Combo 2 result confirmed CT positivity (RLU = 479), in agreement with the cobas CT/NG result.

There was also one specimen that was cobas CT/NG positive and Aptima Combo 2 negative for NG (cobas CT/NG Ct = 35.77; Aptima Combo 2 RLU = 616): reference to the original Aptima Combo 2 result confirmed NG negativity (RLU = 983), in accordance with the Aptima Combo 2 result. Of note, this specimen was also positive for CT by both the cobas CT/NG and Aptima Combo 2 assays (cobas CT/NG Ct = 33.70; Aptima Combo 2 RLU = 616). No additional discrepant testing was performed.

Discussion

National screening programs are an important aspect of STI prevention and support the need for high-volume testing systems [2, 5,6,7,8]. The cobas 6800 system and the Panther system are commonly used high-volume automated molecular platforms for routine CT/NG testing. Workflow analysis showed that the cobas 6800 system and the Panther system had similar HoT and number of manual interventions. However, the workflow analysis found the number of manual interventions with risk of a cross-contamination event was fewer with the cobas 6800 system compared with the Panther system (2 versus 4 events), mainly due to the ready-to-load reagents that did not require manual reconstitution and mixing. While the cobas p 480 is needed for the sample de-capping, the ready-to-load reagents of the cobas 6800 system can go directly on the system without pre-analytical preparation in a separate clean area, thus providing workflow efficiencies. The pierceable caps utilized on the Panther system are convenient for direct loading of specimens; however, after assaying, the tubes should be sealed with a barrier for storage [10]. The de-capping and re-capping of specimens with the cobas p 480 mitigates the risk of potential source contamination from direct contact with the foil caps during specimen handling, both pre-analytically in the clinic or in transit, as well as pre-analytically in the laboratory [10].

In other aspects, the cobas 6800 system had specific throughput advantages in comparison with the Panther system. The time to first result was sooner for a larger number of samples and, overall, the results were delivered 3 h (28%) faster on the cobas 6800 system.

Method correlation between the cobas CT/NG and the Aptima Combo 2 assays demonstrated excellent agreement, suggesting that both assays and platforms can be confidently used to screen for infections in a high-throughput setting. Although two specimens generated discrepant results, this was likely due to low titers of the microorganisms in the specimens.

Whilst this study has some limitations, including the use of convenience sampling, it is a useful tool to evaluate workflow differences and will aid decision making in clinical microbiology laboratories seeking to implement high-volume STI screening protocols. Additionally, the cobas 6800 system has more onboard test accessibility than the Panther system (12 polymerase chain reaction cobas assays versus four transcription-mediated amplification Hologic assays, respectively) [11]. Of note, the Panther Fusion system allows for additional menu but does not expand access to the core assays [11]. Advances in high-throughput molecular systems have provided solutions for laboratory consolidation and increasing laboratory services.

Conclusion

The cobas 6800 system and the Panther system both provide improved automation with comparable performance for CT/NG testing. The cobas 6800 system, when utilized with the cobas p 480 pre-analytic system, would allow a growing laboratory service to deliver more results in a single shift. When deciding on automated molecular platforms laboratories will need to balance the needs of their workflow, resources, and service demands.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

CI:

Confidence intervals

CT:

Chlamydia trachomatis

Ct:

Cycle threshold

HIV:

Human immunodeficiency virus

HoT:

Hands-on time

NG:

Neisseria gonorrhoeae

NPA:

Negative percent agreement

OPA:

Overall percent agreement

PPA:

Positive percent agreement

RLU:

Relative light unit

STIs:

Sexually transmitted infections

References

  1. Newman L, Rowley J, Vander Hoorn S, Wijesooriya NS, Unemo M, Low N, et al. Global estimates of the prevalence and incidence of four curable sexually transmitted infections in 2012 based on systematic review and global reporting. PLoS One. 2015;10(12):e0143304.

    Article  Google Scholar 

  2. Centers for Disease Control and Prevention (CDC). Sexually Transmitted Disease Surveillance 2017. 2018. https://www.cdc.gov/std/stats17/2017-STDSurveillance-Report_CDC-clearance-9.10.18.pdf.

  3. Traeger MW, Schroeder SE, Wright EJ, Hellard ME, Cornelisse VJ, Doyle JS, et al. Effects of pre-exposure prophylaxis for the prevention of human immunodeficiency virus infection on sexual risk behavior in men who have sex with men: a systematic review and meta-analysis. Clin Infect Dis. 2018;67(5):676–86.

    Article  Google Scholar 

  4. Enomoto C, Noor S, Widner N. Is social media to blame for the sharp rise in STDs? Soc Sci. 2017;6(3):78. https://0-doi-org.brum.beds.ac.uk/10.3390/socsci6030078.

  5. Lanjouw E, Ouburg S, de Vries HJ, Stary A, Radcliffe K, Unemo M, et al. 2015 European guideline on the management of Chlamydia trachomatis infections. Int J STD AIDS. 2016;27(5):333–48.

    Article  Google Scholar 

  6. Bignell C, Unemo M, European STI. Guidelines editorial board. 2012 European guideline on the diagnosis and treatment of gonorrhoea in adults. Int J STD AIDS. 2013;24(2):85–92.

    Article  CAS  Google Scholar 

  7. Fairley TA, Cohen DA, Elkins W. Asymptomatic sexually transmitted diseases: the case for screening. Prev Med. 2003;36(4):502–9.

    Article  Google Scholar 

  8. Centers for Disease Control and Prevention (CDC). Recommendations for the laboratory-based detection of chlamydia trachomatis and Neisseria gonorrhoeae-2014. MMWR Recomm Rep. 2014;63:RR–02):1–19.

  9. Roche Molecular Systems. cobas® CT/NG assay [Package insert]. 2018. https://pim-eservices.roche.com/eLD_SF/gb/en/Documents/GetDocument?documentId=b2f93133-7cc8-e611-6498-00215a9b3428.

  10. Hologic. Aptima Combo 2® Assay [Package insert]. 2017.

  11. College of American Pathologists. CAP TODAY. 2018. p. 58–62. http://digital.olivesoftware.com/Olive/ODN/CAPToday/default.aspx.

Download references

Acknowledgements

Enrique Marino (Roche Molecular Systems, Inc., Pleasanton, California, USA) provided biostatistical input as well as statistical analysis for the study. Nexus (Plano, Texas, USA) performed the independent third-party workflow observations and analysis. Editorial support was provided by Corrinne Segal of Elements Communications Ltd. (Westerham, Kent, UK) and funded by Roche Molecular Diagnostics.

Endnote

COBAS is a trademark of Roche.

Funding

The study was supported by Roche Molecular Systems, Inc. (Pleasanton, California, USA), study ID RD003259. Roche Molecular Systems assisted with the design of the study, analysis and interpretation of the data, and in writing of the manuscript.

Author information

Authors and Affiliations

Authors

Contributions

EMM, DD, and AF conceived this article. EMM wrote the first draft with contributions from AF. AF, GA, and DW were responsible for laboratory testing. All authors interpreted data, reviewed successive drafts, and approved the final version of the article.

Corresponding author

Correspondence to André Frontzek.

Ethics declarations

Ethics approval and consent to participate

The Ethics Committee of the Ärztekammer Nordrhein, North Rhine Westphalia, Germany waived the need for ethics approval and the need to obtain consent for the collection, analysis, and publication of the retrospectively obtained and anonymized data for this non-interventional study.

Consent for publication

Not applicable.

Competing interests

EMM and DD are employees of Roche Molecular Diagnostics. AF has received honoraria as a consultant and speaker for Roche Molecular Systems, Inc. and Roche Diagnostics Deutschland GmbH. The other authors have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

“Illustration of testing and specimens evaluated for (A) workflow and (B) performance”, flow chart. (PPTX 43 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Frontzek, A., Aretzweiler, G., Winkens, D. et al. High-volume workflow and performance comparisons for Chlamydia trachomatis and Neisseria gonorrhoeae testing using automated molecular platforms. BMC Infect Dis 19, 797 (2019). https://0-doi-org.brum.beds.ac.uk/10.1186/s12879-019-4442-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s12879-019-4442-0

Keywords