Comparison of Analytical and Clinical Performance of HPV 9G DNA Chip, PANArray HPV Genotyping Chip, and Hybrid-Capture II Assay in Cervicovaginal Swabs
Article information
Abstract
Background:
Human papillomavirus (HPV) infection can be detected by using several molecular methods, including Hybrid-Capture II (HC2) assay and variable HPV DNA chip tests, although each method has different sensitivities and specificities.
Methods:
We performed HPV 9G DNA Chip (9G) and PANArray HPV Genotyping Chip (PANArray) tests on 118 cervicovaginal swabs and compared the results with HC2, cytology, histology, and direct sequencing results.
Results
The overall and high-risk HPV (HR-HPV) positivity rates were 62.7% and 44.9% using 9G, and 61.0% and 30.5% using PANArray, respectively. The positivity rates for HR-HPV with these two chips were significantly lower than 55.1% when HC2 was used. The sensitivity of overall HPV positivity in detecting histologically confirmed low-grade cervical squamous intraepithelial lesions or higher was 88.7% for all three tests. The specificity was 58.5% for 9G and 61.5% for PANArray, which was significantly lower than the 72.3% for HC2. With the HR-HPV+ genotype threshold, the sensitivity decreased to 75.5% for 9G and 52.8% for PANArray, which was significantly lower than the 88.7% for HC2. Comparison of the two chips showed concordant results in 55.1% of the samples, compatible results in 16.9%, and discordant results in 28.0%, exhibiting poor agreement in detecting certain HPV genotypes. Compared with direct sequencing, 9G yielded no discordant results, whereas PANArray yielded 31 discordant results (26.7%).
Conclusions
Compared with HC2, the HPV genotyping tests showed lower sensitivity in histologic correlation. When the two chips were compared, the 9G was more sensitive and accurate for detecting HR-HPV than the PANArray.
The causal role of human papillomavirus (HPV) infection has been well established in the pathogenesis of cervical cancer and precancerous lesions [1]. Among the large number of HPV genotypes identified to date, about 40 genotypes infect the mucosal lining of the human body, including the anogenital tract. Based on their epidemiological association with cervical lesions, the mucosal HPV strains are classified into high-risk (HR) and lowrisk (LR) genotypes [2,3]. The HR-HPV genotypes are detected more frequently in precancerous or cancerous cervical lesions, whereas the LR-HPV genotypes cause genital warts and are rarely associated with premalignant or malignant cervical lesions [3,4]. Although most LR- or HR-HPV infections are transient and successfully controlled by the host immune system, persistent infection with HR-HPV was highly associated with the development of high-grade dysplasia or invasive cervical cancers [5,6]. Overall, 14 HR-HPV genotypes (HPV 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66, and 68) have been identified in almost all cervical cancers worldwide [4]. Among these types, HPV 16 is the most important genotype that causes more than 50% of cervical carcinomas, followed by HPV 18 that causes 10%–15% and HPV 45 that causes approximately 7% [7]. HPV 18 is also identified in more than 35% of cervical adenocarcinomas [8]. Therefore, detecting HR-HPV in cervical samples is an important ancillary test for screening cervical lesions [9]. Furthermore, HPV testing has been shown to have a high negative predictive value (NPV) close to 100% for high-grade squamous intraepithelial lesions (HSILs) and invasive cancers, thereby emphasizing the usefulness of HPV tests in the triage of equivocal or lowgrade cytological smears [10,11].
HPV infection can be detected by many molecular methods, including Hybrid-Capture II assay (HC2; Qiagen, Gaithersburg, MD, USA) and variable HPV genotyping tests, although these methods have different sensitivities and specificities. HC2 detects 13 types of HR-HPV by means of an RNA cocktail probe and provides pooled data about HR-HPV infection, but does not indicate individual HPV types; further, it is the most widely used screening test with proven clinical performance [12-14]. However, because a few HPV types, including HPV 16 and 18 among the many types of HPV, are responsible for most HPV-related cancers, a clinical necessity for HPV genotyping that can distinguish individual types has arisen. Genotyping for specific oncogenic HPV types has also been shown to improve the positive predictive value (PPV) and specificity in predicting HSIL and carcinoma in women with atypical squamous cells of undetermined significance (ASCUS) and low-grade cervical squamous intraepithelial lesions (LSIL) [15]. In addition, in vaccinated women, type-specific HPV testing can be useful to assess the prevalence of specific HPV types, including HPV types that do not have vaccines available [15]. Reflecting this trend, the recently introduced Cobas HPV test (Roche Molecular Systems Inc., Branchburg, NJ, USA) provides specific genotyping data about HPV 16 and 18 in conjunction with pooled HR-HPV results [16].
In Korea, several HPV genotyping tests that use polymerase chain reaction (PCR)-based microarray methods are commercially available, but they lack standardization compared to HC2 and show variable clinical efficacy. We therefore selected two widely used DNA chips, the HPV 9G DNA Chip (9G; Diatech Korea Co. Ltd., Seoul, Korea) and PANArray HPV Genotyping Chip (PANArray; Panagene, Daejeon, Korea), and compared their results with the results from HC2, cytological diagnoses, and histological diagnoses in terms of clinical efficacy. Additionally, genotyping results of these two chips were further validated by direct sequencing.
MATERIALS AND METHODS
Selection of study samples
A total of 118 histologically confirmed cervicovaginal swab specimens in which HPV infection status was examined by HC2, between January 2015 and March 2015 at Konkuk University Medical Center, were collected. Cytological diagnosis was obtained in 109 patients using liquid-based cervical cytology (Sure-Path, TriPath Imaging Inc., Burlington, NC, USA) according to the Bethesda System for reporting cervical cytology [17]. Histological confirmation was available in all 118 cases using cervical biopsy, loop electrosurgical excision procedure, and hysterectomy within one month from HC2. In total, nine cases have not been sent for cytological examination, but were submitted for histological examination. The cytological and histological diagnoses were reviewed and confirmed by two pathologists (H.S.H. and W.Y.K.). This study was approved by the Institutional Review Board (KUH 1210039).
HC2
Cervical specimens were collected using brush samplers and placed in a collection tube containing a preservative solution (Qiagen). A 200-µL aliquot of each sample was removed and stored for DNA extraction and subsequent testing for HPV genotyping assays. The HC2 assay method was performed using residual sample materials with the automated HC2 assay system, according to the manufacturer’s protocol (Qiagen). The samples were analyzed for the presence of 13 HR-HPV types (16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, and 68). Positive and negative controls (provided by the manufacturer) were included in each run. The relative light unit for all the samples was set to the degree of relative brightness compared to the positive control group. This ratio was considered positive when it was ≥1.0 and negative when it was <1.0.
DNA extraction
DNA was extracted from the above-mentioned 200-µL aliquots of samples, as previously described [18].
HPV genotype assay using the 9G
The 9G test detected 14 HR- and 5 LR-HPV types (Table 1). Analyses were performed according to the manufacturer’s instructions [19]. Briefly, the PCR mixture consisted of 10 µL of the extracted target DNA, 10 µL of the primer set (provided by the manufacturer), and PCR premix (provided by the manufacturer) that contained deoxyribonucleotide triphosphate and Taq DNA polymerase in an amplification buffer. Amplification was performed using the following steps: predenaturation for 5 minutes at 94°C, 40 cycles of 30 seconds each for denaturation at 94°C, 40 cycles of 30 seconds each for annealing at 45°C, 40 cycles of 30 seconds each for elongation at 72°C, and a final elongation step of 5 minutes at 72°C. The PCR products were electrophoresed in a 2% agarose gel to confirm successful amplification of the PCR product. Each hybridization chamber of the 9G was covered with a mixture of 35 µL of the hybridization reaction mixture and 15 µL of the PCR product and incubated at 25°C for 30 minutes. After washing, array images were scanned and taken using a fluorescent scanner (EasyScan-100, Xillux Co. Ltd., Seoul, Korea).
HPV genotypes identified by the 9G and PANArray
HPV genotype assay using the PANArray
The PANArray test detected 19 HR- and 13 LR-HPV types (Table 1). Analyses were performed according to the manufacturer’s instructions [20]. Briefly, one PCR mix contained 5 µL of target DNA, 3 µL of PCR primer No. 1, and 17 µL of reaction mixture No. 1 supplied by the manufacturer (containing Taq DNA polymerase, PCR buffer, and deoxynucleoside triphosphate mixture) for a total volume of 25 µL. Another PCR mix contained 5 µL of the same target DNA, 3 µL of PCR primer No. 2, and 17 µL of reaction mixture No. 2. All tubes were incubated for 2 minutes at 50°C before PCR was started. Ten cycles of 30 seconds at 94°C, 30 seconds at 60°C, and 30 seconds at 72°C were performed, including predenaturation for 15 minutes at 94°C at the first step, followed by 40 cycles of 30 seconds at 94°C, 1 minute at 47°C, and 30 seconds at 72°C. The PCR products were electrophoresed in a 2% agarose gel to confirm successful amplification of the PCR product. A mixture of hybridization buffer No. 1 and No. 2 (70 µL) was mixed with 5 µL of PCR product No. 1 and 5 µL of PCR product No. 2 and then applied to the PANArray chip and incubated for 1 hour at 50°C. After washing, array images were scanned and taken using a fluorescent scanner (EasyScan-100, Xillux Co. Ltd.).
Direct sequencing
Genotyping results were confirmed by direct sequencing, as previously described [19]. Direct sequencing was performed by an outside laboratory (Fammed Co. Ltd., Seongnam, Korea) using a universal HPV primer set (MY 09/11) that detects a wide range of HPV types. The primer sequences were as follows: MY 09 (reverse) 5'-CGTCCMARRGGAWACTGATC-3'; MY 11 (forward) 5'-GCMCAGGGWCATAAYAATGG-3'. In samples with multiple infections that were identified by the two HPV genotyping assays, type-specific primers were used to obtain specific PCR products, which were analyzed again by direct sequencing. Gene accession numbers of type-specific primers for each HPV genotype were as follows: HPV 6 (HG793922), 16 (KP874716), 18 (KT070102), 31 (KF700156), 33 (KF700164), 34 (X74476), 35 (JX129488), 39 (KC470245), 40 (HE793074), 42 (GQ472847), 45 (KC470255), 51 (M62877), 52 (KF700237), 53 (JN393901), 56 (JX912947), 58 (AB819279), 59 (KC470266), 66 (JN122292), 68 (KC70283), and 70 (KC470291). Sequences of type-specific primers are shown in Table 2. A total of 116 cases were available by direct sequencing, with the exception of two cases.
Type-specific primer sets used in direct sequencing
Statistical analysis
We defined true-positive, true-negative, false-positive, and false-negative results of each detection method to compare with the histological diagnosis as the gold standard. Sensitivity, specificity, PPV, and NPV were calculated according to the definitions. The McNemar test was used to compare the diagnostic sensitivity and specificity between HPV DNA tests, and to test the difference between paired proportions. Agreement between tests was assessed using Cohen’s kappa statistic, where values in the range of 0.00–0.20 indicated poor agreement, 0.21–0.40 indicated fair agreement, 0.41–0.60 indicated moderate agreement, 0.61–0.80 indicated good agreement, and 0.81–1.00 indicated very good agreement. Two-sided p<.05 were considered statistically significant. The SPSS ver. 17.0 statistical software (SPSS Inc., Chicago, IL, USA) was used for the statistical evaluation.
RESULTS
Cytological and histological diagnoses
Of the 109 cases with cytological diagnoses, 47 cases were diagnosed as negative for intraepithelial lesion, 24 cases were diagnosed as ASCUS, 24 cases were diagnosed as LSIL, 10 cases were diagnosed as HSIL, two cases were diagnosed as atypical squamous cells–cannot exclude HSIL, and two cases were diagnosed as invasive squamous cell carcinomas (Table 3). Histological diagnosis was obtained in 118 cases, which included 65 cases of cervicitis, 24 cases of LSIL, 25 cases of HSIL, and four cases of cancers, including three cases of invasive squamous cell carcinomas, and one case of adenocarcinoma. The cytology was normal, but the histological findings indicated LSIL in four patients and cancer (adenocarcinoma) in one patient. Among the 24 patients with ASCUS, 15 patients had LSIL or HSIL. Although cytology indicated LSIL, the histological findings indicated HSIL in seven patients.
Diagnostic comparison between histology and cytology
HPV positivity by HC2 and two genotyping assays
The overall HPV positivity rate was 62.7% for 9G and 61.0% for PANArray. The HR-HPV positivity rate was 44.9% using the 9G and 30.5% with the PANArray, which was significantly lower than the HR-HPV positivity rate of 55.1% by HC2 (p=.002 and p=.000, respectively) (Table 4).
Prevalence of HPV and its distribution according to histological and cytological diagnosis
The rates of HPV positivity using the HC2 and two DNA chips according to the histological diagnosis are summarized in Table 4. LSIL or higher lesions (n=53) showed significantly higher rates of HPV positivity compared to cervicitis by HC2 and two chips. The positivity rates for HR-HPV by PANArray in LSIL (41.7%) and HSIL (72.0%) were significantly lower than those by HC2: 79.2% and 100%, respectively (p=.004 and p=.016, respectively), but the positivity rates for HR-HPV using 9G in LSIL (58.3%) and HSIL (96.0%) did not show any statistically significant differences from those by HC2 (p=.063 and p=1.000, respectively).
In comparison according to the cytological diagnoses, the prevalence of HR-HPV detected by PANArray in cases of ASCUS (50.0%) was significantly lower than that of HR-HPV detected by HC2 (83.3%) and that of HR-HPV detected by 9G (79.2%) (p=.008 and p=.016, respectively) (Table 4). The PANArray also showed lower positivity rates for HR-HPV in cases of LSIL (54.2%) or HSIL (30.0%) compared to HC2, which were 91.7% and 100%, respectively (p=.004 and p=.016, respectively).
Comparative evaluation of 9G and PANArray
All HPV genotypes detected by two chips were genotypes detectable by both 9G and PANArray, with the exception of one case. This case showed HPV type 70 (detectable by PANArray only) using PANArray and other type HPV by 9G.
Among the HPV+ cases detected using two chips, the frequency of multiple infections detected by 9G (23/74, 31.1%) was significantly higher compared to that detected by PANArray (5/72, 6.9%) (p=.000) (Table 4). This difference occurred because the majority of multiple infections detected by 9G (16/23, 69.5%) were identified as a single infection by PANArray. The pattern of mixed infection was mostly the combination of HR- and LR-HPV types, followed by co-infection with two or more HR-HPV types, but rarely presented as co-infection with LR types.
When the two chips were compared, the detection rate of HR-HPV by 9G (44.9%) was significantly higher than that of PANArray (30.5%) (p=.000), whereas the positivity rate for the other type HPV with 9G (12.7%) was lower than that with PANArray (23.7%) (Table 4). When the genotyping results were classified as negative, LR-, HR-, or other type HPV, the two chip comparison showed that a substantial proportion of HR-HPV+ cases detected by 9G were detected as other type (18/53, 34.0%) by PANArray, which caused lower detection rates of HR-HPV by PANArray (Table 5). As a result, the overall strength of agreement between the two assays was considered to be moderate in risk-stratified comparison (k=0.579).
Comparison of two HPV genotyping assays according to risk-stratified results
Overall frequency of detection of each HPV genotype was compared in assay-common genotypes using both chip tests, and the results are shown in Table 6. The highest kappa values were observed for HPV 6, 45, 51, and 58 (very good agreement), followed by HPV 16, 34, 39, and 40 (good agreement), HPV 18 and 42 (moderate agreement), and HPV 33 and 56 (fair agreement). The lowest kappa values were observed for HPV types 11, 31, 35, 52, 66, and 68, which showed no agreement between the two chips.
Type-specific agreement for each HPV type, detected by both the 9G and the PANArray in 118 samples
Comparative evaluation of 9G and PANArray by direct sequencing
When genotyping results were classified as concordant (complete agreement in genotypes detected by two assays), compatible (partial agreement), or discordant (no same genotype result), the two chips showed concordant results in 55.1% of the samples, compatible results in 16.9%, and discordant results in 28.0% (Table 7).
Agreement for HPV genotype, detected by the 9G, PANArray, and direct sequencing in 118 samples
Comparisons of two genotyping assays and direct sequencing were also classified as concordant, compatible, or discordant (Table 7). The 9G yielded no discordant results, whereas PANArray yielded 31 discordant results (26.7%). Twenty-three cases with multiple HPV infection by 9G were identified as a concordant multiple infection in 15 cases, a compatible multiple infection in six cases, and a compatible single infection in two cases by direct sequencing. Five cases with multiple HPV infection by PANArray were identified as a compatible multiple infection in three cases and a compatible single infection in one case by direct sequencing, with the exception of one case, which was not available by direct sequencing.
Clinical performance of 9G, PANArray, and HC2
To evaluate the diagnostic value of each method for HPV detection, we compared the results of each method with the histological diagnosis as the gold standard (Table 8). The sensitivity of overall HPV positivity in detecting LSIL or higher was 88.7% for all three tests. The specificity was 58.5% for 9G and 61.5% for PANArray, which was significantly lower than the specificity of 72.3% for HC2 (p=.002 and p=.03, respectively). The PPV of HC2 (72.3%) was higher than that of the two genotyping assays (63.5% and 65.2%).
Diagnostic accuracies of HPV DNA tests
With the HR-HPV+ genotype threshold, the sensitivity decreased to 75.5% for 9G and 52.8% for PANArray, which was significantly lower than the sensitivity of 88.7% for HC2 (p=.008 and p=.000, respectively). The PANArray also had lower sensitivity than 9G (p=.0005). The NPV of PANArray (69.5%) was lower than both the HC2 (88.7%) and 9G (80.0%). In contrast, the specificity of PANArray (87.7%) was significantly higher than that of HC2 (72.3%) (p=.003).
DISCUSSION
In this study, we performed 9G and PANArray tests on 118 cervicovaginal swabs and compared their clinical performance with HC2 through histological correlation. Genotyping results of the two DNA chips were further validated by direct sequencing. The detection rates of HR-HPV by the two genotyping assays were significantly lower than HC2 detection rates. The lower HR-HPV positivity rates also caused lower sensitivity of the two DNA chips in detecting LSIL or higher lesions in histologic correlation compared to that of HC2, which could limit their clinical applicability, considering the importance of HR-HPV detection in screening cervical lesions.
The PANArray, in particular, showed lowest positivity rates for HR-HPV among the three tests and it did not detect a substantial portion of HR-HPV+ cases identified by HC2. The PANArray also showed a distinctly lower prevalence of HR-HPV than HC2 in each group, according to the histological or cytological diagnosis. In contrast, this assay had higher detection rates of other-type HPV infection compared to 9G, because a substantial proportion of HR-HPV+ cases identified by the two other tests were detected as other-type or negative by PANArray. Due to the false negative results on PANArray, its sensitivity for HR-HPV detection showed a much lower value compared to that of HC2 and 9G, although the sensitivity for overall HPV detection, including LR- and other-type HPV, was identical to that of HC2 and 9G. Similarly, the NPV of this assay was inferior to that of the two tests with HR-HPV+ genotype threshold, whereas it showed a similar NPV for overall HPV detection. The lower sensitivity and NPV of PANArray indicate that it is not optimal to use as a screening tool for cervicovaginal samples.
There have been conflicting results about the PCR-based DNA chip method in previous studies in Korea. The majority of these studies reported that the HPV DNA chip method had higher or similar sensitivity or detection rates, compared to HC2 or other DNA tests [21-24], whereas some reports showed that commercial DNA chips had lower sensitivity or detection rates for HR-HPV [25,26]. Specifically, there were contradictory reports about comparative results of the PANArray method in the respective studies [24,26]. These differences may be due to insufficient standardization of the DNA chip method.
Insufficient standardization was also observed in the differences in the frequency of multiple infections between the two chips. The proportion of cases with multiple infections using the 9G chip was approximately four times higher than when PANArray was used. These differences between DNA chips were consistently found in previous studies, showing a variable percentage of multiple infections that ranged from about 6%–30% in HPV+ cases [21,23-25]. The high frequency of multiple infections may be associated with cross-reactivity due to the similarities in the DNA sequence between type-specific DNA probes attached within DNA chips, thereby yielding false-positive results that affect the sensitivity or specificity of DNA chips. Therefore, well-balanced standardization is required during DNA chip manufacturing to achieve the desired level of HPV detection.
In this study, the HR-HPV positivity rate in normal cytology was 10.6%–12.8%, which was similar to those reported in previous studies, which ranged from 8.3% to 14.5% [13,27-29]. The HR-HPV positivity rate in ASCUS was 83.3% based on HC2, which was higher than those described in previous studies, with ranges from 33.3% to 55.9% [13,27,29]. This difference might occur because the majority of cases diagnosed with ASCUS, due to insufficient sampling or poor quality in cytology, were determined to be LSIL or HSIL in histology. In addition, our study population, which consisted only of histologically-confirmed cases, might have an enhanced HPV positivity in ASCUS cases because histological samples were more frequently taken from patients with HPV+ ASCUS lesions than those that were not related to HPV.
In this study, HPV 16 was the most frequently identified HR genotype, followed by HPV 68, 58, 51, 18, 52, 56, 33, and 35 by 9G. In PANArray, HPV 16, 58, 18, and 51 were frequently detected. The prevalence of HR-HPV genotypes is known to differ according to geographic distribution [25]. HPV 16, 18, 31, and 45 are frequently detected in Western countries, whereas HPV 16, 58, 52, and 56 are predominant in Korea [24,25]. Considering the number of cases that were HPV 52 and 56 positive in two tests, 9G showed relatively similar patterns in the distribution of HPV genotypes compared to the PANArray method.
Comparison of the two chips indicated that the HPV genotypes showed variable differences in positivity for individual HPV types. Among the frequently detected genotypes, HPV 68, 35, and 52 were markedly different between the two chips with no agreement, as the PANArray showed negative results in HPV 68+ (n=9), 35+ (n=5), or 52+ (n=5) cases detected by 9G. These differences need to be corrected, especially because these types are relatively common in Korean women [24,25]. In contrast, the most important HR-HPV types that caused cervical cancer, HPV 16 and 18, showed relatively better agreement between the two genotyping assays.
When the results of the two chips were confirmed by direct sequencing, 9G yielded concordant or compatible results in all cases. In contrast, the PANArray yielded discordant results in approximately one fourth of the cases. Most of these discrepancies were attributable to the erroneous results of PANArray, discussed above, which detected HR-HPV as other-type. Given that the frequency of multiple infections was much higher with the 9G than by PANArray, these results suggest that the threshold value for detecting each HPV type may vary between chips and 9G may have a more sensitive threshold for detection compared to PANArray.
In this study, HC2 was more sensitive with a high NPV, thereby increasing the possibility of avoiding unnecessary surgical procedures. However, the HC2 assay has some disadvantages compared to other HPV genotyping tests. This assay does not provide specific information about individual HPV genotypes, and it does not distinguish between persistent infection with the same genotype and infection with a new HPV genotype [25]. In addition, multiple infections also cannot be detected. However, the two HPV genotyping assays tested in the present study did not have diagnostic accuracies that were comparable to HC2. Therefore, it is necessary to improve the quality of HPV DNA chips, considering the importance of HPV genotyping, especially for detecting HPV 16 and 18, in clinical practice.
There were some limitations to this study. The present data included a small number of cases collected over a short time period. Furthermore, the number of HSIL or higher lesions in histology (n=29) was too small to generalize the results. A larger study series conducted over a longer period of time is needed to confirm the clinical and analytical performance of the DNA chip method.
In conclusion, the HPV genotyping tests using the PCR-based DNA chip method showed lower sensitivity in histologic correlation compared with HC2, limiting their clinical applicability as an HPV screening test. When comparing the two chips, the 9G was more sensitive and accurate for detecting HR-HPV than the PANArray. The two DNA chips showed poor agreement in detecting certain HPV genotypes. These results indicate the necessity for standardization and validation of the HPV genotyping assays.
Notes
Conflicts of Interest
This study was supported by Diatech Korea Co., Ltd. The sponsor had no involvement in the study design, data interpretation, or writing of the manuscript.