Papnet-assisted cytological diagnosis intensifies the already marked variability among cytological laboratories

P. Mudu, G. Migliore, M. Alderisio, P. Morosini, G. Douglas, R. Navone, G. Montanari, L. Di Bonito, A. Vitale, D. Moretti, M. R. Giovagnoli, F. Fulciniti, M. Branca

Research output: Contribution to journalArticle

Abstract

Objective: The main objective was to assess the sensitivity, specificity and reliability of PAPNET-assisted diagnosis in comparison with conventional screening. Setting: Seven Italian and one English University or Research Institutes, and a random sample of an other 20 Italian Laboratories of the Italian National Health Service (INHS) provided the cervical smears. Methods: During the training phase every center examined in rotation four sets of slides for a total of 300 representative slides. Afterwards, 900 "positive" slides were added to the 3,100 slides which were collected consecutively without any selection or exclusion. The eight main centers were divided into four couples and each couple of centers examined 775 slides with the PAPNET system, "blindly" to the original diagnosis. An expert cytopathologist (M.A.) of the National Institute of Health (NIH) reassessed 40% of the slides with an original negative diagnosis to evaluate the false negative rate. Two expert NIH cytopathologists (M.A., G.M.) re-examined all slides where a disagreement had been observed between the original and one or both of the study diagnoses. The main analyses concerned the following three main categories: WNL and unsatisfactory for evaluation; ASCUS, AGUS and LSIL; HSIL and carcinoma, A special algorithm was devised to define the reference diagnosis for sensitivity and specificity assessment. Results: Laboratories, even belonging to the same couple, classified as "no review" a very different proportion of slides ranging from 35% to 74%. The index of kappa agreement between the members of couples examining the same sets of slides was low or very low, ranging from 0.30 to 0.03. The sensitivity of the review classification was particularly low in some laboratories. Surprisingly, only a small correlation was observed between the sensitivity of the review classification and the proportion of slides classified as "review". The "tentative" diagnosis on PAPNET tiles of the "review" slides was almost as reliable as the microscopic diagnosis. In the overall performance, there were many significant differences among the eight laboratories. The best laboratory had a sensitivity of 95% and a specificity of 96%. At least three laboratories displayed unacceptably low sensitivity and one a very low specificity. Conclusion: Altogether these results seem to confirm that there are wide differences among cytological laboratories per se, and that these differences are intensified by the use of an instrument like PAPNET. The huge variation in performance may be explained by differences in basic skills and by different training, but it is difficult to understand exactly what could have been done to reduce it.

Original languageEnglish
Pages (from-to)211-215
Number of pages5
JournalEuropean Journal of Gynaecological Oncology
Volume23
Issue number3
Publication statusPublished - 2002

Fingerprint

National Institutes of Health (U.S.)
Sensitivity and Specificity
Vaginal Smears
National Health Programs
Carcinoma
Atypical Squamous Cells of the Cervix

Keywords

  • Cervical Cytology
  • Diagnosis validation
  • Inter-observer variation
  • Pap-net
  • Screening

ASJC Scopus subject areas

  • Obstetrics and Gynaecology
  • Oncology

Cite this

Mudu, P., Migliore, G., Alderisio, M., Morosini, P., Douglas, G., Navone, R., ... Branca, M. (2002). Papnet-assisted cytological diagnosis intensifies the already marked variability among cytological laboratories. European Journal of Gynaecological Oncology, 23(3), 211-215.

Papnet-assisted cytological diagnosis intensifies the already marked variability among cytological laboratories. / Mudu, P.; Migliore, G.; Alderisio, M.; Morosini, P.; Douglas, G.; Navone, R.; Montanari, G.; Di Bonito, L.; Vitale, A.; Moretti, D.; Giovagnoli, M. R.; Fulciniti, F.; Branca, M.

In: European Journal of Gynaecological Oncology, Vol. 23, No. 3, 2002, p. 211-215.

Research output: Contribution to journalArticle

Mudu, P, Migliore, G, Alderisio, M, Morosini, P, Douglas, G, Navone, R, Montanari, G, Di Bonito, L, Vitale, A, Moretti, D, Giovagnoli, MR, Fulciniti, F & Branca, M 2002, 'Papnet-assisted cytological diagnosis intensifies the already marked variability among cytological laboratories', European Journal of Gynaecological Oncology, vol. 23, no. 3, pp. 211-215.
Mudu, P. ; Migliore, G. ; Alderisio, M. ; Morosini, P. ; Douglas, G. ; Navone, R. ; Montanari, G. ; Di Bonito, L. ; Vitale, A. ; Moretti, D. ; Giovagnoli, M. R. ; Fulciniti, F. ; Branca, M. / Papnet-assisted cytological diagnosis intensifies the already marked variability among cytological laboratories. In: European Journal of Gynaecological Oncology. 2002 ; Vol. 23, No. 3. pp. 211-215.
@article{f47aac7d2ce846fca247f503226ae435,
title = "Papnet-assisted cytological diagnosis intensifies the already marked variability among cytological laboratories",
abstract = "Objective: The main objective was to assess the sensitivity, specificity and reliability of PAPNET-assisted diagnosis in comparison with conventional screening. Setting: Seven Italian and one English University or Research Institutes, and a random sample of an other 20 Italian Laboratories of the Italian National Health Service (INHS) provided the cervical smears. Methods: During the training phase every center examined in rotation four sets of slides for a total of 300 representative slides. Afterwards, 900 {"}positive{"} slides were added to the 3,100 slides which were collected consecutively without any selection or exclusion. The eight main centers were divided into four couples and each couple of centers examined 775 slides with the PAPNET system, {"}blindly{"} to the original diagnosis. An expert cytopathologist (M.A.) of the National Institute of Health (NIH) reassessed 40{\%} of the slides with an original negative diagnosis to evaluate the false negative rate. Two expert NIH cytopathologists (M.A., G.M.) re-examined all slides where a disagreement had been observed between the original and one or both of the study diagnoses. The main analyses concerned the following three main categories: WNL and unsatisfactory for evaluation; ASCUS, AGUS and LSIL; HSIL and carcinoma, A special algorithm was devised to define the reference diagnosis for sensitivity and specificity assessment. Results: Laboratories, even belonging to the same couple, classified as {"}no review{"} a very different proportion of slides ranging from 35{\%} to 74{\%}. The index of kappa agreement between the members of couples examining the same sets of slides was low or very low, ranging from 0.30 to 0.03. The sensitivity of the review classification was particularly low in some laboratories. Surprisingly, only a small correlation was observed between the sensitivity of the review classification and the proportion of slides classified as {"}review{"}. The {"}tentative{"} diagnosis on PAPNET tiles of the {"}review{"} slides was almost as reliable as the microscopic diagnosis. In the overall performance, there were many significant differences among the eight laboratories. The best laboratory had a sensitivity of 95{\%} and a specificity of 96{\%}. At least three laboratories displayed unacceptably low sensitivity and one a very low specificity. Conclusion: Altogether these results seem to confirm that there are wide differences among cytological laboratories per se, and that these differences are intensified by the use of an instrument like PAPNET. The huge variation in performance may be explained by differences in basic skills and by different training, but it is difficult to understand exactly what could have been done to reduce it.",
keywords = "Cervical Cytology, Diagnosis validation, Inter-observer variation, Pap-net, Screening",
author = "P. Mudu and G. Migliore and M. Alderisio and P. Morosini and G. Douglas and R. Navone and G. Montanari and {Di Bonito}, L. and A. Vitale and D. Moretti and Giovagnoli, {M. R.} and F. Fulciniti and M. Branca",
year = "2002",
language = "English",
volume = "23",
pages = "211--215",
journal = "European Journal of Gynaecological Oncology",
issn = "0392-2936",
publisher = "S.O.G. CANADA Inc.",
number = "3",

}

TY - JOUR

T1 - Papnet-assisted cytological diagnosis intensifies the already marked variability among cytological laboratories

AU - Mudu, P.

AU - Migliore, G.

AU - Alderisio, M.

AU - Morosini, P.

AU - Douglas, G.

AU - Navone, R.

AU - Montanari, G.

AU - Di Bonito, L.

AU - Vitale, A.

AU - Moretti, D.

AU - Giovagnoli, M. R.

AU - Fulciniti, F.

AU - Branca, M.

PY - 2002

Y1 - 2002

N2 - Objective: The main objective was to assess the sensitivity, specificity and reliability of PAPNET-assisted diagnosis in comparison with conventional screening. Setting: Seven Italian and one English University or Research Institutes, and a random sample of an other 20 Italian Laboratories of the Italian National Health Service (INHS) provided the cervical smears. Methods: During the training phase every center examined in rotation four sets of slides for a total of 300 representative slides. Afterwards, 900 "positive" slides were added to the 3,100 slides which were collected consecutively without any selection or exclusion. The eight main centers were divided into four couples and each couple of centers examined 775 slides with the PAPNET system, "blindly" to the original diagnosis. An expert cytopathologist (M.A.) of the National Institute of Health (NIH) reassessed 40% of the slides with an original negative diagnosis to evaluate the false negative rate. Two expert NIH cytopathologists (M.A., G.M.) re-examined all slides where a disagreement had been observed between the original and one or both of the study diagnoses. The main analyses concerned the following three main categories: WNL and unsatisfactory for evaluation; ASCUS, AGUS and LSIL; HSIL and carcinoma, A special algorithm was devised to define the reference diagnosis for sensitivity and specificity assessment. Results: Laboratories, even belonging to the same couple, classified as "no review" a very different proportion of slides ranging from 35% to 74%. The index of kappa agreement between the members of couples examining the same sets of slides was low or very low, ranging from 0.30 to 0.03. The sensitivity of the review classification was particularly low in some laboratories. Surprisingly, only a small correlation was observed between the sensitivity of the review classification and the proportion of slides classified as "review". The "tentative" diagnosis on PAPNET tiles of the "review" slides was almost as reliable as the microscopic diagnosis. In the overall performance, there were many significant differences among the eight laboratories. The best laboratory had a sensitivity of 95% and a specificity of 96%. At least three laboratories displayed unacceptably low sensitivity and one a very low specificity. Conclusion: Altogether these results seem to confirm that there are wide differences among cytological laboratories per se, and that these differences are intensified by the use of an instrument like PAPNET. The huge variation in performance may be explained by differences in basic skills and by different training, but it is difficult to understand exactly what could have been done to reduce it.

AB - Objective: The main objective was to assess the sensitivity, specificity and reliability of PAPNET-assisted diagnosis in comparison with conventional screening. Setting: Seven Italian and one English University or Research Institutes, and a random sample of an other 20 Italian Laboratories of the Italian National Health Service (INHS) provided the cervical smears. Methods: During the training phase every center examined in rotation four sets of slides for a total of 300 representative slides. Afterwards, 900 "positive" slides were added to the 3,100 slides which were collected consecutively without any selection or exclusion. The eight main centers were divided into four couples and each couple of centers examined 775 slides with the PAPNET system, "blindly" to the original diagnosis. An expert cytopathologist (M.A.) of the National Institute of Health (NIH) reassessed 40% of the slides with an original negative diagnosis to evaluate the false negative rate. Two expert NIH cytopathologists (M.A., G.M.) re-examined all slides where a disagreement had been observed between the original and one or both of the study diagnoses. The main analyses concerned the following three main categories: WNL and unsatisfactory for evaluation; ASCUS, AGUS and LSIL; HSIL and carcinoma, A special algorithm was devised to define the reference diagnosis for sensitivity and specificity assessment. Results: Laboratories, even belonging to the same couple, classified as "no review" a very different proportion of slides ranging from 35% to 74%. The index of kappa agreement between the members of couples examining the same sets of slides was low or very low, ranging from 0.30 to 0.03. The sensitivity of the review classification was particularly low in some laboratories. Surprisingly, only a small correlation was observed between the sensitivity of the review classification and the proportion of slides classified as "review". The "tentative" diagnosis on PAPNET tiles of the "review" slides was almost as reliable as the microscopic diagnosis. In the overall performance, there were many significant differences among the eight laboratories. The best laboratory had a sensitivity of 95% and a specificity of 96%. At least three laboratories displayed unacceptably low sensitivity and one a very low specificity. Conclusion: Altogether these results seem to confirm that there are wide differences among cytological laboratories per se, and that these differences are intensified by the use of an instrument like PAPNET. The huge variation in performance may be explained by differences in basic skills and by different training, but it is difficult to understand exactly what could have been done to reduce it.

KW - Cervical Cytology

KW - Diagnosis validation

KW - Inter-observer variation

KW - Pap-net

KW - Screening

UR - http://www.scopus.com/inward/record.url?scp=0035986041&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0035986041&partnerID=8YFLogxK

M3 - Article

C2 - 12094957

AN - SCOPUS:0035986041

VL - 23

SP - 211

EP - 215

JO - European Journal of Gynaecological Oncology

JF - European Journal of Gynaecological Oncology

SN - 0392-2936

IS - 3

ER -