Artificial Intelligence Algorithms and Natural Language Processing for the Recognition of Syncope Patients on Emergency Department Medical Records

Franca Dipaola, Mauro Gatti, Veronica Pacetti, Anna Giulia Bottaccioli, Dana Shiffer, Maura Minonzio, Roberto Menè, Alessandro Giaj Levra, Monica Solbiati, Giorgio Costantino, Marco Anastasio, Elena Sini, Franca Barbic, Enrico Brunetta, Raffaello Furlan

Research output: Contribution to journalArticle

Abstract

BACKGROUND: Enrollment of large cohorts of syncope patients from administrative data is crucial for proper risk stratification but is limited by the enormous amount of time required for manual revision of medical records.

AIM: To develop a Natural Language Processing (NLP) algorithm to automatically identify syncope from Emergency Department (ED) electronic medical records (EMRs).

METHODS: De-identified EMRs of all consecutive patients evaluated at Humanitas Research Hospital ED from 1 December 2013 to 31 March 2014 and from 1 December 2015 to 31 March 2016 were manually annotated to identify syncope. Records were combined in a single dataset and classified. The performance of combined multiple NLP feature selectors and classifiers was tested. Primary Outcomes: NLP algorithms' accuracy, sensitivity, specificity, positive predictive value, negative predictive value, and F3 score.

RESULTS: 15,098 and 15,222 records from 2013 and 2015 datasets were analyzed. Syncope was present in 571 records. Normalized Gini Index feature selector combined with Support Vector Machines classifier obtained the best F3 value (84.0%), with 92.2% sensitivity and 47.4% positive predictive value. A 96% analysis time reduction was computed, compared with EMRs manual review.

CONCLUSIONS: This artificial intelligence algorithm enabled the automatic identification of a large population of syncope patients using EMRs.

Original languageEnglish
Article numberE1677
JournalJournal of Clinical Medicine
Volume8
Issue number10
DOIs
Publication statusPublished - Oct 14 2019

Fingerprint

Natural Language Processing
Artificial Intelligence
Syncope
Electronic Health Records
Medical Records
Hospital Emergency Service
Hospital Departments
Sensitivity and Specificity
Research
Population

Cite this

Artificial Intelligence Algorithms and Natural Language Processing for the Recognition of Syncope Patients on Emergency Department Medical Records. / Dipaola, Franca; Gatti, Mauro; Pacetti, Veronica; Bottaccioli, Anna Giulia; Shiffer, Dana; Minonzio, Maura; Menè, Roberto; Giaj Levra, Alessandro; Solbiati, Monica; Costantino, Giorgio; Anastasio, Marco; Sini, Elena; Barbic, Franca; Brunetta, Enrico; Furlan, Raffaello.

In: Journal of Clinical Medicine, Vol. 8, No. 10, E1677, 14.10.2019.

Research output: Contribution to journalArticle

Dipaola, Franca ; Gatti, Mauro ; Pacetti, Veronica ; Bottaccioli, Anna Giulia ; Shiffer, Dana ; Minonzio, Maura ; Menè, Roberto ; Giaj Levra, Alessandro ; Solbiati, Monica ; Costantino, Giorgio ; Anastasio, Marco ; Sini, Elena ; Barbic, Franca ; Brunetta, Enrico ; Furlan, Raffaello. / Artificial Intelligence Algorithms and Natural Language Processing for the Recognition of Syncope Patients on Emergency Department Medical Records. In: Journal of Clinical Medicine. 2019 ; Vol. 8, No. 10.
@article{0a5eed1408d0423197a233ac2241baa2,
title = "Artificial Intelligence Algorithms and Natural Language Processing for the Recognition of Syncope Patients on Emergency Department Medical Records",
abstract = "BACKGROUND: Enrollment of large cohorts of syncope patients from administrative data is crucial for proper risk stratification but is limited by the enormous amount of time required for manual revision of medical records.AIM: To develop a Natural Language Processing (NLP) algorithm to automatically identify syncope from Emergency Department (ED) electronic medical records (EMRs).METHODS: De-identified EMRs of all consecutive patients evaluated at Humanitas Research Hospital ED from 1 December 2013 to 31 March 2014 and from 1 December 2015 to 31 March 2016 were manually annotated to identify syncope. Records were combined in a single dataset and classified. The performance of combined multiple NLP feature selectors and classifiers was tested. Primary Outcomes: NLP algorithms' accuracy, sensitivity, specificity, positive predictive value, negative predictive value, and F3 score.RESULTS: 15,098 and 15,222 records from 2013 and 2015 datasets were analyzed. Syncope was present in 571 records. Normalized Gini Index feature selector combined with Support Vector Machines classifier obtained the best F3 value (84.0{\%}), with 92.2{\%} sensitivity and 47.4{\%} positive predictive value. A 96{\%} analysis time reduction was computed, compared with EMRs manual review.CONCLUSIONS: This artificial intelligence algorithm enabled the automatic identification of a large population of syncope patients using EMRs.",
author = "Franca Dipaola and Mauro Gatti and Veronica Pacetti and Bottaccioli, {Anna Giulia} and Dana Shiffer and Maura Minonzio and Roberto Men{\`e} and {Giaj Levra}, Alessandro and Monica Solbiati and Giorgio Costantino and Marco Anastasio and Elena Sini and Franca Barbic and Enrico Brunetta and Raffaello Furlan",
year = "2019",
month = "10",
day = "14",
doi = "10.3390/jcm8101677",
language = "English",
volume = "8",
journal = "Journal of Clinical Medicine",
issn = "2077-0383",
publisher = "Multidisciplinary Digital Publishing Institute (MDPI)",
number = "10",

}

TY - JOUR

T1 - Artificial Intelligence Algorithms and Natural Language Processing for the Recognition of Syncope Patients on Emergency Department Medical Records

AU - Dipaola, Franca

AU - Gatti, Mauro

AU - Pacetti, Veronica

AU - Bottaccioli, Anna Giulia

AU - Shiffer, Dana

AU - Minonzio, Maura

AU - Menè, Roberto

AU - Giaj Levra, Alessandro

AU - Solbiati, Monica

AU - Costantino, Giorgio

AU - Anastasio, Marco

AU - Sini, Elena

AU - Barbic, Franca

AU - Brunetta, Enrico

AU - Furlan, Raffaello

PY - 2019/10/14

Y1 - 2019/10/14

N2 - BACKGROUND: Enrollment of large cohorts of syncope patients from administrative data is crucial for proper risk stratification but is limited by the enormous amount of time required for manual revision of medical records.AIM: To develop a Natural Language Processing (NLP) algorithm to automatically identify syncope from Emergency Department (ED) electronic medical records (EMRs).METHODS: De-identified EMRs of all consecutive patients evaluated at Humanitas Research Hospital ED from 1 December 2013 to 31 March 2014 and from 1 December 2015 to 31 March 2016 were manually annotated to identify syncope. Records were combined in a single dataset and classified. The performance of combined multiple NLP feature selectors and classifiers was tested. Primary Outcomes: NLP algorithms' accuracy, sensitivity, specificity, positive predictive value, negative predictive value, and F3 score.RESULTS: 15,098 and 15,222 records from 2013 and 2015 datasets were analyzed. Syncope was present in 571 records. Normalized Gini Index feature selector combined with Support Vector Machines classifier obtained the best F3 value (84.0%), with 92.2% sensitivity and 47.4% positive predictive value. A 96% analysis time reduction was computed, compared with EMRs manual review.CONCLUSIONS: This artificial intelligence algorithm enabled the automatic identification of a large population of syncope patients using EMRs.

AB - BACKGROUND: Enrollment of large cohorts of syncope patients from administrative data is crucial for proper risk stratification but is limited by the enormous amount of time required for manual revision of medical records.AIM: To develop a Natural Language Processing (NLP) algorithm to automatically identify syncope from Emergency Department (ED) electronic medical records (EMRs).METHODS: De-identified EMRs of all consecutive patients evaluated at Humanitas Research Hospital ED from 1 December 2013 to 31 March 2014 and from 1 December 2015 to 31 March 2016 were manually annotated to identify syncope. Records were combined in a single dataset and classified. The performance of combined multiple NLP feature selectors and classifiers was tested. Primary Outcomes: NLP algorithms' accuracy, sensitivity, specificity, positive predictive value, negative predictive value, and F3 score.RESULTS: 15,098 and 15,222 records from 2013 and 2015 datasets were analyzed. Syncope was present in 571 records. Normalized Gini Index feature selector combined with Support Vector Machines classifier obtained the best F3 value (84.0%), with 92.2% sensitivity and 47.4% positive predictive value. A 96% analysis time reduction was computed, compared with EMRs manual review.CONCLUSIONS: This artificial intelligence algorithm enabled the automatic identification of a large population of syncope patients using EMRs.

U2 - 10.3390/jcm8101677

DO - 10.3390/jcm8101677

M3 - Article

C2 - 31614982

VL - 8

JO - Journal of Clinical Medicine

JF - Journal of Clinical Medicine

SN - 2077-0383

IS - 10

M1 - E1677

ER -