Machine Matches Docs for Arrhythmia Detection

For detecting a variety of arrhythmias, ambulatory ECG data analysis with a deep neural network (DNN) did better than cardiologists in most cases, researchers found in an experimental setting.

The DNN developed by Awni Hannun, PhD, of Stanford University in California, and colleagues, achieved an average area under the receiver-operating characteristic curve (ROC) of 0.97 when confirmed against independent test data annotated by a committee of board-certified practicing cardiologists.

A so-called F₁ score that combined sensitivity and positive predictive value was 0.780 for human cardiologists versus 0.837 for the DNN, Hannun and colleagues wrote in Nature Medicine.

There is limited data available regarding whether an end-to-end deep learning method can be implemented to analyze the raw ECG information used to group a broad spectrum of diagnoses, the researchers explained. They said previous research aimed at using DNNs for ECG interpretation had looked at single facets of the ECG process, like feature extraction or noise reduction, or else was limited by detecting only certain heartbeat types such as fusion, ventricular, normal, and supraventricular ectopic, or rhythm diagnoses such as ventricular tachycardia or atrial fibrillation.

“Lack of appropriate data has limited many efforts beyond these applications. Most prior efforts used data from the MIT-BIH Arrhythmia database (PhysioNet), which is limited by the small number of patients and rhythm episodes present in the dataset,” Hannun and colleagues wrote.

Clinical practice could benefit from an accurate machine system for differential diagnosis of arrhythmia, Hannun told MedPage Today. “Automating arrhythmia detection can make heart monitoring with ECG more accessible and useful as a first-line diagnostic tool. In certain clinical settings, these results have the potential to lead to reduced rates of currently misdiagnosed computerized ECG interpretations and improvements in efficiency of expert-human ECG interpretation,” he said.

The researchers developed the DNN from a training dataset comprising 53,549 patients (mean age 69 and 43% women) with data recorded on the Zio ambulatory ECG monitor and provided by the device’s manufacturer with patients’ identities hidden. They grouped the data into 12 rhythm classes including noise, sinus rhythm, trigeminy, atrial fibrillation and flutter, ventricular tachycardia, and Wenckebach, among others.

Hannun and colleagues then validated the DNN on a different dataset of 328 ECG records from 328 patients (mean age 70 and 38% women), which was annotated by a team of cardiologists. The researchers then calculated F₁ values for the “average cardiologist” from these data.

Notably, when it came to ventricular tachycardia, the model had greater sensitivity (94.1%) than the average cardiologist (78.4%) — as was the case for all the other arrhythmia types — but in this case specificity suffered markedly. “Ventricular tachycardia is a clinically important rhythm for which the model had a lower F₁ score than cardiologists,” the researchers highlighted. F₁ scores for the other 11 groups all favored the DNN.

A total of 16 recordings were misclassified by the DNN as ventricular tachycardia, but the researchers said “that ‘mistakes’ made by the algorithm were very reasonable.”

“One of the key limitations,” Hannun told MedPage Today, “is that we only studied single-lead ambulatory ECG as opposed to 12-lead which is another important standard. Detecting arrhythmias in ambulatory ECG is in some ways a more difficult problem given the amount of noise present and the lower resolution. However, it remains to be seen if our work generalizes to other settings.”