User login
Use of a machine-learning model that incorporates information from a single troponin test as well as other clinical data was superior to current practice as an aid to the diagnosis of myocardial infarction in the emergency department in a new study.
“Our results suggest that and free up space in the emergency department,” senior author Nicholas L. Mills, MD, University of Edinburgh, Scotland, said in an interview.
“And, perhaps even more importantly, use of this model could also increase the proportion of patients who are correctly identified as at a high probability of having an MI,” he added.
The study was published online in Nature Medicine.
The authors explained that at present, the likelihood of an MI diagnosis for patients presenting to the emergency department with chest pain is based on a fixed troponin threshold in serial tests at specific time points, but there are several problems with this approach.
First, a fixed troponin threshold is generally used for all patients, which does not account for age, sex, or comorbidities that are known to influence cardiac troponin concentrations. Second, the need to perform tests at specific time points for serial testing can be challenging in busy emergency departments.
And third, patients are categorized as being at low, intermediate, or high risk of MI on the basis of troponin thresholds alone, and the test does not take into account other important factors, such as the time of symptom onset or findings on the electrocardiogram.
“Our current practice of using the same threshold to rule in and rule out an MI for everyone, regardless of whether they are an 18-year-old female without a history of heart disease or an 85-year-old male with known heart failure, doesn’t perform well, and there’s a significant risk of misdiagnosis. There is also a high likelihood for inequalities in care, particularly between men and women,” Dr. Mills said.
The current study evaluated whether use of a machine learning model known as CoDE-ACS to guide decision-making could overcome some of these challenges.
The machine learning model assesses the whole spectrum of troponin levels as a continuous variable (rather than use of a single threshold) and turns this measurement into a probability that an individual patient is having an MI after accounting for other factors, including age, sex, comorbidities, and time from symptom onset.
For the current study, the CoDE-ACS model was trained in 10,000 patients with suspected acute coronary syndrome (ACS) who presented to 10 hospitals in Scotland as part of the High-STEACS trial evaluating the implementation of a high-sensitivity cardiac troponin I assay. The results were then validated in another 10,000 patients from six countries around the world.
“Using this model, the patient can have a troponin test on arrival at the emergency department. The other information on age, sex, clinical history, and time since symptom onset is keyed in, and it gives a probability on a scale of 0–100 as to whether the patient is having an MI,” Dr. Mills noted.
“It also has the capacity to incorporate more information over time. So, if there is a second troponin measurement made, then the model automatically refines the probability score,” he added.
The current study showed that use of the CoDE-ACS model identified more patients at presentation as having a low probability of having an MI than fixed cardiac troponin thresholds (61% vs. 27%) with a similar negative predictive value.
It also identified fewer patients as having a high probability of having an MI (10% vs. 16%) with a greater positive predictive value.
Among patients who were identified as having a low probability of MI, the rate of cardiac death was lower than the rate among those with intermediate or high probability at 30 days (0.1% vs. 0.5% and 1.8%) and 1 year (0.3% vs. 2.8% and 4.2%).
“The results show that the machine learning model doubles the proportion of patients who can be discharged with a single test compared to the current practice of using the threshold approach. It really is a game changer in terms of its potential to improve health efficiency,” Dr. Mills said.
In terms of ruling patients in as possibly having an MI, he pointed out that troponin levels are increased in patients with a wide range of other conditions, including heart failure, kidney failure, and atrial fibrillation.
“When using the threshold approach, only one in four patients with an elevated troponin level will actually be having an MI, and that leads to confusion,” he said. “This model takes into consideration these other conditions and so it can correctly identify three out of four patients with a high probability of having an MI. We can therefore be more confident that it is appropriate to refer those patients to cardiology and save a lot of potentially unnecessary investigations and treatments in the others.”
Dr. Mills said the model also seems to work when assessing patients early on.
“Around one-third of patients present within 3 hours of symptom onset, and actually these are a high-risk group because people who have genuine cardiac pain are normally extremely uncomfortable and tend to present quickly. Current guidelines require that we do two tests in all these individuals, but this new model incorporates the time of symptom onset into its estimates of probability and therefore allows us to rule out patients even when they present very early.”
He reported that a second test is required in only one in five patients – those whose first test indicated intermediate probability.
“The second test allows us to refine the probability further, allowing us to rule another half of those patients out. We are then left with a small proportion of patients – about 1 in 10 – who remain of intermediate probability and will require additional clinical judgment.”
Should improve inequities in MI diagnosis
Dr. Mills said the CoDE-ACS model will improve current inequities in MI diagnosis, because of which MI is underrecognized in women and younger people.
“Women have troponin concentrations that are half those of men, and although sex-specific troponin thresholds are recommended in the guidelines, they are not widely used. This automatically leads to underrecognition of heart disease in women. But this new machine learning model performs identically in men and women because it has been trained to recognize the different normal levels in men and women,” he explained.
“It will also help us not to underdiagnose MI in younger people who often have a less classical presentation of MI, and they also generally have very low concentrations of troponin, so any increase in troponin way below the current diagnostic threshold may be very relevant to their risk,” he added.
The researchers are planning a randomized trial of the new model to demonstrate the impact it could have on equality of care and overcrowding in the emergency department. In the trial, patients will be randomly assigned to undergo decision-making on the basis of troponin thresholds (current practice) or to undergo decision-making through the CoDE-ACS model.
“The hope is that this trial will show reductions in unnecessary hospital admissions and length of stay in the emergency department,” Dr. Mills said. Results are expected sometime next year.
“This algorithm is very well trained. It has learned on 20,000 patients, so it has a lot more experience than I have, and I have been a professor of cardiology for 20 years,” Dr. Mills said.
He said he believes these models will get even smarter in the future as more data are added.
“I think the future for good decision-making in emergency care will be informed by clinical decision support from well-trained machine learning algorithms and they will help us guide not just the diagnosis of MI but also heart failure and other important cardiac conditions,” he said.
‘Elegant and exciting’ data
Commenting on the study, John W. McEvoy, MB, University of Galway, Ireland, said: “These are elegant and exciting data; however, the inputs into the machine learning algorithm include all the necessary information to actually diagnose MI. So why predict MI, when a human diagnosis can just be made directly? The answer to this question may depend on whether we trust machines more than humans.”
Dr. Mills noted that clinical judgment will always be an important part of MI diagnosis.
“Currently, using the troponin threshold approach, experienced clinicians will be able to nuance the results, but invariably, there is disagreement on this, and this can be a major source of tension within clinical care. By providing more individualized information, this will help enormously in the decision-making process,” he said.
“This model is not about replacing clinical decision-making. It’s more about augmenting decision-making and giving clinicians guidance to be able to improve efficiency and reduce inequality,” he added.
The study was funded with support from the National Institute for Health Research and NHSX, the British Heart Foundation, and Wellcome Leap. Dr. Mills has received honoraria or consultancy from Abbott Diagnostics, Roche Diagnostics, Siemens Healthineers, and LumiraDx. He is employed by the University of Edinburgh, which has filed a patent on the Collaboration for the Diagnosis and Evaluation of Acute Coronary Syndrome score.
A version of this article first appeared on Medscape.com.
Use of a machine-learning model that incorporates information from a single troponin test as well as other clinical data was superior to current practice as an aid to the diagnosis of myocardial infarction in the emergency department in a new study.
“Our results suggest that and free up space in the emergency department,” senior author Nicholas L. Mills, MD, University of Edinburgh, Scotland, said in an interview.
“And, perhaps even more importantly, use of this model could also increase the proportion of patients who are correctly identified as at a high probability of having an MI,” he added.
The study was published online in Nature Medicine.
The authors explained that at present, the likelihood of an MI diagnosis for patients presenting to the emergency department with chest pain is based on a fixed troponin threshold in serial tests at specific time points, but there are several problems with this approach.
First, a fixed troponin threshold is generally used for all patients, which does not account for age, sex, or comorbidities that are known to influence cardiac troponin concentrations. Second, the need to perform tests at specific time points for serial testing can be challenging in busy emergency departments.
And third, patients are categorized as being at low, intermediate, or high risk of MI on the basis of troponin thresholds alone, and the test does not take into account other important factors, such as the time of symptom onset or findings on the electrocardiogram.
“Our current practice of using the same threshold to rule in and rule out an MI for everyone, regardless of whether they are an 18-year-old female without a history of heart disease or an 85-year-old male with known heart failure, doesn’t perform well, and there’s a significant risk of misdiagnosis. There is also a high likelihood for inequalities in care, particularly between men and women,” Dr. Mills said.
The current study evaluated whether use of a machine learning model known as CoDE-ACS to guide decision-making could overcome some of these challenges.
The machine learning model assesses the whole spectrum of troponin levels as a continuous variable (rather than use of a single threshold) and turns this measurement into a probability that an individual patient is having an MI after accounting for other factors, including age, sex, comorbidities, and time from symptom onset.
For the current study, the CoDE-ACS model was trained in 10,000 patients with suspected acute coronary syndrome (ACS) who presented to 10 hospitals in Scotland as part of the High-STEACS trial evaluating the implementation of a high-sensitivity cardiac troponin I assay. The results were then validated in another 10,000 patients from six countries around the world.
“Using this model, the patient can have a troponin test on arrival at the emergency department. The other information on age, sex, clinical history, and time since symptom onset is keyed in, and it gives a probability on a scale of 0–100 as to whether the patient is having an MI,” Dr. Mills noted.
“It also has the capacity to incorporate more information over time. So, if there is a second troponin measurement made, then the model automatically refines the probability score,” he added.
The current study showed that use of the CoDE-ACS model identified more patients at presentation as having a low probability of having an MI than fixed cardiac troponin thresholds (61% vs. 27%) with a similar negative predictive value.
It also identified fewer patients as having a high probability of having an MI (10% vs. 16%) with a greater positive predictive value.
Among patients who were identified as having a low probability of MI, the rate of cardiac death was lower than the rate among those with intermediate or high probability at 30 days (0.1% vs. 0.5% and 1.8%) and 1 year (0.3% vs. 2.8% and 4.2%).
“The results show that the machine learning model doubles the proportion of patients who can be discharged with a single test compared to the current practice of using the threshold approach. It really is a game changer in terms of its potential to improve health efficiency,” Dr. Mills said.
In terms of ruling patients in as possibly having an MI, he pointed out that troponin levels are increased in patients with a wide range of other conditions, including heart failure, kidney failure, and atrial fibrillation.
“When using the threshold approach, only one in four patients with an elevated troponin level will actually be having an MI, and that leads to confusion,” he said. “This model takes into consideration these other conditions and so it can correctly identify three out of four patients with a high probability of having an MI. We can therefore be more confident that it is appropriate to refer those patients to cardiology and save a lot of potentially unnecessary investigations and treatments in the others.”
Dr. Mills said the model also seems to work when assessing patients early on.
“Around one-third of patients present within 3 hours of symptom onset, and actually these are a high-risk group because people who have genuine cardiac pain are normally extremely uncomfortable and tend to present quickly. Current guidelines require that we do two tests in all these individuals, but this new model incorporates the time of symptom onset into its estimates of probability and therefore allows us to rule out patients even when they present very early.”
He reported that a second test is required in only one in five patients – those whose first test indicated intermediate probability.
“The second test allows us to refine the probability further, allowing us to rule another half of those patients out. We are then left with a small proportion of patients – about 1 in 10 – who remain of intermediate probability and will require additional clinical judgment.”
Should improve inequities in MI diagnosis
Dr. Mills said the CoDE-ACS model will improve current inequities in MI diagnosis, because of which MI is underrecognized in women and younger people.
“Women have troponin concentrations that are half those of men, and although sex-specific troponin thresholds are recommended in the guidelines, they are not widely used. This automatically leads to underrecognition of heart disease in women. But this new machine learning model performs identically in men and women because it has been trained to recognize the different normal levels in men and women,” he explained.
“It will also help us not to underdiagnose MI in younger people who often have a less classical presentation of MI, and they also generally have very low concentrations of troponin, so any increase in troponin way below the current diagnostic threshold may be very relevant to their risk,” he added.
The researchers are planning a randomized trial of the new model to demonstrate the impact it could have on equality of care and overcrowding in the emergency department. In the trial, patients will be randomly assigned to undergo decision-making on the basis of troponin thresholds (current practice) or to undergo decision-making through the CoDE-ACS model.
“The hope is that this trial will show reductions in unnecessary hospital admissions and length of stay in the emergency department,” Dr. Mills said. Results are expected sometime next year.
“This algorithm is very well trained. It has learned on 20,000 patients, so it has a lot more experience than I have, and I have been a professor of cardiology for 20 years,” Dr. Mills said.
He said he believes these models will get even smarter in the future as more data are added.
“I think the future for good decision-making in emergency care will be informed by clinical decision support from well-trained machine learning algorithms and they will help us guide not just the diagnosis of MI but also heart failure and other important cardiac conditions,” he said.
‘Elegant and exciting’ data
Commenting on the study, John W. McEvoy, MB, University of Galway, Ireland, said: “These are elegant and exciting data; however, the inputs into the machine learning algorithm include all the necessary information to actually diagnose MI. So why predict MI, when a human diagnosis can just be made directly? The answer to this question may depend on whether we trust machines more than humans.”
Dr. Mills noted that clinical judgment will always be an important part of MI diagnosis.
“Currently, using the troponin threshold approach, experienced clinicians will be able to nuance the results, but invariably, there is disagreement on this, and this can be a major source of tension within clinical care. By providing more individualized information, this will help enormously in the decision-making process,” he said.
“This model is not about replacing clinical decision-making. It’s more about augmenting decision-making and giving clinicians guidance to be able to improve efficiency and reduce inequality,” he added.
The study was funded with support from the National Institute for Health Research and NHSX, the British Heart Foundation, and Wellcome Leap. Dr. Mills has received honoraria or consultancy from Abbott Diagnostics, Roche Diagnostics, Siemens Healthineers, and LumiraDx. He is employed by the University of Edinburgh, which has filed a patent on the Collaboration for the Diagnosis and Evaluation of Acute Coronary Syndrome score.
A version of this article first appeared on Medscape.com.
Use of a machine-learning model that incorporates information from a single troponin test as well as other clinical data was superior to current practice as an aid to the diagnosis of myocardial infarction in the emergency department in a new study.
“Our results suggest that and free up space in the emergency department,” senior author Nicholas L. Mills, MD, University of Edinburgh, Scotland, said in an interview.
“And, perhaps even more importantly, use of this model could also increase the proportion of patients who are correctly identified as at a high probability of having an MI,” he added.
The study was published online in Nature Medicine.
The authors explained that at present, the likelihood of an MI diagnosis for patients presenting to the emergency department with chest pain is based on a fixed troponin threshold in serial tests at specific time points, but there are several problems with this approach.
First, a fixed troponin threshold is generally used for all patients, which does not account for age, sex, or comorbidities that are known to influence cardiac troponin concentrations. Second, the need to perform tests at specific time points for serial testing can be challenging in busy emergency departments.
And third, patients are categorized as being at low, intermediate, or high risk of MI on the basis of troponin thresholds alone, and the test does not take into account other important factors, such as the time of symptom onset or findings on the electrocardiogram.
“Our current practice of using the same threshold to rule in and rule out an MI for everyone, regardless of whether they are an 18-year-old female without a history of heart disease or an 85-year-old male with known heart failure, doesn’t perform well, and there’s a significant risk of misdiagnosis. There is also a high likelihood for inequalities in care, particularly between men and women,” Dr. Mills said.
The current study evaluated whether use of a machine learning model known as CoDE-ACS to guide decision-making could overcome some of these challenges.
The machine learning model assesses the whole spectrum of troponin levels as a continuous variable (rather than use of a single threshold) and turns this measurement into a probability that an individual patient is having an MI after accounting for other factors, including age, sex, comorbidities, and time from symptom onset.
For the current study, the CoDE-ACS model was trained in 10,000 patients with suspected acute coronary syndrome (ACS) who presented to 10 hospitals in Scotland as part of the High-STEACS trial evaluating the implementation of a high-sensitivity cardiac troponin I assay. The results were then validated in another 10,000 patients from six countries around the world.
“Using this model, the patient can have a troponin test on arrival at the emergency department. The other information on age, sex, clinical history, and time since symptom onset is keyed in, and it gives a probability on a scale of 0–100 as to whether the patient is having an MI,” Dr. Mills noted.
“It also has the capacity to incorporate more information over time. So, if there is a second troponin measurement made, then the model automatically refines the probability score,” he added.
The current study showed that use of the CoDE-ACS model identified more patients at presentation as having a low probability of having an MI than fixed cardiac troponin thresholds (61% vs. 27%) with a similar negative predictive value.
It also identified fewer patients as having a high probability of having an MI (10% vs. 16%) with a greater positive predictive value.
Among patients who were identified as having a low probability of MI, the rate of cardiac death was lower than the rate among those with intermediate or high probability at 30 days (0.1% vs. 0.5% and 1.8%) and 1 year (0.3% vs. 2.8% and 4.2%).
“The results show that the machine learning model doubles the proportion of patients who can be discharged with a single test compared to the current practice of using the threshold approach. It really is a game changer in terms of its potential to improve health efficiency,” Dr. Mills said.
In terms of ruling patients in as possibly having an MI, he pointed out that troponin levels are increased in patients with a wide range of other conditions, including heart failure, kidney failure, and atrial fibrillation.
“When using the threshold approach, only one in four patients with an elevated troponin level will actually be having an MI, and that leads to confusion,” he said. “This model takes into consideration these other conditions and so it can correctly identify three out of four patients with a high probability of having an MI. We can therefore be more confident that it is appropriate to refer those patients to cardiology and save a lot of potentially unnecessary investigations and treatments in the others.”
Dr. Mills said the model also seems to work when assessing patients early on.
“Around one-third of patients present within 3 hours of symptom onset, and actually these are a high-risk group because people who have genuine cardiac pain are normally extremely uncomfortable and tend to present quickly. Current guidelines require that we do two tests in all these individuals, but this new model incorporates the time of symptom onset into its estimates of probability and therefore allows us to rule out patients even when they present very early.”
He reported that a second test is required in only one in five patients – those whose first test indicated intermediate probability.
“The second test allows us to refine the probability further, allowing us to rule another half of those patients out. We are then left with a small proportion of patients – about 1 in 10 – who remain of intermediate probability and will require additional clinical judgment.”
Should improve inequities in MI diagnosis
Dr. Mills said the CoDE-ACS model will improve current inequities in MI diagnosis, because of which MI is underrecognized in women and younger people.
“Women have troponin concentrations that are half those of men, and although sex-specific troponin thresholds are recommended in the guidelines, they are not widely used. This automatically leads to underrecognition of heart disease in women. But this new machine learning model performs identically in men and women because it has been trained to recognize the different normal levels in men and women,” he explained.
“It will also help us not to underdiagnose MI in younger people who often have a less classical presentation of MI, and they also generally have very low concentrations of troponin, so any increase in troponin way below the current diagnostic threshold may be very relevant to their risk,” he added.
The researchers are planning a randomized trial of the new model to demonstrate the impact it could have on equality of care and overcrowding in the emergency department. In the trial, patients will be randomly assigned to undergo decision-making on the basis of troponin thresholds (current practice) or to undergo decision-making through the CoDE-ACS model.
“The hope is that this trial will show reductions in unnecessary hospital admissions and length of stay in the emergency department,” Dr. Mills said. Results are expected sometime next year.
“This algorithm is very well trained. It has learned on 20,000 patients, so it has a lot more experience than I have, and I have been a professor of cardiology for 20 years,” Dr. Mills said.
He said he believes these models will get even smarter in the future as more data are added.
“I think the future for good decision-making in emergency care will be informed by clinical decision support from well-trained machine learning algorithms and they will help us guide not just the diagnosis of MI but also heart failure and other important cardiac conditions,” he said.
‘Elegant and exciting’ data
Commenting on the study, John W. McEvoy, MB, University of Galway, Ireland, said: “These are elegant and exciting data; however, the inputs into the machine learning algorithm include all the necessary information to actually diagnose MI. So why predict MI, when a human diagnosis can just be made directly? The answer to this question may depend on whether we trust machines more than humans.”
Dr. Mills noted that clinical judgment will always be an important part of MI diagnosis.
“Currently, using the troponin threshold approach, experienced clinicians will be able to nuance the results, but invariably, there is disagreement on this, and this can be a major source of tension within clinical care. By providing more individualized information, this will help enormously in the decision-making process,” he said.
“This model is not about replacing clinical decision-making. It’s more about augmenting decision-making and giving clinicians guidance to be able to improve efficiency and reduce inequality,” he added.
The study was funded with support from the National Institute for Health Research and NHSX, the British Heart Foundation, and Wellcome Leap. Dr. Mills has received honoraria or consultancy from Abbott Diagnostics, Roche Diagnostics, Siemens Healthineers, and LumiraDx. He is employed by the University of Edinburgh, which has filed a patent on the Collaboration for the Diagnosis and Evaluation of Acute Coronary Syndrome score.
A version of this article first appeared on Medscape.com.
FROM NATURE MEDICINE