User login
In a randomized clinical trial using EndoVigilant, there wasn’t a significant difference in adenomas per colonoscopy (APC) in procedures with the CADe tool versus those without it. In addition, the adenoma detection rate (ADR) and serrated polyp detection rate were similar in the CADe and non-CADe groups.
“Although we were disappointed that AI [artificial intelligence] did not improve detection of adenomas or serrated polyps in our study, we are still optimistic that this exciting technology will eventually impact endoscopy in a very positive way,” senior author Shai Friedland, MD, a professor of medicine at Stanford (Calif.) University and gastroenterologist with the Veterans Affairs Palo Alto Health Care System, said in an interview.
“The ultimate goal should be to improve the ability of colonoscopy to prevent morbidity and mortality from colon cancer, especially for endoscopists who may not be performing as well as they could be,” he said. “AI can potentially help prevent missed lesions due to fatigue or distraction, much like a warning system that averts car accidents. It can also potentially help endoscopists recognize dangerous – but rare – subtle lesions such as small, flat, and depressed cancers.”
The study was published online in the American Journal of Gastroenterology.
Analyzing detection rates
Several studies have evaluated the use of different CADe devices to reduce adenoma miss rates during colonoscopy, and some have found that the technology contributed to significantly higher ADR and APC, the study authors write. However, most of these studies have been performed in academic settings.
Dr. Friedland and colleagues conducted a randomized controlled trial, called AI-SEE, to evaluate the use of CADe during colonoscopy in four community-based endoscopy centers located in California, Connecticut, Maryland, and New Jersey between September 2020 and September 2021. The trial included seven board-certified clinicians, who had ADR of 25%-37% before the study. The participants were randomly assigned to colonoscopies with or without CADe in blocks of 16 patients to ensure masking. Both groups had similar patient demographics.
The research team enrolled patients aged 45 years or older who presented for screening or low-risk surveillance colonoscopy, which was defined as a patient qualifying for a surveillance interval of 3 years or greater based on the U.S. Multi-Society Task Force 2020 Guidelines. Patients were excluded if they had a history of inflammatory bowel disease, known or suspected polyposis or hereditary colon cancer syndrome, history of colon resection, or a referral for a diagnostic colonoscopy.
Among 769 enrolled patients, 387 were randomly assigned to undergo colonoscopy with EndoVigilant, an AI-enabled CADe software for colonoscopy. It augments existing white-light colonoscopy in real time by highlighting colon polyps and displaying a graphic box around the lesion on the monitor. It can be deployed as a single- or dual-monitor device. Although the study was originally designed to use two monitors, three investigators expressed strong preference for the single-monitor mode, so the protocol allowed endoscopists to choose.
Primary outcomes included APC and adenoma per extraction (APE), which is the percentage of polyps removed that are adenomas. Secondary endpoints included procedural time, ADR, serrated polyp detection rate, serrated polyps per colonoscopy, and nonadenomatous, nonserrated polyps per colonoscopy.
Overall, the use of CADe didn’t show a significant difference in APC, at 0.73, compared with 0.67 for non-CADe.
Although the use of CADe didn’t lead to increased identification of serrated polyps per colonoscopy – both at 0.08 – CADe led to increased identification of nonadenomatous, nonserrated polyps per colonoscopy, at 0.90 versus 0.51.
There also wasn’t a significant difference in distribution regarding adenomatous polyp location, size, or morphology. However, there was a trend toward greater identification of 6-9 mm APC using CADe, at 0.13 versus 0.08.
Mean withdrawal time was longer in the CADe group, at 11.7 minutes versus 10.7 minutes. However, when no polyps were identified, the withdrawal times were similar, at 9.1 minutes versus 8.8 minutes.
In addition, there was no difference in ADR for screening colonoscopies between the non-CADe and CADe groups, at 34.6% versus 34.3%, or for surveillance procedures, at 43.9% versus 40%. CADe also didn’t improve serrated polyp detection rates for screening or surveillance.
CADe was also associated with decreased APE in all colonoscopies (44.8 vs. 56.8) as well as in screening colonoscopies (43 vs. 57.8).
A comparison of single-monitor CADe with dual-monitor CADe found no significant difference in the average number of adenomas or serrated polyps identified per colonoscopy. However, dual-monitor CADe identified significantly more non-adenomatous, nonserrated polyps per colonoscopy (1.18 vs. 0.42), more adenomas sized at least 10 mm (0.19 vs. 0.05), and more flat polyps (0.18 vs. 0).
The study was terminated early after the interim analysis point, marked by 769 valid subjects. At this point, the comparison of APC between the two groups resulted in a new sample size estimate required for final analysis of 6,557 per group. This revised large study size estimate made it impractical to continue, the study authors wrote. No adverse events were observed during the study.
“What our study shows is that current systems – and the one we used in this study performs very well when tested on a database of images or videos – don’t make a major impact on very crude outcome measures, such as the total number of adenomas detected by a group of endoscopists at typical private endoscopy centers,” Dr. Friedland said. “I’m not convinced that we have a good answer yet for where to go from here, but we need to keep working with our AI colleagues to figure out how to use this exciting technology to improve outcomes in colon cancer.”
Additional considerations
In a separate evaluation of EndoVigilant, the frame level sensitivity was 0.9 and the frame level specificity was 0.97. These calculations were conducted on a dataset not used in training or validation of this model, the authors noted.
In this study, it’s possible that experienced community-based endoscopists are proficient at detecting the adenomas highlighted by the CADe system, so the technology may not detect a significant number of additional adenomas, the authors wrote. It’s also possible that some endoscopists ignore lesions highlighted by CADe, including small lesions that might be difficult to identify as adenomas or are seen as clinically unimportant, which could reduce the potential benefit of CADe.
“It’s important to remember that these tools are meant to be endoscopist assistance devices, not endoscopist replacements. They provide added benefit by pointing out polyps while we do the best exam we can,” Aasma Shaukat, MD, a professor of medicine and gastroenterologist at NYU Langone Health, New York, said in an interview.
Dr. Shaukat, who wasn’t involved with this study, has researched CADe for screening and surveillance colonoscopies. She and colleagues found that CADe use improved APC without an increase in resection of nonneoplastic lesions.
“Different trials have reported different results, and at the end of the day, it’s an endoscopist assistance tool, like spellcheck in a document,” she said. “It’s nice if spellcheck points to an incorrect spelling, but you don’t have to use it. Similarly, we often don’t know in these studies what an endoscopist felt or believed about the tool when using it.”
The benefits of CADe could vary based on its software, setting, number of patients, patient characteristics, number of clinicians, provider experience and training, dual- versus single-monitor setup, and even time of day, she noted. Future studies could clarify these factors, as well as improve the technology.
“This is just the beginning of AI in this field, and while bounding boxes to indicate potential polyps is a good start, it’s not the be-all, end-all,” Dr. Shaukat said.
“We want AI software to be able to tell us more about the size of the polyp, histology, prep quality, landmarks in the colon, adequacy of resection, and more. There’s some work being geared toward developing the algorithms to do these additional aspects,” she added.
The study was sponsored by EndoVigilant. Some of the authors reported consultant roles with Neptune Medical, AgilTx, Intuitive Surgical, Capsovision, and EndoVigilant. Dr. Shaukat reported no relevant financial relationships.
A version of this article first appeared on Medscape.com.
In a randomized clinical trial using EndoVigilant, there wasn’t a significant difference in adenomas per colonoscopy (APC) in procedures with the CADe tool versus those without it. In addition, the adenoma detection rate (ADR) and serrated polyp detection rate were similar in the CADe and non-CADe groups.
“Although we were disappointed that AI [artificial intelligence] did not improve detection of adenomas or serrated polyps in our study, we are still optimistic that this exciting technology will eventually impact endoscopy in a very positive way,” senior author Shai Friedland, MD, a professor of medicine at Stanford (Calif.) University and gastroenterologist with the Veterans Affairs Palo Alto Health Care System, said in an interview.
“The ultimate goal should be to improve the ability of colonoscopy to prevent morbidity and mortality from colon cancer, especially for endoscopists who may not be performing as well as they could be,” he said. “AI can potentially help prevent missed lesions due to fatigue or distraction, much like a warning system that averts car accidents. It can also potentially help endoscopists recognize dangerous – but rare – subtle lesions such as small, flat, and depressed cancers.”
The study was published online in the American Journal of Gastroenterology.
Analyzing detection rates
Several studies have evaluated the use of different CADe devices to reduce adenoma miss rates during colonoscopy, and some have found that the technology contributed to significantly higher ADR and APC, the study authors write. However, most of these studies have been performed in academic settings.
Dr. Friedland and colleagues conducted a randomized controlled trial, called AI-SEE, to evaluate the use of CADe during colonoscopy in four community-based endoscopy centers located in California, Connecticut, Maryland, and New Jersey between September 2020 and September 2021. The trial included seven board-certified clinicians, who had ADR of 25%-37% before the study. The participants were randomly assigned to colonoscopies with or without CADe in blocks of 16 patients to ensure masking. Both groups had similar patient demographics.
The research team enrolled patients aged 45 years or older who presented for screening or low-risk surveillance colonoscopy, which was defined as a patient qualifying for a surveillance interval of 3 years or greater based on the U.S. Multi-Society Task Force 2020 Guidelines. Patients were excluded if they had a history of inflammatory bowel disease, known or suspected polyposis or hereditary colon cancer syndrome, history of colon resection, or a referral for a diagnostic colonoscopy.
Among 769 enrolled patients, 387 were randomly assigned to undergo colonoscopy with EndoVigilant, an AI-enabled CADe software for colonoscopy. It augments existing white-light colonoscopy in real time by highlighting colon polyps and displaying a graphic box around the lesion on the monitor. It can be deployed as a single- or dual-monitor device. Although the study was originally designed to use two monitors, three investigators expressed strong preference for the single-monitor mode, so the protocol allowed endoscopists to choose.
Primary outcomes included APC and adenoma per extraction (APE), which is the percentage of polyps removed that are adenomas. Secondary endpoints included procedural time, ADR, serrated polyp detection rate, serrated polyps per colonoscopy, and nonadenomatous, nonserrated polyps per colonoscopy.
Overall, the use of CADe didn’t show a significant difference in APC, at 0.73, compared with 0.67 for non-CADe.
Although the use of CADe didn’t lead to increased identification of serrated polyps per colonoscopy – both at 0.08 – CADe led to increased identification of nonadenomatous, nonserrated polyps per colonoscopy, at 0.90 versus 0.51.
There also wasn’t a significant difference in distribution regarding adenomatous polyp location, size, or morphology. However, there was a trend toward greater identification of 6-9 mm APC using CADe, at 0.13 versus 0.08.
Mean withdrawal time was longer in the CADe group, at 11.7 minutes versus 10.7 minutes. However, when no polyps were identified, the withdrawal times were similar, at 9.1 minutes versus 8.8 minutes.
In addition, there was no difference in ADR for screening colonoscopies between the non-CADe and CADe groups, at 34.6% versus 34.3%, or for surveillance procedures, at 43.9% versus 40%. CADe also didn’t improve serrated polyp detection rates for screening or surveillance.
CADe was also associated with decreased APE in all colonoscopies (44.8 vs. 56.8) as well as in screening colonoscopies (43 vs. 57.8).
A comparison of single-monitor CADe with dual-monitor CADe found no significant difference in the average number of adenomas or serrated polyps identified per colonoscopy. However, dual-monitor CADe identified significantly more non-adenomatous, nonserrated polyps per colonoscopy (1.18 vs. 0.42), more adenomas sized at least 10 mm (0.19 vs. 0.05), and more flat polyps (0.18 vs. 0).
The study was terminated early after the interim analysis point, marked by 769 valid subjects. At this point, the comparison of APC between the two groups resulted in a new sample size estimate required for final analysis of 6,557 per group. This revised large study size estimate made it impractical to continue, the study authors wrote. No adverse events were observed during the study.
“What our study shows is that current systems – and the one we used in this study performs very well when tested on a database of images or videos – don’t make a major impact on very crude outcome measures, such as the total number of adenomas detected by a group of endoscopists at typical private endoscopy centers,” Dr. Friedland said. “I’m not convinced that we have a good answer yet for where to go from here, but we need to keep working with our AI colleagues to figure out how to use this exciting technology to improve outcomes in colon cancer.”
Additional considerations
In a separate evaluation of EndoVigilant, the frame level sensitivity was 0.9 and the frame level specificity was 0.97. These calculations were conducted on a dataset not used in training or validation of this model, the authors noted.
In this study, it’s possible that experienced community-based endoscopists are proficient at detecting the adenomas highlighted by the CADe system, so the technology may not detect a significant number of additional adenomas, the authors wrote. It’s also possible that some endoscopists ignore lesions highlighted by CADe, including small lesions that might be difficult to identify as adenomas or are seen as clinically unimportant, which could reduce the potential benefit of CADe.
“It’s important to remember that these tools are meant to be endoscopist assistance devices, not endoscopist replacements. They provide added benefit by pointing out polyps while we do the best exam we can,” Aasma Shaukat, MD, a professor of medicine and gastroenterologist at NYU Langone Health, New York, said in an interview.
Dr. Shaukat, who wasn’t involved with this study, has researched CADe for screening and surveillance colonoscopies. She and colleagues found that CADe use improved APC without an increase in resection of nonneoplastic lesions.
“Different trials have reported different results, and at the end of the day, it’s an endoscopist assistance tool, like spellcheck in a document,” she said. “It’s nice if spellcheck points to an incorrect spelling, but you don’t have to use it. Similarly, we often don’t know in these studies what an endoscopist felt or believed about the tool when using it.”
The benefits of CADe could vary based on its software, setting, number of patients, patient characteristics, number of clinicians, provider experience and training, dual- versus single-monitor setup, and even time of day, she noted. Future studies could clarify these factors, as well as improve the technology.
“This is just the beginning of AI in this field, and while bounding boxes to indicate potential polyps is a good start, it’s not the be-all, end-all,” Dr. Shaukat said.
“We want AI software to be able to tell us more about the size of the polyp, histology, prep quality, landmarks in the colon, adequacy of resection, and more. There’s some work being geared toward developing the algorithms to do these additional aspects,” she added.
The study was sponsored by EndoVigilant. Some of the authors reported consultant roles with Neptune Medical, AgilTx, Intuitive Surgical, Capsovision, and EndoVigilant. Dr. Shaukat reported no relevant financial relationships.
A version of this article first appeared on Medscape.com.
In a randomized clinical trial using EndoVigilant, there wasn’t a significant difference in adenomas per colonoscopy (APC) in procedures with the CADe tool versus those without it. In addition, the adenoma detection rate (ADR) and serrated polyp detection rate were similar in the CADe and non-CADe groups.
“Although we were disappointed that AI [artificial intelligence] did not improve detection of adenomas or serrated polyps in our study, we are still optimistic that this exciting technology will eventually impact endoscopy in a very positive way,” senior author Shai Friedland, MD, a professor of medicine at Stanford (Calif.) University and gastroenterologist with the Veterans Affairs Palo Alto Health Care System, said in an interview.
“The ultimate goal should be to improve the ability of colonoscopy to prevent morbidity and mortality from colon cancer, especially for endoscopists who may not be performing as well as they could be,” he said. “AI can potentially help prevent missed lesions due to fatigue or distraction, much like a warning system that averts car accidents. It can also potentially help endoscopists recognize dangerous – but rare – subtle lesions such as small, flat, and depressed cancers.”
The study was published online in the American Journal of Gastroenterology.
Analyzing detection rates
Several studies have evaluated the use of different CADe devices to reduce adenoma miss rates during colonoscopy, and some have found that the technology contributed to significantly higher ADR and APC, the study authors write. However, most of these studies have been performed in academic settings.
Dr. Friedland and colleagues conducted a randomized controlled trial, called AI-SEE, to evaluate the use of CADe during colonoscopy in four community-based endoscopy centers located in California, Connecticut, Maryland, and New Jersey between September 2020 and September 2021. The trial included seven board-certified clinicians, who had ADR of 25%-37% before the study. The participants were randomly assigned to colonoscopies with or without CADe in blocks of 16 patients to ensure masking. Both groups had similar patient demographics.
The research team enrolled patients aged 45 years or older who presented for screening or low-risk surveillance colonoscopy, which was defined as a patient qualifying for a surveillance interval of 3 years or greater based on the U.S. Multi-Society Task Force 2020 Guidelines. Patients were excluded if they had a history of inflammatory bowel disease, known or suspected polyposis or hereditary colon cancer syndrome, history of colon resection, or a referral for a diagnostic colonoscopy.
Among 769 enrolled patients, 387 were randomly assigned to undergo colonoscopy with EndoVigilant, an AI-enabled CADe software for colonoscopy. It augments existing white-light colonoscopy in real time by highlighting colon polyps and displaying a graphic box around the lesion on the monitor. It can be deployed as a single- or dual-monitor device. Although the study was originally designed to use two monitors, three investigators expressed strong preference for the single-monitor mode, so the protocol allowed endoscopists to choose.
Primary outcomes included APC and adenoma per extraction (APE), which is the percentage of polyps removed that are adenomas. Secondary endpoints included procedural time, ADR, serrated polyp detection rate, serrated polyps per colonoscopy, and nonadenomatous, nonserrated polyps per colonoscopy.
Overall, the use of CADe didn’t show a significant difference in APC, at 0.73, compared with 0.67 for non-CADe.
Although the use of CADe didn’t lead to increased identification of serrated polyps per colonoscopy – both at 0.08 – CADe led to increased identification of nonadenomatous, nonserrated polyps per colonoscopy, at 0.90 versus 0.51.
There also wasn’t a significant difference in distribution regarding adenomatous polyp location, size, or morphology. However, there was a trend toward greater identification of 6-9 mm APC using CADe, at 0.13 versus 0.08.
Mean withdrawal time was longer in the CADe group, at 11.7 minutes versus 10.7 minutes. However, when no polyps were identified, the withdrawal times were similar, at 9.1 minutes versus 8.8 minutes.
In addition, there was no difference in ADR for screening colonoscopies between the non-CADe and CADe groups, at 34.6% versus 34.3%, or for surveillance procedures, at 43.9% versus 40%. CADe also didn’t improve serrated polyp detection rates for screening or surveillance.
CADe was also associated with decreased APE in all colonoscopies (44.8 vs. 56.8) as well as in screening colonoscopies (43 vs. 57.8).
A comparison of single-monitor CADe with dual-monitor CADe found no significant difference in the average number of adenomas or serrated polyps identified per colonoscopy. However, dual-monitor CADe identified significantly more non-adenomatous, nonserrated polyps per colonoscopy (1.18 vs. 0.42), more adenomas sized at least 10 mm (0.19 vs. 0.05), and more flat polyps (0.18 vs. 0).
The study was terminated early after the interim analysis point, marked by 769 valid subjects. At this point, the comparison of APC between the two groups resulted in a new sample size estimate required for final analysis of 6,557 per group. This revised large study size estimate made it impractical to continue, the study authors wrote. No adverse events were observed during the study.
“What our study shows is that current systems – and the one we used in this study performs very well when tested on a database of images or videos – don’t make a major impact on very crude outcome measures, such as the total number of adenomas detected by a group of endoscopists at typical private endoscopy centers,” Dr. Friedland said. “I’m not convinced that we have a good answer yet for where to go from here, but we need to keep working with our AI colleagues to figure out how to use this exciting technology to improve outcomes in colon cancer.”
Additional considerations
In a separate evaluation of EndoVigilant, the frame level sensitivity was 0.9 and the frame level specificity was 0.97. These calculations were conducted on a dataset not used in training or validation of this model, the authors noted.
In this study, it’s possible that experienced community-based endoscopists are proficient at detecting the adenomas highlighted by the CADe system, so the technology may not detect a significant number of additional adenomas, the authors wrote. It’s also possible that some endoscopists ignore lesions highlighted by CADe, including small lesions that might be difficult to identify as adenomas or are seen as clinically unimportant, which could reduce the potential benefit of CADe.
“It’s important to remember that these tools are meant to be endoscopist assistance devices, not endoscopist replacements. They provide added benefit by pointing out polyps while we do the best exam we can,” Aasma Shaukat, MD, a professor of medicine and gastroenterologist at NYU Langone Health, New York, said in an interview.
Dr. Shaukat, who wasn’t involved with this study, has researched CADe for screening and surveillance colonoscopies. She and colleagues found that CADe use improved APC without an increase in resection of nonneoplastic lesions.
“Different trials have reported different results, and at the end of the day, it’s an endoscopist assistance tool, like spellcheck in a document,” she said. “It’s nice if spellcheck points to an incorrect spelling, but you don’t have to use it. Similarly, we often don’t know in these studies what an endoscopist felt or believed about the tool when using it.”
The benefits of CADe could vary based on its software, setting, number of patients, patient characteristics, number of clinicians, provider experience and training, dual- versus single-monitor setup, and even time of day, she noted. Future studies could clarify these factors, as well as improve the technology.
“This is just the beginning of AI in this field, and while bounding boxes to indicate potential polyps is a good start, it’s not the be-all, end-all,” Dr. Shaukat said.
“We want AI software to be able to tell us more about the size of the polyp, histology, prep quality, landmarks in the colon, adequacy of resection, and more. There’s some work being geared toward developing the algorithms to do these additional aspects,” she added.
The study was sponsored by EndoVigilant. Some of the authors reported consultant roles with Neptune Medical, AgilTx, Intuitive Surgical, Capsovision, and EndoVigilant. Dr. Shaukat reported no relevant financial relationships.
A version of this article first appeared on Medscape.com.
FROM THE AMERICAN JOURNAL OF GASTROENTEROLOGY