User login
In a head-to-head comparison, two commercially available computer-aided diagnosis systems appeared clinically equivalent for the optical diagnosis of small colorectal polyps, according to a research letter published in Gastroenterology.
For the optical diagnosis of diminutive colorectal polyps, the comparable performances of both CAD EYE (Fujifilm Co.) and GI Genius (Medtronic) met cutoff guidelines to implement the cost-saving leave-in-situ and resect-and-discard strategies, wrote Cesare Hassan, MD, PhD, associate professor of gastroenterology at Humanitas University and member of the endoscopy unit at Humanitas Clinical Research Hospital in Milan, and colleagues.
“Screening colonoscopy is effective in reducing colorectal cancer risk but also represents a substantial financial burden,” the authors wrote. “Novel strategies based on artificial intelligence may enable targeted removal only of polyps deemed to be neoplastic, thus reducing patient burden for unnecessary removal of nonneoplastic polyps and reducing costs for histopathology.”
Several computer-aided diagnosis (CADx) systems are commercially available for optical diagnosis of colorectal polyps, the authors wrote. However, each artificial intelligence (AI) system has been trained and validated with different polyp datasets, which may contribute to variability and affect the clinical outcome of optical diagnosis-based strategies.
Dr. Hassan and colleagues conducted a prospective comparison trial at a single center to look at the real-life performances of two CADx systems on optical diagnosis of polyps smaller than 5 mm.
At colonoscopy, the same polyp was visualized by the same endoscopist on two different monitors simultaneously with the respective output from each of the two CADx systems. Pre- and post-CADx human diagnoses were also collected.
Between January 2022 and March 2022, 176 consecutive patients age 40 and older underwent colonoscopy for colorectal cancer screening, polypectomy surveillance, or gastrointestinal symptoms. About 60.8% of participants were men, and the average age was 60.
Among 543 polyps detected and removed, 169 (31.3%) were adenomas, and 373 (68.7%) were nonadenomas. Of those, 325 (59.9%) were rectosigmoid polyps of 5 mm or less in diameter and eligible for analyses in the study. This included 44 adenomas (13.5%) and 281 nonadenomas (86.5%).
The two CADx systems were grouped as CADx-A for CAD EYE and CADx-B for GI Genius. CADx-A provided prediction output for all 325 rectosigmoid polyps of 5 mm or less, whereas CADx-B wasn’t able to provide output for six of the nonadenomas, which were excluded from the analysis.
The negative predictive value (NPV) for rectosigmoid polyps of 5 mm or less was 97% for CADx-A and 97.7% for CADx-B, the authors wrote. The American Society for Gastrointestinal Endoscopy recommends a threshold for optical diagnosis of at least 90%.
In addition, the sensitivity for adenomas was 81.8% for CADx-A and 86.4% for CADx-B. The accuracy of CADx-A was slightly higher, at 93.2%, as compared with 91.5% for CADx-B.
Based on AI prediction alone, 269 of 319 polyps (84.3%) with CADx-A and 260 of 319 polyps (81.5%) with CADx-B would have been classified as nonneoplastic and avoided removal. This corresponded to a specificity of 94.9% for CADx-A and 92.4% for CADx-B, which wasn’t significantly different, the authors wrote. Concordance in histology prediction between the two systems was 94.7%.
Based on the 2020 U.S. Multi-Society Task Force on Colorectal Cancer (USMSTF) guidelines, the agreement with histopathology in surveillance interval assignment was 84.7% for CADx-A and 89.2% for CADx-B. Based on the 2020 European Society of Gastrointestinal Endoscopy (ESGE) guidelines, the agreement was 98.3% for both systems.
For rectosigmoid polyps of 5 mm or less, the NPV of unassisted optical diagnosis was 97.8% for a high-confidence diagnosis, but it wasn’t significantly different from the NPV of CADx-A (96.9%) or CADx-B (97.6%). The NPV of a CADx-assisted optical diagnosis at high confidence was 97.7%, without statistically significant differences as compared with unassisted interpretation.
Based on the 2020 USMSTF and ESGE guidelines, the agreement between unassisted interpretation and histopathology in surveillance interval assignment was 92.6% and 98.9%, respectively. There was total agreement between unassisted interpretation and CADx-assisted interpretation in surveillance interval assignment based on both guidelines.
As in previous findings, unassisted endoscopic diagnosis was on par with CADx-assisted, both in technical accuracy and clinical outcomes. The study authors attributed the lack of additional benefit from CADx to a high performance of unassisted-endoscopist diagnosis, with the 97.8% NPV for rectosigmoid polyps and 90% or greater concordance in postpolypectomy surveillance intervals with histology. In addition, a human endoscopist was the only one to achieve 90% or greater agreement in postpolypectomy surveillance intervals under the U.S. guidelines, mainly due to a very high specificity.
“This confirms the complexity of the human-machine interaction that should not be marginalized in the stand-alone performance of the machine,” the authors wrote.
However, the high accuracy of unassisted endoscopists in the academic center in Italy is unlikely to mirror the real performance in community settings, they added. Future studies should focus on nontertiary centers to show the additional benefit, if any, that CADx provides for leave-in-situ colorectal polyps.
“A high degree of concordance in clinical outcomes was shown when directly comparing in vivo two different systems of CADx,” the authors concluded. “This reassured our confidence in the standardization of performance that may be achieved with the incorporation of AI in clinical practice, irrespective of the availability of multiple systems.”
The study authors declared no funding source for this study. Several authors reported consulting relationships with numerous companies, including Fuji and Medtronic, which make the CAD EYE and GI Genius systems, respectively.
Colonoscopy is the gold standard test to reduce an individual’s chance of developing colorectal cancer. The latest tool to improve colonoscopy outcomes is integrating artificial intelligence (AI) during the exam. AI systems offer both computer aided detection (CADe) as well as diagnosis (CADx). Accurate CADx could lead to a cost-effective strategy of removing only neoplastic polyps.
The study by Hassan et al. compared two AI CADx systems for optical diagnosis of colorectal polyps ≤ 5 mm. Polyps were simultaneously evaluated by both AI systems, but initially the endoscopist performed a CADx unassisted diagnosis. The two systems (CAD EYE [Fujifilm Co.] and GI Genius [Medtronic]) had similar specificity: 94.9% and 92.4%, respectively. Furthermore, the systems demonstrated negative predictive values of 96.9% and 97.6%, respectively, which exceeds the American Society of Gastrointestinal Endoscopy’s threshold of at least 90%.
A surprising finding was the unassisted endoscopist before CADx interpretation had a polyp diagnosis accuracy of 97.8%, resulting in negligible benefit when CADx was activated. However, this level of polyp interpretation is likely lower in community practice, but clinical trials will be needed.
There is rapid development of CADx and CADe systems entering the clinical realm of colonoscopy. It is critical to have the ability to objectively review the performance of these AI systems in a real-life clinical setting to assess accuracy for both CADx and CADe. Clinicians must balance striving for high quality colonoscopy outcomes with the cost of innovative technology like AI. However, it is reassuring that the initial CADx systems have similar high-performance accuracy for polyp interpretation, since most practices will incorporate a single system. Future studies will be needed to compare not only the accuracy of AI platforms offering CADx and CADe, but also the many other features that will be entering the endoscopy space.
Seth A. Gross, MD, is professor of medicine at NYU Grossman School of Medicine and clinical chief of gastroenterology and hepatology at NYU Langone Health. He disclosed financial relationships with Medtronic, Olympus, Iterative Scopes, and Micro-Tech Endoscopy.
Colonoscopy is the gold standard test to reduce an individual’s chance of developing colorectal cancer. The latest tool to improve colonoscopy outcomes is integrating artificial intelligence (AI) during the exam. AI systems offer both computer aided detection (CADe) as well as diagnosis (CADx). Accurate CADx could lead to a cost-effective strategy of removing only neoplastic polyps.
The study by Hassan et al. compared two AI CADx systems for optical diagnosis of colorectal polyps ≤ 5 mm. Polyps were simultaneously evaluated by both AI systems, but initially the endoscopist performed a CADx unassisted diagnosis. The two systems (CAD EYE [Fujifilm Co.] and GI Genius [Medtronic]) had similar specificity: 94.9% and 92.4%, respectively. Furthermore, the systems demonstrated negative predictive values of 96.9% and 97.6%, respectively, which exceeds the American Society of Gastrointestinal Endoscopy’s threshold of at least 90%.
A surprising finding was the unassisted endoscopist before CADx interpretation had a polyp diagnosis accuracy of 97.8%, resulting in negligible benefit when CADx was activated. However, this level of polyp interpretation is likely lower in community practice, but clinical trials will be needed.
There is rapid development of CADx and CADe systems entering the clinical realm of colonoscopy. It is critical to have the ability to objectively review the performance of these AI systems in a real-life clinical setting to assess accuracy for both CADx and CADe. Clinicians must balance striving for high quality colonoscopy outcomes with the cost of innovative technology like AI. However, it is reassuring that the initial CADx systems have similar high-performance accuracy for polyp interpretation, since most practices will incorporate a single system. Future studies will be needed to compare not only the accuracy of AI platforms offering CADx and CADe, but also the many other features that will be entering the endoscopy space.
Seth A. Gross, MD, is professor of medicine at NYU Grossman School of Medicine and clinical chief of gastroenterology and hepatology at NYU Langone Health. He disclosed financial relationships with Medtronic, Olympus, Iterative Scopes, and Micro-Tech Endoscopy.
Colonoscopy is the gold standard test to reduce an individual’s chance of developing colorectal cancer. The latest tool to improve colonoscopy outcomes is integrating artificial intelligence (AI) during the exam. AI systems offer both computer aided detection (CADe) as well as diagnosis (CADx). Accurate CADx could lead to a cost-effective strategy of removing only neoplastic polyps.
The study by Hassan et al. compared two AI CADx systems for optical diagnosis of colorectal polyps ≤ 5 mm. Polyps were simultaneously evaluated by both AI systems, but initially the endoscopist performed a CADx unassisted diagnosis. The two systems (CAD EYE [Fujifilm Co.] and GI Genius [Medtronic]) had similar specificity: 94.9% and 92.4%, respectively. Furthermore, the systems demonstrated negative predictive values of 96.9% and 97.6%, respectively, which exceeds the American Society of Gastrointestinal Endoscopy’s threshold of at least 90%.
A surprising finding was the unassisted endoscopist before CADx interpretation had a polyp diagnosis accuracy of 97.8%, resulting in negligible benefit when CADx was activated. However, this level of polyp interpretation is likely lower in community practice, but clinical trials will be needed.
There is rapid development of CADx and CADe systems entering the clinical realm of colonoscopy. It is critical to have the ability to objectively review the performance of these AI systems in a real-life clinical setting to assess accuracy for both CADx and CADe. Clinicians must balance striving for high quality colonoscopy outcomes with the cost of innovative technology like AI. However, it is reassuring that the initial CADx systems have similar high-performance accuracy for polyp interpretation, since most practices will incorporate a single system. Future studies will be needed to compare not only the accuracy of AI platforms offering CADx and CADe, but also the many other features that will be entering the endoscopy space.
Seth A. Gross, MD, is professor of medicine at NYU Grossman School of Medicine and clinical chief of gastroenterology and hepatology at NYU Langone Health. He disclosed financial relationships with Medtronic, Olympus, Iterative Scopes, and Micro-Tech Endoscopy.
In a head-to-head comparison, two commercially available computer-aided diagnosis systems appeared clinically equivalent for the optical diagnosis of small colorectal polyps, according to a research letter published in Gastroenterology.
For the optical diagnosis of diminutive colorectal polyps, the comparable performances of both CAD EYE (Fujifilm Co.) and GI Genius (Medtronic) met cutoff guidelines to implement the cost-saving leave-in-situ and resect-and-discard strategies, wrote Cesare Hassan, MD, PhD, associate professor of gastroenterology at Humanitas University and member of the endoscopy unit at Humanitas Clinical Research Hospital in Milan, and colleagues.
“Screening colonoscopy is effective in reducing colorectal cancer risk but also represents a substantial financial burden,” the authors wrote. “Novel strategies based on artificial intelligence may enable targeted removal only of polyps deemed to be neoplastic, thus reducing patient burden for unnecessary removal of nonneoplastic polyps and reducing costs for histopathology.”
Several computer-aided diagnosis (CADx) systems are commercially available for optical diagnosis of colorectal polyps, the authors wrote. However, each artificial intelligence (AI) system has been trained and validated with different polyp datasets, which may contribute to variability and affect the clinical outcome of optical diagnosis-based strategies.
Dr. Hassan and colleagues conducted a prospective comparison trial at a single center to look at the real-life performances of two CADx systems on optical diagnosis of polyps smaller than 5 mm.
At colonoscopy, the same polyp was visualized by the same endoscopist on two different monitors simultaneously with the respective output from each of the two CADx systems. Pre- and post-CADx human diagnoses were also collected.
Between January 2022 and March 2022, 176 consecutive patients age 40 and older underwent colonoscopy for colorectal cancer screening, polypectomy surveillance, or gastrointestinal symptoms. About 60.8% of participants were men, and the average age was 60.
Among 543 polyps detected and removed, 169 (31.3%) were adenomas, and 373 (68.7%) were nonadenomas. Of those, 325 (59.9%) were rectosigmoid polyps of 5 mm or less in diameter and eligible for analyses in the study. This included 44 adenomas (13.5%) and 281 nonadenomas (86.5%).
The two CADx systems were grouped as CADx-A for CAD EYE and CADx-B for GI Genius. CADx-A provided prediction output for all 325 rectosigmoid polyps of 5 mm or less, whereas CADx-B wasn’t able to provide output for six of the nonadenomas, which were excluded from the analysis.
The negative predictive value (NPV) for rectosigmoid polyps of 5 mm or less was 97% for CADx-A and 97.7% for CADx-B, the authors wrote. The American Society for Gastrointestinal Endoscopy recommends a threshold for optical diagnosis of at least 90%.
In addition, the sensitivity for adenomas was 81.8% for CADx-A and 86.4% for CADx-B. The accuracy of CADx-A was slightly higher, at 93.2%, as compared with 91.5% for CADx-B.
Based on AI prediction alone, 269 of 319 polyps (84.3%) with CADx-A and 260 of 319 polyps (81.5%) with CADx-B would have been classified as nonneoplastic and avoided removal. This corresponded to a specificity of 94.9% for CADx-A and 92.4% for CADx-B, which wasn’t significantly different, the authors wrote. Concordance in histology prediction between the two systems was 94.7%.
Based on the 2020 U.S. Multi-Society Task Force on Colorectal Cancer (USMSTF) guidelines, the agreement with histopathology in surveillance interval assignment was 84.7% for CADx-A and 89.2% for CADx-B. Based on the 2020 European Society of Gastrointestinal Endoscopy (ESGE) guidelines, the agreement was 98.3% for both systems.
For rectosigmoid polyps of 5 mm or less, the NPV of unassisted optical diagnosis was 97.8% for a high-confidence diagnosis, but it wasn’t significantly different from the NPV of CADx-A (96.9%) or CADx-B (97.6%). The NPV of a CADx-assisted optical diagnosis at high confidence was 97.7%, without statistically significant differences as compared with unassisted interpretation.
Based on the 2020 USMSTF and ESGE guidelines, the agreement between unassisted interpretation and histopathology in surveillance interval assignment was 92.6% and 98.9%, respectively. There was total agreement between unassisted interpretation and CADx-assisted interpretation in surveillance interval assignment based on both guidelines.
As in previous findings, unassisted endoscopic diagnosis was on par with CADx-assisted, both in technical accuracy and clinical outcomes. The study authors attributed the lack of additional benefit from CADx to a high performance of unassisted-endoscopist diagnosis, with the 97.8% NPV for rectosigmoid polyps and 90% or greater concordance in postpolypectomy surveillance intervals with histology. In addition, a human endoscopist was the only one to achieve 90% or greater agreement in postpolypectomy surveillance intervals under the U.S. guidelines, mainly due to a very high specificity.
“This confirms the complexity of the human-machine interaction that should not be marginalized in the stand-alone performance of the machine,” the authors wrote.
However, the high accuracy of unassisted endoscopists in the academic center in Italy is unlikely to mirror the real performance in community settings, they added. Future studies should focus on nontertiary centers to show the additional benefit, if any, that CADx provides for leave-in-situ colorectal polyps.
“A high degree of concordance in clinical outcomes was shown when directly comparing in vivo two different systems of CADx,” the authors concluded. “This reassured our confidence in the standardization of performance that may be achieved with the incorporation of AI in clinical practice, irrespective of the availability of multiple systems.”
The study authors declared no funding source for this study. Several authors reported consulting relationships with numerous companies, including Fuji and Medtronic, which make the CAD EYE and GI Genius systems, respectively.
In a head-to-head comparison, two commercially available computer-aided diagnosis systems appeared clinically equivalent for the optical diagnosis of small colorectal polyps, according to a research letter published in Gastroenterology.
For the optical diagnosis of diminutive colorectal polyps, the comparable performances of both CAD EYE (Fujifilm Co.) and GI Genius (Medtronic) met cutoff guidelines to implement the cost-saving leave-in-situ and resect-and-discard strategies, wrote Cesare Hassan, MD, PhD, associate professor of gastroenterology at Humanitas University and member of the endoscopy unit at Humanitas Clinical Research Hospital in Milan, and colleagues.
“Screening colonoscopy is effective in reducing colorectal cancer risk but also represents a substantial financial burden,” the authors wrote. “Novel strategies based on artificial intelligence may enable targeted removal only of polyps deemed to be neoplastic, thus reducing patient burden for unnecessary removal of nonneoplastic polyps and reducing costs for histopathology.”
Several computer-aided diagnosis (CADx) systems are commercially available for optical diagnosis of colorectal polyps, the authors wrote. However, each artificial intelligence (AI) system has been trained and validated with different polyp datasets, which may contribute to variability and affect the clinical outcome of optical diagnosis-based strategies.
Dr. Hassan and colleagues conducted a prospective comparison trial at a single center to look at the real-life performances of two CADx systems on optical diagnosis of polyps smaller than 5 mm.
At colonoscopy, the same polyp was visualized by the same endoscopist on two different monitors simultaneously with the respective output from each of the two CADx systems. Pre- and post-CADx human diagnoses were also collected.
Between January 2022 and March 2022, 176 consecutive patients age 40 and older underwent colonoscopy for colorectal cancer screening, polypectomy surveillance, or gastrointestinal symptoms. About 60.8% of participants were men, and the average age was 60.
Among 543 polyps detected and removed, 169 (31.3%) were adenomas, and 373 (68.7%) were nonadenomas. Of those, 325 (59.9%) were rectosigmoid polyps of 5 mm or less in diameter and eligible for analyses in the study. This included 44 adenomas (13.5%) and 281 nonadenomas (86.5%).
The two CADx systems were grouped as CADx-A for CAD EYE and CADx-B for GI Genius. CADx-A provided prediction output for all 325 rectosigmoid polyps of 5 mm or less, whereas CADx-B wasn’t able to provide output for six of the nonadenomas, which were excluded from the analysis.
The negative predictive value (NPV) for rectosigmoid polyps of 5 mm or less was 97% for CADx-A and 97.7% for CADx-B, the authors wrote. The American Society for Gastrointestinal Endoscopy recommends a threshold for optical diagnosis of at least 90%.
In addition, the sensitivity for adenomas was 81.8% for CADx-A and 86.4% for CADx-B. The accuracy of CADx-A was slightly higher, at 93.2%, as compared with 91.5% for CADx-B.
Based on AI prediction alone, 269 of 319 polyps (84.3%) with CADx-A and 260 of 319 polyps (81.5%) with CADx-B would have been classified as nonneoplastic and avoided removal. This corresponded to a specificity of 94.9% for CADx-A and 92.4% for CADx-B, which wasn’t significantly different, the authors wrote. Concordance in histology prediction between the two systems was 94.7%.
Based on the 2020 U.S. Multi-Society Task Force on Colorectal Cancer (USMSTF) guidelines, the agreement with histopathology in surveillance interval assignment was 84.7% for CADx-A and 89.2% for CADx-B. Based on the 2020 European Society of Gastrointestinal Endoscopy (ESGE) guidelines, the agreement was 98.3% for both systems.
For rectosigmoid polyps of 5 mm or less, the NPV of unassisted optical diagnosis was 97.8% for a high-confidence diagnosis, but it wasn’t significantly different from the NPV of CADx-A (96.9%) or CADx-B (97.6%). The NPV of a CADx-assisted optical diagnosis at high confidence was 97.7%, without statistically significant differences as compared with unassisted interpretation.
Based on the 2020 USMSTF and ESGE guidelines, the agreement between unassisted interpretation and histopathology in surveillance interval assignment was 92.6% and 98.9%, respectively. There was total agreement between unassisted interpretation and CADx-assisted interpretation in surveillance interval assignment based on both guidelines.
As in previous findings, unassisted endoscopic diagnosis was on par with CADx-assisted, both in technical accuracy and clinical outcomes. The study authors attributed the lack of additional benefit from CADx to a high performance of unassisted-endoscopist diagnosis, with the 97.8% NPV for rectosigmoid polyps and 90% or greater concordance in postpolypectomy surveillance intervals with histology. In addition, a human endoscopist was the only one to achieve 90% or greater agreement in postpolypectomy surveillance intervals under the U.S. guidelines, mainly due to a very high specificity.
“This confirms the complexity of the human-machine interaction that should not be marginalized in the stand-alone performance of the machine,” the authors wrote.
However, the high accuracy of unassisted endoscopists in the academic center in Italy is unlikely to mirror the real performance in community settings, they added. Future studies should focus on nontertiary centers to show the additional benefit, if any, that CADx provides for leave-in-situ colorectal polyps.
“A high degree of concordance in clinical outcomes was shown when directly comparing in vivo two different systems of CADx,” the authors concluded. “This reassured our confidence in the standardization of performance that may be achieved with the incorporation of AI in clinical practice, irrespective of the availability of multiple systems.”
The study authors declared no funding source for this study. Several authors reported consulting relationships with numerous companies, including Fuji and Medtronic, which make the CAD EYE and GI Genius systems, respectively.
FROM GASTROENTEROLOGY