Pancreatic Surgery Scorecard Aligns With IOM Quality Metrics

SAN DIEGO – A proposed quality measure for pancreatic surgery has the potential to assess performance more thoroughly than current volume and mortality measures, judging by surgeons’ responses to a recent survey.

The 12-item "Quality Scorecard," developed by Dr. Brian T. Kalish of Beth Israel Deaconess Medical Center, Boston, and a team of pancreatic surgeons from multiple academic medical centers, consists of actionable and meaningful measures. The scorecard is aligned with the Institute of Medicine’s health care quality domains: safety, timeliness, effectiveness, patient centeredness, efficiency, and equitability.

"Traditional quality metrics in high-acuity surgery on their own cannot measure or satisfy the [IOM] domains," Dr. Kalish explained in a plenary presentation at the annual Digestive Disease Week. "Our goal was to evaluate the need for broader quality metrics and whether such broader metrics would align to contemporary IOM domains."

"We expect the scorecard to reveal quality to an extent that volume and mortality alone cannot."

Toward this end, the development team worked with a professional market research firm to create a web-based survey and distribute it to expert pancreatic surgeons identified through specialty societies and verified by survey demographics, Dr. Kalish explained. "The survey asked respondents to rank [62] proposed quality metrics on level of importance, from essential to not important, and to align the metric to one or more of the [IOM] quality domains." Points were awarded for level of importance and multidomain alignment, and the two scores for a given quality metric were averaged to render a total quality score that was then normalized to a 100-point scale, he said.

The 21% survey response rate represented 106 surgeons primarily from academic medical centers in North America who perform an average of 43 pancreatic operations per year, said Dr. Kalish. The need for improved quality metrics was indicated by 90% of the respondents, while 81% believed that a "quality scorecard" in pancreatic surgery would probably or definitely be of value, he reported. More than one-third of the proposed metrics aligned to more than one IOM domain, and at least half of the respondents rated these as essential or very important, he said.

Of the 62 metrics, 12 emerged with the highest total quality score. In rank order, they are: multidisciplinary services for pancreatic diseases, major complication rate, perioperative mortality, overall complication rate, incidence of postoperative hemorrhage, venous thromboembolism prophylaxis, patients with malignancy who undergo adjuvant therapy, readmission rates (30 day, 90 day, total), incidence of postoperative pancreatic fistula, timely and appropriate perioperative antibiotics, survival rates (1 year and 5 year), and timing from diagnosis to surgical consultation.

"The metrics related to mortality, the rate and severity of complications, and access to multidisciplinary services for pancreatic disease had the highest total quality scores; technical and perioperative metrics had intermediate scores; and metrics related to patient satisfaction with care, costs of care, and patient demographics had the lowest total quality scores," Dr. Kalish observed. With respect to the IOM domains, "the least represented domains were equitability, efficiency, and patient-centeredness," he said.

Although the actual performance thresholds for each of the metrics require further definition and validation, "we expect the scorecard to reveal quality to an extent that volume and mortality alone cannot," Dr. Kalish stated, noting that the development process is ongoing.

Future efforts include the organization of patient focus groups and a formal survey of patients and family members to attain insight into which quality metrics are important to those receiving care, as well as a multicenter prospective validation.

Dr. Kalish reported having no relevant financial conflicts of interest.

SAN DIEGO – A proposed quality measure for pancreatic surgery has the potential to assess performance more thoroughly than current volume and mortality measures, judging by surgeons’ responses to a recent survey.

The 12-item "Quality Scorecard," developed by Dr. Brian T. Kalish of Beth Israel Deaconess Medical Center, Boston, and a team of pancreatic surgeons from multiple academic medical centers, consists of actionable and meaningful measures. The scorecard is aligned with the Institute of Medicine’s health care quality domains: safety, timeliness, effectiveness, patient centeredness, efficiency, and equitability.

"Traditional quality metrics in high-acuity surgery on their own cannot measure or satisfy the [IOM] domains," Dr. Kalish explained in a plenary presentation at the annual Digestive Disease Week. "Our goal was to evaluate the need for broader quality metrics and whether such broader metrics would align to contemporary IOM domains."

"We expect the scorecard to reveal quality to an extent that volume and mortality alone cannot."

Toward this end, the development team worked with a professional market research firm to create a web-based survey and distribute it to expert pancreatic surgeons identified through specialty societies and verified by survey demographics, Dr. Kalish explained. "The survey asked respondents to rank [62] proposed quality metrics on level of importance, from essential to not important, and to align the metric to one or more of the [IOM] quality domains." Points were awarded for level of importance and multidomain alignment, and the two scores for a given quality metric were averaged to render a total quality score that was then normalized to a 100-point scale, he said.

The 21% survey response rate represented 106 surgeons primarily from academic medical centers in North America who perform an average of 43 pancreatic operations per year, said Dr. Kalish. The need for improved quality metrics was indicated by 90% of the respondents, while 81% believed that a "quality scorecard" in pancreatic surgery would probably or definitely be of value, he reported. More than one-third of the proposed metrics aligned to more than one IOM domain, and at least half of the respondents rated these as essential or very important, he said.

Of the 62 metrics, 12 emerged with the highest total quality score. In rank order, they are: multidisciplinary services for pancreatic diseases, major complication rate, perioperative mortality, overall complication rate, incidence of postoperative hemorrhage, venous thromboembolism prophylaxis, patients with malignancy who undergo adjuvant therapy, readmission rates (30 day, 90 day, total), incidence of postoperative pancreatic fistula, timely and appropriate perioperative antibiotics, survival rates (1 year and 5 year), and timing from diagnosis to surgical consultation.

"The metrics related to mortality, the rate and severity of complications, and access to multidisciplinary services for pancreatic disease had the highest total quality scores; technical and perioperative metrics had intermediate scores; and metrics related to patient satisfaction with care, costs of care, and patient demographics had the lowest total quality scores," Dr. Kalish observed. With respect to the IOM domains, "the least represented domains were equitability, efficiency, and patient-centeredness," he said.

Although the actual performance thresholds for each of the metrics require further definition and validation, "we expect the scorecard to reveal quality to an extent that volume and mortality alone cannot," Dr. Kalish stated, noting that the development process is ongoing.

Future efforts include the organization of patient focus groups and a formal survey of patients and family members to attain insight into which quality metrics are important to those receiving care, as well as a multicenter prospective validation.

Dr. Kalish reported having no relevant financial conflicts of interest.

SAN DIEGO – A proposed quality measure for pancreatic surgery has the potential to assess performance more thoroughly than current volume and mortality measures, judging by surgeons’ responses to a recent survey.

The 12-item "Quality Scorecard," developed by Dr. Brian T. Kalish of Beth Israel Deaconess Medical Center, Boston, and a team of pancreatic surgeons from multiple academic medical centers, consists of actionable and meaningful measures. The scorecard is aligned with the Institute of Medicine’s health care quality domains: safety, timeliness, effectiveness, patient centeredness, efficiency, and equitability.

"Traditional quality metrics in high-acuity surgery on their own cannot measure or satisfy the [IOM] domains," Dr. Kalish explained in a plenary presentation at the annual Digestive Disease Week. "Our goal was to evaluate the need for broader quality metrics and whether such broader metrics would align to contemporary IOM domains."

"We expect the scorecard to reveal quality to an extent that volume and mortality alone cannot."

Toward this end, the development team worked with a professional market research firm to create a web-based survey and distribute it to expert pancreatic surgeons identified through specialty societies and verified by survey demographics, Dr. Kalish explained. "The survey asked respondents to rank [62] proposed quality metrics on level of importance, from essential to not important, and to align the metric to one or more of the [IOM] quality domains." Points were awarded for level of importance and multidomain alignment, and the two scores for a given quality metric were averaged to render a total quality score that was then normalized to a 100-point scale, he said.

The 21% survey response rate represented 106 surgeons primarily from academic medical centers in North America who perform an average of 43 pancreatic operations per year, said Dr. Kalish. The need for improved quality metrics was indicated by 90% of the respondents, while 81% believed that a "quality scorecard" in pancreatic surgery would probably or definitely be of value, he reported. More than one-third of the proposed metrics aligned to more than one IOM domain, and at least half of the respondents rated these as essential or very important, he said.

Of the 62 metrics, 12 emerged with the highest total quality score. In rank order, they are: multidisciplinary services for pancreatic diseases, major complication rate, perioperative mortality, overall complication rate, incidence of postoperative hemorrhage, venous thromboembolism prophylaxis, patients with malignancy who undergo adjuvant therapy, readmission rates (30 day, 90 day, total), incidence of postoperative pancreatic fistula, timely and appropriate perioperative antibiotics, survival rates (1 year and 5 year), and timing from diagnosis to surgical consultation.

"The metrics related to mortality, the rate and severity of complications, and access to multidisciplinary services for pancreatic disease had the highest total quality scores; technical and perioperative metrics had intermediate scores; and metrics related to patient satisfaction with care, costs of care, and patient demographics had the lowest total quality scores," Dr. Kalish observed. With respect to the IOM domains, "the least represented domains were equitability, efficiency, and patient-centeredness," he said.

Although the actual performance thresholds for each of the metrics require further definition and validation, "we expect the scorecard to reveal quality to an extent that volume and mortality alone cannot," Dr. Kalish stated, noting that the development process is ongoing.

Future efforts include the organization of patient focus groups and a formal survey of patients and family members to attain insight into which quality metrics are important to those receiving care, as well as a multicenter prospective validation.

Dr. Kalish reported having no relevant financial conflicts of interest.