Researchers question validity of NCCN guidelines

Photo by Rhoda Baer

New research suggests guidelines from the National Comprehensive Cancer Network (NCCN) may sometimes be supported by low-quality evidence or no evidence at all.

Researchers compared NCCN recommendations for cancer drugs to US cancer drug approvals over a 5-year period.

Thirty-nine percent of NCCN’s treatment recommendations did not coincide with uses approved by the US Food and Drug Administration (FDA).

For most of these recommendations (84%), NCCN did not provide supporting data from randomized, phase 3 trials.

For 36% of the recommendations, NCCN gave no supporting evidence.

Vinay Prasad, MD, of Oregon Health & Science University in Portland, Oregon, and his colleagues reported these findings in The BMJ.

Dr Prasad and his colleagues compared FDA approvals of cancer drugs between 2011 and 2015 with NCCN recommendations as of March 25, 2016.

When NCCN made recommendations beyond FDA approvals, the researchers evaluated the evidence used to support those recommendations.

Forty-seven new cancer drugs were approved by the FDA for 69 indications between 2011 and 2015. NCCN recommended the 47 drugs for 113 indications, including the 69 FDA-approved indications.

So 39% (n=44) of NCCN’s recommendations were not approved by the FDA, and NCCN gave the following evidence to support these recommendations:

• No evidence—36% (n=16)
• Phase 2 trial without randomization—30% (n=13)
• Randomized, phase 3 trial—16% (n=7)
• Phase 2 trial with randomization—7% (n=3)
• Case report or series of less than 5 patients—5% (n=2)
• Book chapter or review article—2% (n=1)
• Phase 1 trial—2% (n=1)
• Ongoing trial—2% (n=1).

Dr Prasad and his colleagues did point out that not all FDA approvals are supported by randomized, phase 3 trials.

And when the team followed-up 21 months after their initial analysis, they found that 6 of the 44 (14%) additional recommendations by NCCN had received FDA approval.

The researchers also noted that they did not search for independent evidence to support NCCN recommendations beyond the references NCCN provided. So some of the recommendations may have had more or better supporting evidence than what was provided.

Still, the team said these results suggest NCCN “frequently” makes recommendations that go beyond FDA approvals and “often fails to cite evidence or relies on low levels of evidence.” Therefore, NCCN should cite all evidence used to formulate its recommendations.

NCCN argues that it does provide ample evidence to support the recommendations in its guidelines.

“The NCCN guidelines contain more than 24,500 references to inform users of the evidence used in making its decisions,” said Robert W. Carlson, MD, chief executive officer of NCCN.

“These data are supplemented by the analysis of the available evidence by expert clinician researchers and patient advocates who evaluate each recommendation and come to consensus. Each recommendation is labeled with a Category of Evidence, and the vast majority of those for systemic therapies are accompanied by Evidence Blocks, which outline, on 1-5 scales, the efficacy, safety, quality of the evidence, consistency of the evidence, and affordability of the treatment.”

Photo by Rhoda Baer

New research suggests guidelines from the National Comprehensive Cancer Network (NCCN) may sometimes be supported by low-quality evidence or no evidence at all.

Researchers compared NCCN recommendations for cancer drugs to US cancer drug approvals over a 5-year period.

Thirty-nine percent of NCCN’s treatment recommendations did not coincide with uses approved by the US Food and Drug Administration (FDA).

For most of these recommendations (84%), NCCN did not provide supporting data from randomized, phase 3 trials.

For 36% of the recommendations, NCCN gave no supporting evidence.

Vinay Prasad, MD, of Oregon Health & Science University in Portland, Oregon, and his colleagues reported these findings in The BMJ.

Dr Prasad and his colleagues compared FDA approvals of cancer drugs between 2011 and 2015 with NCCN recommendations as of March 25, 2016.

When NCCN made recommendations beyond FDA approvals, the researchers evaluated the evidence used to support those recommendations.

Forty-seven new cancer drugs were approved by the FDA for 69 indications between 2011 and 2015. NCCN recommended the 47 drugs for 113 indications, including the 69 FDA-approved indications.

So 39% (n=44) of NCCN’s recommendations were not approved by the FDA, and NCCN gave the following evidence to support these recommendations:

• No evidence—36% (n=16)
• Phase 2 trial without randomization—30% (n=13)
• Randomized, phase 3 trial—16% (n=7)
• Phase 2 trial with randomization—7% (n=3)
• Case report or series of less than 5 patients—5% (n=2)
• Book chapter or review article—2% (n=1)
• Phase 1 trial—2% (n=1)
• Ongoing trial—2% (n=1).

Dr Prasad and his colleagues did point out that not all FDA approvals are supported by randomized, phase 3 trials.

And when the team followed-up 21 months after their initial analysis, they found that 6 of the 44 (14%) additional recommendations by NCCN had received FDA approval.

The researchers also noted that they did not search for independent evidence to support NCCN recommendations beyond the references NCCN provided. So some of the recommendations may have had more or better supporting evidence than what was provided.

Still, the team said these results suggest NCCN “frequently” makes recommendations that go beyond FDA approvals and “often fails to cite evidence or relies on low levels of evidence.” Therefore, NCCN should cite all evidence used to formulate its recommendations.

NCCN argues that it does provide ample evidence to support the recommendations in its guidelines.

“The NCCN guidelines contain more than 24,500 references to inform users of the evidence used in making its decisions,” said Robert W. Carlson, MD, chief executive officer of NCCN.

“These data are supplemented by the analysis of the available evidence by expert clinician researchers and patient advocates who evaluate each recommendation and come to consensus. Each recommendation is labeled with a Category of Evidence, and the vast majority of those for systemic therapies are accompanied by Evidence Blocks, which outline, on 1-5 scales, the efficacy, safety, quality of the evidence, consistency of the evidence, and affordability of the treatment.”

Photo by Rhoda Baer

New research suggests guidelines from the National Comprehensive Cancer Network (NCCN) may sometimes be supported by low-quality evidence or no evidence at all.

Researchers compared NCCN recommendations for cancer drugs to US cancer drug approvals over a 5-year period.

Thirty-nine percent of NCCN’s treatment recommendations did not coincide with uses approved by the US Food and Drug Administration (FDA).

For most of these recommendations (84%), NCCN did not provide supporting data from randomized, phase 3 trials.

For 36% of the recommendations, NCCN gave no supporting evidence.

Vinay Prasad, MD, of Oregon Health & Science University in Portland, Oregon, and his colleagues reported these findings in The BMJ.

Dr Prasad and his colleagues compared FDA approvals of cancer drugs between 2011 and 2015 with NCCN recommendations as of March 25, 2016.

When NCCN made recommendations beyond FDA approvals, the researchers evaluated the evidence used to support those recommendations.

Forty-seven new cancer drugs were approved by the FDA for 69 indications between 2011 and 2015. NCCN recommended the 47 drugs for 113 indications, including the 69 FDA-approved indications.

So 39% (n=44) of NCCN’s recommendations were not approved by the FDA, and NCCN gave the following evidence to support these recommendations:

• No evidence—36% (n=16)
• Phase 2 trial without randomization—30% (n=13)
• Randomized, phase 3 trial—16% (n=7)
• Phase 2 trial with randomization—7% (n=3)
• Case report or series of less than 5 patients—5% (n=2)
• Book chapter or review article—2% (n=1)
• Phase 1 trial—2% (n=1)
• Ongoing trial—2% (n=1).

Dr Prasad and his colleagues did point out that not all FDA approvals are supported by randomized, phase 3 trials.

And when the team followed-up 21 months after their initial analysis, they found that 6 of the 44 (14%) additional recommendations by NCCN had received FDA approval.

The researchers also noted that they did not search for independent evidence to support NCCN recommendations beyond the references NCCN provided. So some of the recommendations may have had more or better supporting evidence than what was provided.

Still, the team said these results suggest NCCN “frequently” makes recommendations that go beyond FDA approvals and “often fails to cite evidence or relies on low levels of evidence.” Therefore, NCCN should cite all evidence used to formulate its recommendations.

NCCN argues that it does provide ample evidence to support the recommendations in its guidelines.

“The NCCN guidelines contain more than 24,500 references to inform users of the evidence used in making its decisions,” said Robert W. Carlson, MD, chief executive officer of NCCN.

“These data are supplemented by the analysis of the available evidence by expert clinician researchers and patient advocates who evaluate each recommendation and come to consensus. Each recommendation is labeled with a Category of Evidence, and the vast majority of those for systemic therapies are accompanied by Evidence Blocks, which outline, on 1-5 scales, the efficacy, safety, quality of the evidence, consistency of the evidence, and affordability of the treatment.”