Walter W. Rosser

With more than 1000 new guidelines produced annually over the past decade, it is impossible for the practicing family physician to determine which ones should be adapted into their clinical practice. The Ontario Ministry of Health and Long-Term Care and the Ontario Medical Association formed the Guideline Advisory Committee (GAC) in 1997 to assess and disseminate guidelines that would improve the quality and utilization of health care services in the province. Over the past 3 years the GAC has developed a strategy to identify important topics, to rank guidelines published on these topics based on the quality of their development, and to reformat guidelines as necessary to make them user-friendly for implementation in clinical practice. The GAC is currently assessing a number of strategies to enhance the dissemination of selected guidelines to improve the quality of care delivered in the province.

Key points for clinicians

A method of selecting, reviewing, and endorsing clinical practice guidelines has been established in the province of Ontario, Canada. Recommended guideline summaries are posted on a Web site with links to full text for easy access by practicing physicians (www.gacguidelines.ca).

Strategies for the successful implementation and impact evaluation of recommended guidelines are currently in development.

Clinical practice guidelines are statements that are systematically developed to assist physisican and patient decisions about appropriate health care for specific clinical circumstances.¹ Published guidelines have become widely available through Internet technology; it has been estimated that more than 2500 exist. Most are produced by specific interest groups (eg, national societies and pharmaceutical companies), disseminated by publication in a medical journal or traditional mail, and seldom demonstrate any effect on clinical practice.² Such a large volume of guidelines creates confusion for clinicians who often do not follow any of them because of the time required to assess their quality.³

With this dilemma in mind, the GAC was formed with members representing the Ontario Medical Association (OMA), the Ministry of Health and Long-Term Care (MOHLTC) in the province of Ontario, and one ex-officio member of the Institute for Clinical Evaluative Sciences (ICES). The GAC determined its first priority was to identify the best-quality guidelines available for clinicians on selected topics and to then promote their dissemination across the province. The purpose of our paper is to describe the methods that have been developed over the last 3 years to identify high-quality guidelines and some of the strategies being proposed for their dissemination, implementation, and evaluation. We also identify the best-quality guidelines for 10 common conditions.

Methods to assess the development of clinical practice guidelines

Topic Selection

Using a number of parameters, the GAC initially produced a grid as an assessment tool to identify priority areas for guideline review. Table 1 shows the basic grid incorporating provincial utilization and cost data, outcomes research, feedback from clinicians or health care organizations, and a previously published list of common and important problems in family practice.⁴ Feedback from the OMA sections indicated considerable confusion resulting from conflicting advice in specific areas as to appropriate practice (eg, screening for osteoporosis and diabetes). Utilization data from the MOHLTC demonstrated that the use of numerous procedures had rapidly increased over previous years; for example, diagnostic ultrasound utilization increased 65% in 1998. Practicing physicians also identified areas where there was a need for guidelines to be developed because of a lack of evidence or unknown best practice. The committee took all these factors into account when producing a list of priority topics for guideline assessment Table 2.

Guideline Assessment and Recommendation

Once a topic was chosen for assessment, a literature search was conducted by University of Toronto librarians to find all guidelines published in English over the past 10 years on that specific topic. The search strategy included databases such as MEDLINE and HealthStar, and guideline Web sites such as the National Guideline Clearinghouse and the Canadian Medical Association’s Clinical Practice Guideline Infobase. Copies of all guidelines identified in the search were then obtained. A survey of associations and interest groups in Ontario was also made to determine whether there were any unpublished guidelines that we had not identified in this process.

Initially, members of the committee carried out a literature search to determine if there were any publications about scoring the quality of the process used to produce the guidelines. Our search found some processes, but none that directly suited our needs. As a result, the GAC embarked on the development of a guideline-scoring instrument. After a year of work we realized that it would likely take 2 to 3 more years to adequately validate the instrument, and thus a decision was made to adopt the Appraisal Instrument for Clinical Guidelines⁵ (available at: www.sghms.ac.uk/phs/hceu/form.htm) to help determine quality guidelines in each clinical area, supplemented by the tool developed by the committee. The Appraisal Instrument consists of 37 items addressing 3 dimensions Table 3. The classification system the committee is using to choose top-scoring guidelines after appraisal is as follows. An excellent guideline is one in which the majority of the dimensions (rigor of development, context and content, application) are well addressed by the guideline producers with minimal omission. The evidence is linked to the major recommendations, and the development process is robust. These types of guidelines are highly recommended.

A very good guideline is one in which many of the dimensions are addressed, and some of the recommendations are linked to evidence levels. Objectives and rationale for development are often clearly defined but may be lacking in other areas, such as application (eg, outcome measures, targets, risks, and benefits). These are generally well produced and useful for practicing clinicians and are recommended.

In a fair guideline, some of the dimensions are addressed, but there are some major omissions, often in terms of levels of evidence, literature search strategy, clarity, risks, and benefits. Often these documents are local adaptations of other guidelines. Information can sometimes be used as a general reference if user-friendly materials are incorporated but are generally not very useful as guidelines. These guidelines are recommended under special circumstances.

A poor guideline is one which most of the dimensions are not well addressed, if at all. Often, it is unclear who produced these documents, and there is no description of the individuals involved. Levels of evidence and literature search strategy are rarely included, and there is no description of the methods used to formulate the recommendations. These guidelines are of little use to practicing clinicians and are not recommended.

Recognizing that recommending guidelines based on the quality of the process by which they were produced and the evidence used in their development would be controversial, we felt it was extremely important to develop a rigorous and objective scoring methodology. Fellows from the Department of Family and Community Medicine at the University of Toronto and community-based family physician volunteers from the OMA were brought together in 5 workshops. Each workshop included approximately 20 participants and consisted of a half-day session on the objectives of the GAC, a detailed review of the Appraisal Instrument, and a hands-on session where all participants evaluated the same guideline. Scores were then openly declared, and a discussion held on discrepancies identified in the assessments in an attempt to standardize the process. At the end of the session, interested participants were provided with an additional 5 guidelines to assess in the subsequent 2 weeks. The resulting appraisals were evaluated for consistency and inter-rater reliability (results indicate that using the instrument as an initial filter to determine the best-quality guidelines in each clinical area is a valid approach). To date, 45 assessors have been trained and are reviewing guidelines on an ongoing basis. Each guideline is evaluated a total of 3 times by independent assessors. Those guidelines that have been selected for recommendation in a particular clinical area are then reviewed for clinical relevance and applicability to the Ontario context. More than 250 published guidelines have been identified and distributed to physician assessors in the clinical areas shown in Table 2.

Reformatting

The GAC is in the process of determining the user-friendliness of recommended guidelines. Not infrequently, guidelines that are found to be the most evidence-based and objective are hundreds of pages in length and would be extremely burdensome for the average family physician to use. We anticipate that guidelines found to be of excellent quality but not convenient for use in clinical practice will need to be reformatted into user-friendly summaries. Volunteer physicians from the community will be asked to evaluate such summaries and provide feedback for improvement.

Dissemination

Once the best-quality guideline(s) on a topic are identified and reformatted as necessary; we intend to mount them on the GAC Web site (www.gacguidelines.ca) for use by the profession and the general public. Table 4 shows the results of the guideline selection process for the first 10 clinical areas. The process for choosing guidelines is transparent so that practicing physicians can determine for themselves the usefulness and applicability of the recommendations. Only the most rigorously developed guidelines will be posted on the Web site in the form of structured summaries, although interested clinicians can obtain the outcome of nonrecommended guideline appraisals on request.

Continuing medical education literature on dissemination strategies indicates that a single method, such as posting information on a Web site or mailing guidelines to clinicians has a minimal effect on changing medical practice.⁶ The GAC is currently considering a number of options to enhance the dissemination of the best available guidelines. Since Ontario health data on diagnostic testing, hospitalization records, and office visits are collected provincially, it could be possible to measure clinical outcomes following the dissemination of evidence-based guidelines. We are currently working with provincial groups to disseminate guidelines through medical school continuing medical education (CME) division programs, peer presenter programs, small group CME programs, outreach facilitation programs, and a peer assessment program run by the provincial licensing body.

Conclusions

Over the past 3 years the GAC has developed a method to identify relevant guideline topics and assess the quality of the process by which the guidelines were developed. Clinically excellent guidelines may require some reformatting to make them user-friendly for implementation in clinical practice. The initial product of this process has been posted on the GAC Web site for access by the profession. The GAC is currently assessing and developing a number of strategies to more effectively disseminate guideline information and measure the impact of these interventions on the quality of medical care delivered to the people of Ontario. The GAC will report on the impact of these interventions to facilitate the exchange of successful implementation strategies across jurisdictions.

Acknowledgments

We thank the Physician Services Committee and the members of the Ontario Medical Association and the Ministry of Health and Long-Term Care for their support of this initiative. Conflict of Interest Statement: Dr Rosser and Dr Davis receive stipends for participation on the Guideline Advisory Committee. Ms Gilbart is employed full-time by the Committee through a grant from the Ministry of Health and Long-Term Care. Dr Rosser was a member of the CANMAT Depression Working Group which developed the top-scoring guideline in depression as chosen through the GAC assessment process.

With more than 1000 new guidelines produced annually over the past decade, it is impossible for the practicing family physician to determine which ones should be adapted into their clinical practice. The Ontario Ministry of Health and Long-Term Care and the Ontario Medical Association formed the Guideline Advisory Committee (GAC) in 1997 to assess and disseminate guidelines that would improve the quality and utilization of health care services in the province. Over the past 3 years the GAC has developed a strategy to identify important topics, to rank guidelines published on these topics based on the quality of their development, and to reformat guidelines as necessary to make them user-friendly for implementation in clinical practice. The GAC is currently assessing a number of strategies to enhance the dissemination of selected guidelines to improve the quality of care delivered in the province.

Key points for clinicians

A method of selecting, reviewing, and endorsing clinical practice guidelines has been established in the province of Ontario, Canada. Recommended guideline summaries are posted on a Web site with links to full text for easy access by practicing physicians (www.gacguidelines.ca).

Strategies for the successful implementation and impact evaluation of recommended guidelines are currently in development.

Clinical practice guidelines are statements that are systematically developed to assist physisican and patient decisions about appropriate health care for specific clinical circumstances.¹ Published guidelines have become widely available through Internet technology; it has been estimated that more than 2500 exist. Most are produced by specific interest groups (eg, national societies and pharmaceutical companies), disseminated by publication in a medical journal or traditional mail, and seldom demonstrate any effect on clinical practice.² Such a large volume of guidelines creates confusion for clinicians who often do not follow any of them because of the time required to assess their quality.³

With this dilemma in mind, the GAC was formed with members representing the Ontario Medical Association (OMA), the Ministry of Health and Long-Term Care (MOHLTC) in the province of Ontario, and one ex-officio member of the Institute for Clinical Evaluative Sciences (ICES). The GAC determined its first priority was to identify the best-quality guidelines available for clinicians on selected topics and to then promote their dissemination across the province. The purpose of our paper is to describe the methods that have been developed over the last 3 years to identify high-quality guidelines and some of the strategies being proposed for their dissemination, implementation, and evaluation. We also identify the best-quality guidelines for 10 common conditions.

Methods to assess the development of clinical practice guidelines

Topic Selection

Using a number of parameters, the GAC initially produced a grid as an assessment tool to identify priority areas for guideline review. Table 1 shows the basic grid incorporating provincial utilization and cost data, outcomes research, feedback from clinicians or health care organizations, and a previously published list of common and important problems in family practice.⁴ Feedback from the OMA sections indicated considerable confusion resulting from conflicting advice in specific areas as to appropriate practice (eg, screening for osteoporosis and diabetes). Utilization data from the MOHLTC demonstrated that the use of numerous procedures had rapidly increased over previous years; for example, diagnostic ultrasound utilization increased 65% in 1998. Practicing physicians also identified areas where there was a need for guidelines to be developed because of a lack of evidence or unknown best practice. The committee took all these factors into account when producing a list of priority topics for guideline assessment Table 2.

Guideline Assessment and Recommendation

Once a topic was chosen for assessment, a literature search was conducted by University of Toronto librarians to find all guidelines published in English over the past 10 years on that specific topic. The search strategy included databases such as MEDLINE and HealthStar, and guideline Web sites such as the National Guideline Clearinghouse and the Canadian Medical Association’s Clinical Practice Guideline Infobase. Copies of all guidelines identified in the search were then obtained. A survey of associations and interest groups in Ontario was also made to determine whether there were any unpublished guidelines that we had not identified in this process.

Initially, members of the committee carried out a literature search to determine if there were any publications about scoring the quality of the process used to produce the guidelines. Our search found some processes, but none that directly suited our needs. As a result, the GAC embarked on the development of a guideline-scoring instrument. After a year of work we realized that it would likely take 2 to 3 more years to adequately validate the instrument, and thus a decision was made to adopt the Appraisal Instrument for Clinical Guidelines⁵ (available at: www.sghms.ac.uk/phs/hceu/form.htm) to help determine quality guidelines in each clinical area, supplemented by the tool developed by the committee. The Appraisal Instrument consists of 37 items addressing 3 dimensions Table 3. The classification system the committee is using to choose top-scoring guidelines after appraisal is as follows. An excellent guideline is one in which the majority of the dimensions (rigor of development, context and content, application) are well addressed by the guideline producers with minimal omission. The evidence is linked to the major recommendations, and the development process is robust. These types of guidelines are highly recommended.

A very good guideline is one in which many of the dimensions are addressed, and some of the recommendations are linked to evidence levels. Objectives and rationale for development are often clearly defined but may be lacking in other areas, such as application (eg, outcome measures, targets, risks, and benefits). These are generally well produced and useful for practicing clinicians and are recommended.

In a fair guideline, some of the dimensions are addressed, but there are some major omissions, often in terms of levels of evidence, literature search strategy, clarity, risks, and benefits. Often these documents are local adaptations of other guidelines. Information can sometimes be used as a general reference if user-friendly materials are incorporated but are generally not very useful as guidelines. These guidelines are recommended under special circumstances.

A poor guideline is one which most of the dimensions are not well addressed, if at all. Often, it is unclear who produced these documents, and there is no description of the individuals involved. Levels of evidence and literature search strategy are rarely included, and there is no description of the methods used to formulate the recommendations. These guidelines are of little use to practicing clinicians and are not recommended.

Recognizing that recommending guidelines based on the quality of the process by which they were produced and the evidence used in their development would be controversial, we felt it was extremely important to develop a rigorous and objective scoring methodology. Fellows from the Department of Family and Community Medicine at the University of Toronto and community-based family physician volunteers from the OMA were brought together in 5 workshops. Each workshop included approximately 20 participants and consisted of a half-day session on the objectives of the GAC, a detailed review of the Appraisal Instrument, and a hands-on session where all participants evaluated the same guideline. Scores were then openly declared, and a discussion held on discrepancies identified in the assessments in an attempt to standardize the process. At the end of the session, interested participants were provided with an additional 5 guidelines to assess in the subsequent 2 weeks. The resulting appraisals were evaluated for consistency and inter-rater reliability (results indicate that using the instrument as an initial filter to determine the best-quality guidelines in each clinical area is a valid approach). To date, 45 assessors have been trained and are reviewing guidelines on an ongoing basis. Each guideline is evaluated a total of 3 times by independent assessors. Those guidelines that have been selected for recommendation in a particular clinical area are then reviewed for clinical relevance and applicability to the Ontario context. More than 250 published guidelines have been identified and distributed to physician assessors in the clinical areas shown in Table 2.

Reformatting

The GAC is in the process of determining the user-friendliness of recommended guidelines. Not infrequently, guidelines that are found to be the most evidence-based and objective are hundreds of pages in length and would be extremely burdensome for the average family physician to use. We anticipate that guidelines found to be of excellent quality but not convenient for use in clinical practice will need to be reformatted into user-friendly summaries. Volunteer physicians from the community will be asked to evaluate such summaries and provide feedback for improvement.

Dissemination

Once the best-quality guideline(s) on a topic are identified and reformatted as necessary; we intend to mount them on the GAC Web site (www.gacguidelines.ca) for use by the profession and the general public. Table 4 shows the results of the guideline selection process for the first 10 clinical areas. The process for choosing guidelines is transparent so that practicing physicians can determine for themselves the usefulness and applicability of the recommendations. Only the most rigorously developed guidelines will be posted on the Web site in the form of structured summaries, although interested clinicians can obtain the outcome of nonrecommended guideline appraisals on request.

Continuing medical education literature on dissemination strategies indicates that a single method, such as posting information on a Web site or mailing guidelines to clinicians has a minimal effect on changing medical practice.⁶ The GAC is currently considering a number of options to enhance the dissemination of the best available guidelines. Since Ontario health data on diagnostic testing, hospitalization records, and office visits are collected provincially, it could be possible to measure clinical outcomes following the dissemination of evidence-based guidelines. We are currently working with provincial groups to disseminate guidelines through medical school continuing medical education (CME) division programs, peer presenter programs, small group CME programs, outreach facilitation programs, and a peer assessment program run by the provincial licensing body.

Conclusions

Over the past 3 years the GAC has developed a method to identify relevant guideline topics and assess the quality of the process by which the guidelines were developed. Clinically excellent guidelines may require some reformatting to make them user-friendly for implementation in clinical practice. The initial product of this process has been posted on the GAC Web site for access by the profession. The GAC is currently assessing and developing a number of strategies to more effectively disseminate guideline information and measure the impact of these interventions on the quality of medical care delivered to the people of Ontario. The GAC will report on the impact of these interventions to facilitate the exchange of successful implementation strategies across jurisdictions.

Acknowledgments

We thank the Physician Services Committee and the members of the Ontario Medical Association and the Ministry of Health and Long-Term Care for their support of this initiative. Conflict of Interest Statement: Dr Rosser and Dr Davis receive stipends for participation on the Guideline Advisory Committee. Ms Gilbart is employed full-time by the Committee through a grant from the Ministry of Health and Long-Term Care. Dr Rosser was a member of the CANMAT Depression Working Group which developed the top-scoring guideline in depression as chosen through the GAC assessment process.

With more than 1000 new guidelines produced annually over the past decade, it is impossible for the practicing family physician to determine which ones should be adapted into their clinical practice. The Ontario Ministry of Health and Long-Term Care and the Ontario Medical Association formed the Guideline Advisory Committee (GAC) in 1997 to assess and disseminate guidelines that would improve the quality and utilization of health care services in the province. Over the past 3 years the GAC has developed a strategy to identify important topics, to rank guidelines published on these topics based on the quality of their development, and to reformat guidelines as necessary to make them user-friendly for implementation in clinical practice. The GAC is currently assessing a number of strategies to enhance the dissemination of selected guidelines to improve the quality of care delivered in the province.

Key points for clinicians

A method of selecting, reviewing, and endorsing clinical practice guidelines has been established in the province of Ontario, Canada. Recommended guideline summaries are posted on a Web site with links to full text for easy access by practicing physicians (www.gacguidelines.ca).

Strategies for the successful implementation and impact evaluation of recommended guidelines are currently in development.

Clinical practice guidelines are statements that are systematically developed to assist physisican and patient decisions about appropriate health care for specific clinical circumstances.¹ Published guidelines have become widely available through Internet technology; it has been estimated that more than 2500 exist. Most are produced by specific interest groups (eg, national societies and pharmaceutical companies), disseminated by publication in a medical journal or traditional mail, and seldom demonstrate any effect on clinical practice.² Such a large volume of guidelines creates confusion for clinicians who often do not follow any of them because of the time required to assess their quality.³

With this dilemma in mind, the GAC was formed with members representing the Ontario Medical Association (OMA), the Ministry of Health and Long-Term Care (MOHLTC) in the province of Ontario, and one ex-officio member of the Institute for Clinical Evaluative Sciences (ICES). The GAC determined its first priority was to identify the best-quality guidelines available for clinicians on selected topics and to then promote their dissemination across the province. The purpose of our paper is to describe the methods that have been developed over the last 3 years to identify high-quality guidelines and some of the strategies being proposed for their dissemination, implementation, and evaluation. We also identify the best-quality guidelines for 10 common conditions.

Methods to assess the development of clinical practice guidelines

Topic Selection

Using a number of parameters, the GAC initially produced a grid as an assessment tool to identify priority areas for guideline review. Table 1 shows the basic grid incorporating provincial utilization and cost data, outcomes research, feedback from clinicians or health care organizations, and a previously published list of common and important problems in family practice.⁴ Feedback from the OMA sections indicated considerable confusion resulting from conflicting advice in specific areas as to appropriate practice (eg, screening for osteoporosis and diabetes). Utilization data from the MOHLTC demonstrated that the use of numerous procedures had rapidly increased over previous years; for example, diagnostic ultrasound utilization increased 65% in 1998. Practicing physicians also identified areas where there was a need for guidelines to be developed because of a lack of evidence or unknown best practice. The committee took all these factors into account when producing a list of priority topics for guideline assessment Table 2.

Guideline Assessment and Recommendation

Once a topic was chosen for assessment, a literature search was conducted by University of Toronto librarians to find all guidelines published in English over the past 10 years on that specific topic. The search strategy included databases such as MEDLINE and HealthStar, and guideline Web sites such as the National Guideline Clearinghouse and the Canadian Medical Association’s Clinical Practice Guideline Infobase. Copies of all guidelines identified in the search were then obtained. A survey of associations and interest groups in Ontario was also made to determine whether there were any unpublished guidelines that we had not identified in this process.

Initially, members of the committee carried out a literature search to determine if there were any publications about scoring the quality of the process used to produce the guidelines. Our search found some processes, but none that directly suited our needs. As a result, the GAC embarked on the development of a guideline-scoring instrument. After a year of work we realized that it would likely take 2 to 3 more years to adequately validate the instrument, and thus a decision was made to adopt the Appraisal Instrument for Clinical Guidelines⁵ (available at: www.sghms.ac.uk/phs/hceu/form.htm) to help determine quality guidelines in each clinical area, supplemented by the tool developed by the committee. The Appraisal Instrument consists of 37 items addressing 3 dimensions Table 3. The classification system the committee is using to choose top-scoring guidelines after appraisal is as follows. An excellent guideline is one in which the majority of the dimensions (rigor of development, context and content, application) are well addressed by the guideline producers with minimal omission. The evidence is linked to the major recommendations, and the development process is robust. These types of guidelines are highly recommended.

A very good guideline is one in which many of the dimensions are addressed, and some of the recommendations are linked to evidence levels. Objectives and rationale for development are often clearly defined but may be lacking in other areas, such as application (eg, outcome measures, targets, risks, and benefits). These are generally well produced and useful for practicing clinicians and are recommended.

In a fair guideline, some of the dimensions are addressed, but there are some major omissions, often in terms of levels of evidence, literature search strategy, clarity, risks, and benefits. Often these documents are local adaptations of other guidelines. Information can sometimes be used as a general reference if user-friendly materials are incorporated but are generally not very useful as guidelines. These guidelines are recommended under special circumstances.

A poor guideline is one which most of the dimensions are not well addressed, if at all. Often, it is unclear who produced these documents, and there is no description of the individuals involved. Levels of evidence and literature search strategy are rarely included, and there is no description of the methods used to formulate the recommendations. These guidelines are of little use to practicing clinicians and are not recommended.

Recognizing that recommending guidelines based on the quality of the process by which they were produced and the evidence used in their development would be controversial, we felt it was extremely important to develop a rigorous and objective scoring methodology. Fellows from the Department of Family and Community Medicine at the University of Toronto and community-based family physician volunteers from the OMA were brought together in 5 workshops. Each workshop included approximately 20 participants and consisted of a half-day session on the objectives of the GAC, a detailed review of the Appraisal Instrument, and a hands-on session where all participants evaluated the same guideline. Scores were then openly declared, and a discussion held on discrepancies identified in the assessments in an attempt to standardize the process. At the end of the session, interested participants were provided with an additional 5 guidelines to assess in the subsequent 2 weeks. The resulting appraisals were evaluated for consistency and inter-rater reliability (results indicate that using the instrument as an initial filter to determine the best-quality guidelines in each clinical area is a valid approach). To date, 45 assessors have been trained and are reviewing guidelines on an ongoing basis. Each guideline is evaluated a total of 3 times by independent assessors. Those guidelines that have been selected for recommendation in a particular clinical area are then reviewed for clinical relevance and applicability to the Ontario context. More than 250 published guidelines have been identified and distributed to physician assessors in the clinical areas shown in Table 2.

Reformatting

The GAC is in the process of determining the user-friendliness of recommended guidelines. Not infrequently, guidelines that are found to be the most evidence-based and objective are hundreds of pages in length and would be extremely burdensome for the average family physician to use. We anticipate that guidelines found to be of excellent quality but not convenient for use in clinical practice will need to be reformatted into user-friendly summaries. Volunteer physicians from the community will be asked to evaluate such summaries and provide feedback for improvement.

Dissemination

Once the best-quality guideline(s) on a topic are identified and reformatted as necessary; we intend to mount them on the GAC Web site (www.gacguidelines.ca) for use by the profession and the general public. Table 4 shows the results of the guideline selection process for the first 10 clinical areas. The process for choosing guidelines is transparent so that practicing physicians can determine for themselves the usefulness and applicability of the recommendations. Only the most rigorously developed guidelines will be posted on the Web site in the form of structured summaries, although interested clinicians can obtain the outcome of nonrecommended guideline appraisals on request.

Continuing medical education literature on dissemination strategies indicates that a single method, such as posting information on a Web site or mailing guidelines to clinicians has a minimal effect on changing medical practice.⁶ The GAC is currently considering a number of options to enhance the dissemination of the best available guidelines. Since Ontario health data on diagnostic testing, hospitalization records, and office visits are collected provincially, it could be possible to measure clinical outcomes following the dissemination of evidence-based guidelines. We are currently working with provincial groups to disseminate guidelines through medical school continuing medical education (CME) division programs, peer presenter programs, small group CME programs, outreach facilitation programs, and a peer assessment program run by the provincial licensing body.

Conclusions

Over the past 3 years the GAC has developed a method to identify relevant guideline topics and assess the quality of the process by which the guidelines were developed. Clinically excellent guidelines may require some reformatting to make them user-friendly for implementation in clinical practice. The initial product of this process has been posted on the GAC Web site for access by the profession. The GAC is currently assessing and developing a number of strategies to more effectively disseminate guideline information and measure the impact of these interventions on the quality of medical care delivered to the people of Ontario. The GAC will report on the impact of these interventions to facilitate the exchange of successful implementation strategies across jurisdictions.

Acknowledgments

We thank the Physician Services Committee and the members of the Ontario Medical Association and the Ministry of Health and Long-Term Care for their support of this initiative. Conflict of Interest Statement: Dr Rosser and Dr Davis receive stipends for participation on the Guideline Advisory Committee. Ms Gilbart is employed full-time by the Committee through a grant from the Ministry of Health and Long-Term Care. Dr Rosser was a member of the CANMAT Depression Working Group which developed the top-scoring guideline in depression as chosen through the GAC assessment process.