Ellen Sarcone, MD

A curbside consultation is an informal process whereby a consultant is asked to provide information or advice about a patient's care without doing a formal assessment of the patient.14 Curbside consultations are common in the practice of medicine2, 3, 5 and are frequently requested by physicians caring for hospitalized patients. Several surveys have documented the quantity of curbside consultations requested of various subspecialties, the types of questions asked, the time it takes to respond, and physicians' perceptions about the quality of the information exchanged.111 While curbside consultations have a number of advantages, physicians' perceptions are that the information conveyed may be inaccurate or incomplete and that the advice offered may be erroneous.13, 5, 10, 12, 13

Cartmill and White14 performed a random audit of 10% of the telephone referrals they received for neurosurgical consultation over a 1‐year period and noted discrepancies between the Glascow Coma Scores reported during the telephone referrals and those noted in the medical records, but the frequency of these discrepancies was not reported. To our knowledge, no studies have compared the quality of the information provided in curbside consultations with that obtained in formal consultations that included direct face‐to‐face patient evaluations and primary data collection, and whether the advice provided in curbside and formal consultations on the same patient differed.

We performed a prospective cohort study to compare the information received by hospitalists during curbside consultations on hospitalized patients, with that obtained from formal consultations done the same day on the same patients, by different hospitalists who were unaware of any details regarding the curbside consultation. We also compared the advice provided by the 2 hospitalists following their curbside and formal consultations. Our hypotheses were that the information received during curbside consultations was frequently inaccurate or incomplete, that the recommendations made after the formal consultation would frequently differ from those made in the curbside consultation, and that these differences would have important implications on patient care.

METHODS

This was a quality improvement study conducted at Denver Health, a 500‐bed university‐affiliated urban safety net hospital from January 10, 2011 to January 9, 2012. The study design was a prospective cohort that included all curbside consultations on hospitalized patients received between 7 _AM and 3 _PM, on intermittently selected weekdays, by the Internal Medicine Consultation Service that was staffed by 18 hospitalists. Data were collected intermittently based upon hospitalist availability and was done to limit potential alterations in the consulting practices of the providers requesting consultations.

Consultations were defined as being curbside when the consulting provider asked for advice, suggestions, or opinions about a patient's care but did not ask the hospitalist to see the patient.15, 15 Consultations pertaining to administrative issues (eg, whether a patient should be admitted to an intensive care bed as opposed to an acute care floor bed) or on patients who were already being followed by a hospitalist were excluded.

The hospitalist receiving the curbside consultation was allowed to ask questions as they normally would, but could not verify the accuracy of the information received (eg, could not review any portion of the patient's medical record, such as notes or lab data). A standardized data collection sheet was used to record the service and level of training of the requesting provider, the medical issue(s) of concern, all clinical data offered by the provider, the number of questions asked by the hospitalist of the provider, and whether, on the basis of the information provided, the hospitalist felt that the question(s) being asked was (were) of sufficient complexity that a formal consultation should occur. The hospitalist then offered advice based upon the information given during the curbside consultation.

After completing the curbside consultation, the hospitalist requested verbal permission from the requesting provider to perform a formal consultation. If the request was approved, the hospitalist performing the curbside consultation contacted a different hospitalist who performed the formal consultation within the next few hours. The only information given to the second hospitalist was the patient's identifiers and the clinical question(s) being asked. The formal consultation included a complete face‐to‐face history and physical examination, a review of the patient's medical record, documentation of the provider's findings, and recommendations for care.

Upon completion of the formal consultation, the hospitalists who performed the curbside and the formal consultations met to review the advice each gave to the requesting provider and the information on which this advice was based. The 2 hospitalists jointly determined the following: (a) whether the information received during the curbside consultation was correct and complete, (b) whether the advice provided in the formal consultation differed from that provided in the curbside consultation, (c) whether the advice provided in the formal consultation dealt with issues other than one(s) leading to the curbside consultation, (d) whether differences in the recommendations given in the curbside versus the formal consultation changed patient management in a meaningful way, and (e) whether the curbside consultation alone was felt to be sufficient.

Information obtained by the hospitalist performing the formal consultation that was different from, or not included in, the information recorded during the curbside consultation was considered to be incorrect or incomplete, respectively. A change in management was defined as an alteration in the direction or type of care that the patient would have received as a result of the advice being given. A pulmonary and critical care physician, with >35 years of experience in inpatient medicine, reviewed the information provided in the curbside and formal consultations, and independently assessed whether the curbside consultation alone would have been sufficient and whether the formal consultation changed management.

Curbside consultations were neither solicited nor discouraged during the course of the study. The provider requesting the curbside consultation was not informed or debriefed about the study in an attempt to avoid affecting future consultation practices from that provider or service.

Associations were sought between the frequency of inaccurate or incomplete data and the requesting service and provider, the consultative category and medical issue, the number of questions asked by the hospitalist during the curbside consultation, and whether the hospitalist doing the curbside consultation thought that formal consultation was needed. A chi‐square test was used to analyze all associations. A P value of <0.05 was considered significant. All analyses were performed using SAS Enterprise Guide 4.3 (SAS Institute, Inc, Cary, NC) software. The study was approved by the Colorado Multiple Institutional Review Board.

RESULTS

Fifty curbside consultations were requested on a total of 215 study days. The requesting service declined formal consultation in 3 instances, leaving 47 curbside consultations that had a formal consultation. Curbside consultations came from a variety of services and providers, and addressed a variety of issues and concerns (Table 1).

The hospitalists asked 0 to 2 questions during 8/47 (17%) of the curbside consultations, 3 to 5 questions during 26/47 (55%) consultations, and more than 5 questions during 13/47 (28%). Based on the information received during the curbside consultations, the hospitalists thought that the curbside consultations were insufficient for 18/47 (38%) of patients. In all instances, the opinions of the 2 hospitalists concurred with respect to this conclusion, and the independent reviewer agreed with this assessment in 17 of these 18 (94%).

The advice rendered in the formal consultations differed from that provided in 26/47 (55%) of the curbside consultations, and the formal consultation was thought to have changed management for 28/47 (60%) of patients (Table 2). The independent reviewer thought that the advice provided in the formal consultations changed management in 29/47 (62%) of the cases, and in 24/28 cases (86%) where the hospitalist felt that the formal consult changed management.

Information was felt to be inaccurate or incomplete in 24/47 (51%) of the curbside consultations (13/47 inaccurate, 16/47 incomplete, 5/47 both inaccurate and incomplete), and when inaccurate or incomplete information was obtained, the advice given in the formal consultations more commonly differed from that provided in the curbside consultation (19/24, 79% vs 7/23, 30%; P < 0.001), and was more commonly felt to change management (22/24, 92% vs 6/23, 26%; P < 0.0001) (Table 2). No association was found between whether the curbside consultation contained complete or accurate information and the consulting service from which the curbside originated, the consulting provider, the consultative aspect(s) or medical issue(s) addressed, the number of questions asked by the hospitalist during the curbside consultation, nor whether the hospitalists felt that a formal consultation was needed.

DISCUSSION

The important findings of this study are that (a) the recommendations made by hospitalists in curbside versus formal consultations on the same patient frequently differ, (b) these differences frequently result in changes in clinical management, (c) the information presented in curbside consultations by providers is frequently inaccurate or incomplete, regardless of the providers specialty or seniority, (d) when inaccurate or incomplete information is received, the recommendations made in curbside and formal consultations differ more frequently, and (e) we found no way to predict whether the information provided in a curbside consultation was likely to be inaccurate or incomplete.

Our hospitalists thought that 38% of the curbside consultations they received should have had formal consultations. Manian and McKinsey7 reported that as many as 53% of questions asked of infectious disease consultants were thought to be too complex to be addressed in an informal consultation. Others, however, report that only 11%33% of curbside consultations were thought to require formal consultation.1, 9, 10, 16 Our hospitalists asked 3 or more questions of the consulting providers in more than 80% of the curbside consultations, suggesting that the curbside consultations we received might have had a higher complexity than those seen by others.

Our finding that information provided in curbside consultation was frequently inaccurate or incomplete is consistent with a number of previous studies reporting physicians' perceptions of the accuracy of curbside consultations.2, 3 Hospital medicine is not likely to be the only discipline affected by inaccurate curbside consultation practices, as surveys of specialists in infectious disease, gynecology, and neurosurgery report that practitioners in these disciplines have similar concerns.1, 10, 14 In a survey returned by 34 physicians, Myers1 found that 50% thought the information exchanged during curbside consultations was inaccurate, leading him to conclude that inaccuracies presented during curbside consultations required further study.

We found no way of predicting whether curbside consultations were likely to include inaccurate or incomplete information. This observation is consistent with the results of Bergus et al16 who found that the frequency of curbside consultations being converted to formal consultations was independent of the training status of the consulting physician, and with the data of Myers1 who found no way of predicting the likelihood that a curbside consultation should be converted to a formal consultation.

We found that formal consultations resulted in management changes more often than differences in recommendations (ie, 60% vs 55%, respectively). This small difference occurred because, on occasion, the formal consultations found issues to address other than the one(s) for which the curbside consultation was requested. In the majority of these instances, the management changes were minor and the curbside consultation was still felt to be sufficient.

In some instances, the advice given after the curbside and the formal consultations differed to only a minor extent (eg, varying recommendations for oral diabetes management). In other instances, however, the advice differed substantially (eg, change in antibiotic management in a septic patient with a multidrug resistant organism, when the original curbside question was for when to order a follow‐up chest roentgenogram for hypoxia; see Supporting Information, Appendix, in the online version of this article). In 26 patients (55%), formal consultation resulted in different medications being started or stopped, additional tests being performed, or different decisions being made about admission versus discharge.

Our study has a number of strengths. First, while a number of reports document that physicians' perceptions are that curbside consultations frequently contain errors,2, 3, 5, 12 to our knowledge this is the first study that prospectively compared the information collected and advice given in curbside versus formal consultation. Second, while this study was conducted as a quality improvement project, thereby requiring us to conclude that the results are not generalizable, the data presented were collected by 18 different hospitalists, reducing the potential of bias from an individual provider's knowledge base or practice. Third, there was excellent agreement between the independent reviewer and the 2 hospitalists who performed the curbside and formal consultations regarding whether a curbside consultation would have been sufficient, and whether the formal consultation changed patient management. Fourth, the study was conducted over a 1‐year period, which should have reduced potential bias arising from the increasing experience of residents requesting consultations as their training progressed.

Our study has several limitations. First, the number of curbside consultations we received during the study period (50 over 215 days) was lower than anticipated, and lower than the rates of consultation reported by others.1, 7, 9 This likely relates to the fact that, prior to beginning the study, Denver Health hospitalists already provided mandatory consultations for several surgical services (thereby reducing the number of curbside consultations received from these services), because curbside consultations received during evenings, nights, and weekends were not included in the study for reasons of convenience, and because we excluded all administrative curbside consultations. Our hospitalist service also provides consultative services 24 hours a day, thereby reducing the number of consultations received during daytime hours. Second, the frequency with which curbside consultations included inaccurate or incomplete information might be higher than what occurs in other hospitals, as Denver Health is an urban, university‐affiliated public hospital and the patients encountered may be more complex and trainees may be less adept at recognizing the information that would facilitate accurate curbside consultations (although we found no difference in the frequency with which inaccurate or incomplete information was provided as a function of the seniority of the requesting physician). Third, the disparity between curbside and formal consultations that we observed could have been biased by the Hawthorne effect. We attempted to address this by not providing the hospitalists who did the formal consultation with any information collected by the hospitalist involved with the curbside consultation, and by comparing the conclusions reached by the hospitalists performing the curbside and formal consultations with those of a third party reviewer. Fourth, while we found no association between the frequency of curbside consultations in which information was inaccurate or incomplete and the consulting service, there could be a selection bias of the consulting service requesting the curbside consultations as a result of the mandatory consultations already provided by our hospitalists. Finally, our study was not designed or adequately powered to determine why curbside consultations frequently have inaccurate or incomplete information.

In summary, we found that the information provided to hospitalists during a curbside consultation was often inaccurate and incomplete, and that these problems with information exchange adversely affected the accuracy of the resulting recommendations. While there are a number of advantages to curbside consultations,1, 3, 7, 10, 12, 13 our findings indicate that the risk associated with this practice is substantial.

A curbside consultation is an informal process whereby a consultant is asked to provide information or advice about a patient's care without doing a formal assessment of the patient.14 Curbside consultations are common in the practice of medicine2, 3, 5 and are frequently requested by physicians caring for hospitalized patients. Several surveys have documented the quantity of curbside consultations requested of various subspecialties, the types of questions asked, the time it takes to respond, and physicians' perceptions about the quality of the information exchanged.111 While curbside consultations have a number of advantages, physicians' perceptions are that the information conveyed may be inaccurate or incomplete and that the advice offered may be erroneous.13, 5, 10, 12, 13

Cartmill and White14 performed a random audit of 10% of the telephone referrals they received for neurosurgical consultation over a 1‐year period and noted discrepancies between the Glascow Coma Scores reported during the telephone referrals and those noted in the medical records, but the frequency of these discrepancies was not reported. To our knowledge, no studies have compared the quality of the information provided in curbside consultations with that obtained in formal consultations that included direct face‐to‐face patient evaluations and primary data collection, and whether the advice provided in curbside and formal consultations on the same patient differed.

We performed a prospective cohort study to compare the information received by hospitalists during curbside consultations on hospitalized patients, with that obtained from formal consultations done the same day on the same patients, by different hospitalists who were unaware of any details regarding the curbside consultation. We also compared the advice provided by the 2 hospitalists following their curbside and formal consultations. Our hypotheses were that the information received during curbside consultations was frequently inaccurate or incomplete, that the recommendations made after the formal consultation would frequently differ from those made in the curbside consultation, and that these differences would have important implications on patient care.

METHODS

This was a quality improvement study conducted at Denver Health, a 500‐bed university‐affiliated urban safety net hospital from January 10, 2011 to January 9, 2012. The study design was a prospective cohort that included all curbside consultations on hospitalized patients received between 7 _AM and 3 _PM, on intermittently selected weekdays, by the Internal Medicine Consultation Service that was staffed by 18 hospitalists. Data were collected intermittently based upon hospitalist availability and was done to limit potential alterations in the consulting practices of the providers requesting consultations.

Consultations were defined as being curbside when the consulting provider asked for advice, suggestions, or opinions about a patient's care but did not ask the hospitalist to see the patient.15, 15 Consultations pertaining to administrative issues (eg, whether a patient should be admitted to an intensive care bed as opposed to an acute care floor bed) or on patients who were already being followed by a hospitalist were excluded.

The hospitalist receiving the curbside consultation was allowed to ask questions as they normally would, but could not verify the accuracy of the information received (eg, could not review any portion of the patient's medical record, such as notes or lab data). A standardized data collection sheet was used to record the service and level of training of the requesting provider, the medical issue(s) of concern, all clinical data offered by the provider, the number of questions asked by the hospitalist of the provider, and whether, on the basis of the information provided, the hospitalist felt that the question(s) being asked was (were) of sufficient complexity that a formal consultation should occur. The hospitalist then offered advice based upon the information given during the curbside consultation.

After completing the curbside consultation, the hospitalist requested verbal permission from the requesting provider to perform a formal consultation. If the request was approved, the hospitalist performing the curbside consultation contacted a different hospitalist who performed the formal consultation within the next few hours. The only information given to the second hospitalist was the patient's identifiers and the clinical question(s) being asked. The formal consultation included a complete face‐to‐face history and physical examination, a review of the patient's medical record, documentation of the provider's findings, and recommendations for care.

Upon completion of the formal consultation, the hospitalists who performed the curbside and the formal consultations met to review the advice each gave to the requesting provider and the information on which this advice was based. The 2 hospitalists jointly determined the following: (a) whether the information received during the curbside consultation was correct and complete, (b) whether the advice provided in the formal consultation differed from that provided in the curbside consultation, (c) whether the advice provided in the formal consultation dealt with issues other than one(s) leading to the curbside consultation, (d) whether differences in the recommendations given in the curbside versus the formal consultation changed patient management in a meaningful way, and (e) whether the curbside consultation alone was felt to be sufficient.

Information obtained by the hospitalist performing the formal consultation that was different from, or not included in, the information recorded during the curbside consultation was considered to be incorrect or incomplete, respectively. A change in management was defined as an alteration in the direction or type of care that the patient would have received as a result of the advice being given. A pulmonary and critical care physician, with >35 years of experience in inpatient medicine, reviewed the information provided in the curbside and formal consultations, and independently assessed whether the curbside consultation alone would have been sufficient and whether the formal consultation changed management.

Curbside consultations were neither solicited nor discouraged during the course of the study. The provider requesting the curbside consultation was not informed or debriefed about the study in an attempt to avoid affecting future consultation practices from that provider or service.

Associations were sought between the frequency of inaccurate or incomplete data and the requesting service and provider, the consultative category and medical issue, the number of questions asked by the hospitalist during the curbside consultation, and whether the hospitalist doing the curbside consultation thought that formal consultation was needed. A chi‐square test was used to analyze all associations. A P value of <0.05 was considered significant. All analyses were performed using SAS Enterprise Guide 4.3 (SAS Institute, Inc, Cary, NC) software. The study was approved by the Colorado Multiple Institutional Review Board.

RESULTS

Fifty curbside consultations were requested on a total of 215 study days. The requesting service declined formal consultation in 3 instances, leaving 47 curbside consultations that had a formal consultation. Curbside consultations came from a variety of services and providers, and addressed a variety of issues and concerns (Table 1).

The hospitalists asked 0 to 2 questions during 8/47 (17%) of the curbside consultations, 3 to 5 questions during 26/47 (55%) consultations, and more than 5 questions during 13/47 (28%). Based on the information received during the curbside consultations, the hospitalists thought that the curbside consultations were insufficient for 18/47 (38%) of patients. In all instances, the opinions of the 2 hospitalists concurred with respect to this conclusion, and the independent reviewer agreed with this assessment in 17 of these 18 (94%).

The advice rendered in the formal consultations differed from that provided in 26/47 (55%) of the curbside consultations, and the formal consultation was thought to have changed management for 28/47 (60%) of patients (Table 2). The independent reviewer thought that the advice provided in the formal consultations changed management in 29/47 (62%) of the cases, and in 24/28 cases (86%) where the hospitalist felt that the formal consult changed management.

Information was felt to be inaccurate or incomplete in 24/47 (51%) of the curbside consultations (13/47 inaccurate, 16/47 incomplete, 5/47 both inaccurate and incomplete), and when inaccurate or incomplete information was obtained, the advice given in the formal consultations more commonly differed from that provided in the curbside consultation (19/24, 79% vs 7/23, 30%; P < 0.001), and was more commonly felt to change management (22/24, 92% vs 6/23, 26%; P < 0.0001) (Table 2). No association was found between whether the curbside consultation contained complete or accurate information and the consulting service from which the curbside originated, the consulting provider, the consultative aspect(s) or medical issue(s) addressed, the number of questions asked by the hospitalist during the curbside consultation, nor whether the hospitalists felt that a formal consultation was needed.

DISCUSSION

The important findings of this study are that (a) the recommendations made by hospitalists in curbside versus formal consultations on the same patient frequently differ, (b) these differences frequently result in changes in clinical management, (c) the information presented in curbside consultations by providers is frequently inaccurate or incomplete, regardless of the providers specialty or seniority, (d) when inaccurate or incomplete information is received, the recommendations made in curbside and formal consultations differ more frequently, and (e) we found no way to predict whether the information provided in a curbside consultation was likely to be inaccurate or incomplete.

Our hospitalists thought that 38% of the curbside consultations they received should have had formal consultations. Manian and McKinsey7 reported that as many as 53% of questions asked of infectious disease consultants were thought to be too complex to be addressed in an informal consultation. Others, however, report that only 11%33% of curbside consultations were thought to require formal consultation.1, 9, 10, 16 Our hospitalists asked 3 or more questions of the consulting providers in more than 80% of the curbside consultations, suggesting that the curbside consultations we received might have had a higher complexity than those seen by others.

Our finding that information provided in curbside consultation was frequently inaccurate or incomplete is consistent with a number of previous studies reporting physicians' perceptions of the accuracy of curbside consultations.2, 3 Hospital medicine is not likely to be the only discipline affected by inaccurate curbside consultation practices, as surveys of specialists in infectious disease, gynecology, and neurosurgery report that practitioners in these disciplines have similar concerns.1, 10, 14 In a survey returned by 34 physicians, Myers1 found that 50% thought the information exchanged during curbside consultations was inaccurate, leading him to conclude that inaccuracies presented during curbside consultations required further study.

We found no way of predicting whether curbside consultations were likely to include inaccurate or incomplete information. This observation is consistent with the results of Bergus et al16 who found that the frequency of curbside consultations being converted to formal consultations was independent of the training status of the consulting physician, and with the data of Myers1 who found no way of predicting the likelihood that a curbside consultation should be converted to a formal consultation.

We found that formal consultations resulted in management changes more often than differences in recommendations (ie, 60% vs 55%, respectively). This small difference occurred because, on occasion, the formal consultations found issues to address other than the one(s) for which the curbside consultation was requested. In the majority of these instances, the management changes were minor and the curbside consultation was still felt to be sufficient.

In some instances, the advice given after the curbside and the formal consultations differed to only a minor extent (eg, varying recommendations for oral diabetes management). In other instances, however, the advice differed substantially (eg, change in antibiotic management in a septic patient with a multidrug resistant organism, when the original curbside question was for when to order a follow‐up chest roentgenogram for hypoxia; see Supporting Information, Appendix, in the online version of this article). In 26 patients (55%), formal consultation resulted in different medications being started or stopped, additional tests being performed, or different decisions being made about admission versus discharge.

Our study has a number of strengths. First, while a number of reports document that physicians' perceptions are that curbside consultations frequently contain errors,2, 3, 5, 12 to our knowledge this is the first study that prospectively compared the information collected and advice given in curbside versus formal consultation. Second, while this study was conducted as a quality improvement project, thereby requiring us to conclude that the results are not generalizable, the data presented were collected by 18 different hospitalists, reducing the potential of bias from an individual provider's knowledge base or practice. Third, there was excellent agreement between the independent reviewer and the 2 hospitalists who performed the curbside and formal consultations regarding whether a curbside consultation would have been sufficient, and whether the formal consultation changed patient management. Fourth, the study was conducted over a 1‐year period, which should have reduced potential bias arising from the increasing experience of residents requesting consultations as their training progressed.

Our study has several limitations. First, the number of curbside consultations we received during the study period (50 over 215 days) was lower than anticipated, and lower than the rates of consultation reported by others.1, 7, 9 This likely relates to the fact that, prior to beginning the study, Denver Health hospitalists already provided mandatory consultations for several surgical services (thereby reducing the number of curbside consultations received from these services), because curbside consultations received during evenings, nights, and weekends were not included in the study for reasons of convenience, and because we excluded all administrative curbside consultations. Our hospitalist service also provides consultative services 24 hours a day, thereby reducing the number of consultations received during daytime hours. Second, the frequency with which curbside consultations included inaccurate or incomplete information might be higher than what occurs in other hospitals, as Denver Health is an urban, university‐affiliated public hospital and the patients encountered may be more complex and trainees may be less adept at recognizing the information that would facilitate accurate curbside consultations (although we found no difference in the frequency with which inaccurate or incomplete information was provided as a function of the seniority of the requesting physician). Third, the disparity between curbside and formal consultations that we observed could have been biased by the Hawthorne effect. We attempted to address this by not providing the hospitalists who did the formal consultation with any information collected by the hospitalist involved with the curbside consultation, and by comparing the conclusions reached by the hospitalists performing the curbside and formal consultations with those of a third party reviewer. Fourth, while we found no association between the frequency of curbside consultations in which information was inaccurate or incomplete and the consulting service, there could be a selection bias of the consulting service requesting the curbside consultations as a result of the mandatory consultations already provided by our hospitalists. Finally, our study was not designed or adequately powered to determine why curbside consultations frequently have inaccurate or incomplete information.

In summary, we found that the information provided to hospitalists during a curbside consultation was often inaccurate and incomplete, and that these problems with information exchange adversely affected the accuracy of the resulting recommendations. While there are a number of advantages to curbside consultations,1, 3, 7, 10, 12, 13 our findings indicate that the risk associated with this practice is substantial.

A curbside consultation is an informal process whereby a consultant is asked to provide information or advice about a patient's care without doing a formal assessment of the patient.14 Curbside consultations are common in the practice of medicine2, 3, 5 and are frequently requested by physicians caring for hospitalized patients. Several surveys have documented the quantity of curbside consultations requested of various subspecialties, the types of questions asked, the time it takes to respond, and physicians' perceptions about the quality of the information exchanged.111 While curbside consultations have a number of advantages, physicians' perceptions are that the information conveyed may be inaccurate or incomplete and that the advice offered may be erroneous.13, 5, 10, 12, 13

Cartmill and White14 performed a random audit of 10% of the telephone referrals they received for neurosurgical consultation over a 1‐year period and noted discrepancies between the Glascow Coma Scores reported during the telephone referrals and those noted in the medical records, but the frequency of these discrepancies was not reported. To our knowledge, no studies have compared the quality of the information provided in curbside consultations with that obtained in formal consultations that included direct face‐to‐face patient evaluations and primary data collection, and whether the advice provided in curbside and formal consultations on the same patient differed.

We performed a prospective cohort study to compare the information received by hospitalists during curbside consultations on hospitalized patients, with that obtained from formal consultations done the same day on the same patients, by different hospitalists who were unaware of any details regarding the curbside consultation. We also compared the advice provided by the 2 hospitalists following their curbside and formal consultations. Our hypotheses were that the information received during curbside consultations was frequently inaccurate or incomplete, that the recommendations made after the formal consultation would frequently differ from those made in the curbside consultation, and that these differences would have important implications on patient care.

METHODS

This was a quality improvement study conducted at Denver Health, a 500‐bed university‐affiliated urban safety net hospital from January 10, 2011 to January 9, 2012. The study design was a prospective cohort that included all curbside consultations on hospitalized patients received between 7 _AM and 3 _PM, on intermittently selected weekdays, by the Internal Medicine Consultation Service that was staffed by 18 hospitalists. Data were collected intermittently based upon hospitalist availability and was done to limit potential alterations in the consulting practices of the providers requesting consultations.

Consultations were defined as being curbside when the consulting provider asked for advice, suggestions, or opinions about a patient's care but did not ask the hospitalist to see the patient.15, 15 Consultations pertaining to administrative issues (eg, whether a patient should be admitted to an intensive care bed as opposed to an acute care floor bed) or on patients who were already being followed by a hospitalist were excluded.

The hospitalist receiving the curbside consultation was allowed to ask questions as they normally would, but could not verify the accuracy of the information received (eg, could not review any portion of the patient's medical record, such as notes or lab data). A standardized data collection sheet was used to record the service and level of training of the requesting provider, the medical issue(s) of concern, all clinical data offered by the provider, the number of questions asked by the hospitalist of the provider, and whether, on the basis of the information provided, the hospitalist felt that the question(s) being asked was (were) of sufficient complexity that a formal consultation should occur. The hospitalist then offered advice based upon the information given during the curbside consultation.

After completing the curbside consultation, the hospitalist requested verbal permission from the requesting provider to perform a formal consultation. If the request was approved, the hospitalist performing the curbside consultation contacted a different hospitalist who performed the formal consultation within the next few hours. The only information given to the second hospitalist was the patient's identifiers and the clinical question(s) being asked. The formal consultation included a complete face‐to‐face history and physical examination, a review of the patient's medical record, documentation of the provider's findings, and recommendations for care.

Upon completion of the formal consultation, the hospitalists who performed the curbside and the formal consultations met to review the advice each gave to the requesting provider and the information on which this advice was based. The 2 hospitalists jointly determined the following: (a) whether the information received during the curbside consultation was correct and complete, (b) whether the advice provided in the formal consultation differed from that provided in the curbside consultation, (c) whether the advice provided in the formal consultation dealt with issues other than one(s) leading to the curbside consultation, (d) whether differences in the recommendations given in the curbside versus the formal consultation changed patient management in a meaningful way, and (e) whether the curbside consultation alone was felt to be sufficient.

Information obtained by the hospitalist performing the formal consultation that was different from, or not included in, the information recorded during the curbside consultation was considered to be incorrect or incomplete, respectively. A change in management was defined as an alteration in the direction or type of care that the patient would have received as a result of the advice being given. A pulmonary and critical care physician, with >35 years of experience in inpatient medicine, reviewed the information provided in the curbside and formal consultations, and independently assessed whether the curbside consultation alone would have been sufficient and whether the formal consultation changed management.

Curbside consultations were neither solicited nor discouraged during the course of the study. The provider requesting the curbside consultation was not informed or debriefed about the study in an attempt to avoid affecting future consultation practices from that provider or service.

Associations were sought between the frequency of inaccurate or incomplete data and the requesting service and provider, the consultative category and medical issue, the number of questions asked by the hospitalist during the curbside consultation, and whether the hospitalist doing the curbside consultation thought that formal consultation was needed. A chi‐square test was used to analyze all associations. A P value of <0.05 was considered significant. All analyses were performed using SAS Enterprise Guide 4.3 (SAS Institute, Inc, Cary, NC) software. The study was approved by the Colorado Multiple Institutional Review Board.

RESULTS

Fifty curbside consultations were requested on a total of 215 study days. The requesting service declined formal consultation in 3 instances, leaving 47 curbside consultations that had a formal consultation. Curbside consultations came from a variety of services and providers, and addressed a variety of issues and concerns (Table 1).

The hospitalists asked 0 to 2 questions during 8/47 (17%) of the curbside consultations, 3 to 5 questions during 26/47 (55%) consultations, and more than 5 questions during 13/47 (28%). Based on the information received during the curbside consultations, the hospitalists thought that the curbside consultations were insufficient for 18/47 (38%) of patients. In all instances, the opinions of the 2 hospitalists concurred with respect to this conclusion, and the independent reviewer agreed with this assessment in 17 of these 18 (94%).

The advice rendered in the formal consultations differed from that provided in 26/47 (55%) of the curbside consultations, and the formal consultation was thought to have changed management for 28/47 (60%) of patients (Table 2). The independent reviewer thought that the advice provided in the formal consultations changed management in 29/47 (62%) of the cases, and in 24/28 cases (86%) where the hospitalist felt that the formal consult changed management.

Information was felt to be inaccurate or incomplete in 24/47 (51%) of the curbside consultations (13/47 inaccurate, 16/47 incomplete, 5/47 both inaccurate and incomplete), and when inaccurate or incomplete information was obtained, the advice given in the formal consultations more commonly differed from that provided in the curbside consultation (19/24, 79% vs 7/23, 30%; P < 0.001), and was more commonly felt to change management (22/24, 92% vs 6/23, 26%; P < 0.0001) (Table 2). No association was found between whether the curbside consultation contained complete or accurate information and the consulting service from which the curbside originated, the consulting provider, the consultative aspect(s) or medical issue(s) addressed, the number of questions asked by the hospitalist during the curbside consultation, nor whether the hospitalists felt that a formal consultation was needed.

DISCUSSION

The important findings of this study are that (a) the recommendations made by hospitalists in curbside versus formal consultations on the same patient frequently differ, (b) these differences frequently result in changes in clinical management, (c) the information presented in curbside consultations by providers is frequently inaccurate or incomplete, regardless of the providers specialty or seniority, (d) when inaccurate or incomplete information is received, the recommendations made in curbside and formal consultations differ more frequently, and (e) we found no way to predict whether the information provided in a curbside consultation was likely to be inaccurate or incomplete.

Our hospitalists thought that 38% of the curbside consultations they received should have had formal consultations. Manian and McKinsey7 reported that as many as 53% of questions asked of infectious disease consultants were thought to be too complex to be addressed in an informal consultation. Others, however, report that only 11%33% of curbside consultations were thought to require formal consultation.1, 9, 10, 16 Our hospitalists asked 3 or more questions of the consulting providers in more than 80% of the curbside consultations, suggesting that the curbside consultations we received might have had a higher complexity than those seen by others.

Our finding that information provided in curbside consultation was frequently inaccurate or incomplete is consistent with a number of previous studies reporting physicians' perceptions of the accuracy of curbside consultations.2, 3 Hospital medicine is not likely to be the only discipline affected by inaccurate curbside consultation practices, as surveys of specialists in infectious disease, gynecology, and neurosurgery report that practitioners in these disciplines have similar concerns.1, 10, 14 In a survey returned by 34 physicians, Myers1 found that 50% thought the information exchanged during curbside consultations was inaccurate, leading him to conclude that inaccuracies presented during curbside consultations required further study.

We found no way of predicting whether curbside consultations were likely to include inaccurate or incomplete information. This observation is consistent with the results of Bergus et al16 who found that the frequency of curbside consultations being converted to formal consultations was independent of the training status of the consulting physician, and with the data of Myers1 who found no way of predicting the likelihood that a curbside consultation should be converted to a formal consultation.

We found that formal consultations resulted in management changes more often than differences in recommendations (ie, 60% vs 55%, respectively). This small difference occurred because, on occasion, the formal consultations found issues to address other than the one(s) for which the curbside consultation was requested. In the majority of these instances, the management changes were minor and the curbside consultation was still felt to be sufficient.

In some instances, the advice given after the curbside and the formal consultations differed to only a minor extent (eg, varying recommendations for oral diabetes management). In other instances, however, the advice differed substantially (eg, change in antibiotic management in a septic patient with a multidrug resistant organism, when the original curbside question was for when to order a follow‐up chest roentgenogram for hypoxia; see Supporting Information, Appendix, in the online version of this article). In 26 patients (55%), formal consultation resulted in different medications being started or stopped, additional tests being performed, or different decisions being made about admission versus discharge.

Our study has a number of strengths. First, while a number of reports document that physicians' perceptions are that curbside consultations frequently contain errors,2, 3, 5, 12 to our knowledge this is the first study that prospectively compared the information collected and advice given in curbside versus formal consultation. Second, while this study was conducted as a quality improvement project, thereby requiring us to conclude that the results are not generalizable, the data presented were collected by 18 different hospitalists, reducing the potential of bias from an individual provider's knowledge base or practice. Third, there was excellent agreement between the independent reviewer and the 2 hospitalists who performed the curbside and formal consultations regarding whether a curbside consultation would have been sufficient, and whether the formal consultation changed patient management. Fourth, the study was conducted over a 1‐year period, which should have reduced potential bias arising from the increasing experience of residents requesting consultations as their training progressed.

Our study has several limitations. First, the number of curbside consultations we received during the study period (50 over 215 days) was lower than anticipated, and lower than the rates of consultation reported by others.1, 7, 9 This likely relates to the fact that, prior to beginning the study, Denver Health hospitalists already provided mandatory consultations for several surgical services (thereby reducing the number of curbside consultations received from these services), because curbside consultations received during evenings, nights, and weekends were not included in the study for reasons of convenience, and because we excluded all administrative curbside consultations. Our hospitalist service also provides consultative services 24 hours a day, thereby reducing the number of consultations received during daytime hours. Second, the frequency with which curbside consultations included inaccurate or incomplete information might be higher than what occurs in other hospitals, as Denver Health is an urban, university‐affiliated public hospital and the patients encountered may be more complex and trainees may be less adept at recognizing the information that would facilitate accurate curbside consultations (although we found no difference in the frequency with which inaccurate or incomplete information was provided as a function of the seniority of the requesting physician). Third, the disparity between curbside and formal consultations that we observed could have been biased by the Hawthorne effect. We attempted to address this by not providing the hospitalists who did the formal consultation with any information collected by the hospitalist involved with the curbside consultation, and by comparing the conclusions reached by the hospitalists performing the curbside and formal consultations with those of a third party reviewer. Fourth, while we found no association between the frequency of curbside consultations in which information was inaccurate or incomplete and the consulting service, there could be a selection bias of the consulting service requesting the curbside consultations as a result of the mandatory consultations already provided by our hospitalists. Finally, our study was not designed or adequately powered to determine why curbside consultations frequently have inaccurate or incomplete information.

In summary, we found that the information provided to hospitalists during a curbside consultation was often inaccurate and incomplete, and that these problems with information exchange adversely affected the accuracy of the resulting recommendations. While there are a number of advantages to curbside consultations,1, 3, 7, 10, 12, 13 our findings indicate that the risk associated with this practice is substantial.

Communication failures are well‐recognized as causes of medical errors.1, 2 Specifically, handoffs of patient care responsibilities, which are increasingly prevalent in academic medical centers,3 have been cited as the most frequent cause of teamwork breakdown resulting in the harmful medical errors found in malpractice claims.1 The Institute of Medicine has recently identified patient handoffs as the moment where patient care errors are most likely to occur.4 A survey of 125 U.S. medical schools, however, found that only 8% specifically taught students how to hand off patient care.3

In July 2003, the American Council of Graduate Medical Education (ACGME) mandated that residency programs decrease resident work hours to improve patient care and safety by reducing fatigue,5 and a recent Institute of Medicine report suggests that they be decreased even further.4 Studies examining outcomes during the first 2 years after reducing duty hours did not find reductions in risk‐adjusted mortality.68 One proposed explanation for this lack of improvement is that the reduction in fatigue‐related medical errors is being offset by discontinuity of care with due to the increased number of patient handoffs resulting from shortened duty hours,911 one recent study found that omission of key information during patient sign outs frequently resulted in adverse patient care outcomes.12

In 2007, the Joint Commission developed a new National Patient Safety Goal that requires organizations to improve communication between caregivers.13 We recently developed an approach by which Internal Medicine residents hand off patient care using a structured process, written and verbal templates, formal training about handoffs, and direct attending supervision.14 Because fourth‐year medical students perform the duties of interns when working as subinterns, we recognized that education about handoffs should occur prior to the time students became interns. Accordingly, we developed a course designed to teach patient handoffs to medical students at the transition between their third and fourth years of training.

Setting

The Handoff Selective was developed by faculty of Denver Health and the University of Colorado Denver School of Medicine.

Program Description

The Selective was first offered in April 2007 as part of an Integrated Clinician's Course (ICC), a 2‐week course for students beginning their fourth year, which starts in April at the University of Colorado. The ICC includes both mandatory and selective sessions that are focused on developing clinical skills and preparing them for their subinternships. The Handoff Selective was conducted in a computerized teaching laboratory, lasted a total of 2 hours and consisted of 2 parts. Each of the 5 Denver Health Hospital Medicine faculty members versed in handoff education taught 2 sessions of 6 to 8 students.

Part 1: Didactic

During the first hour of class, the faculty presented a lecture that summarized the relevant literature on handoffs and explained the importance of the topic. The objectives of the didactic were to: (1) understand the importance of handoffs; (2) explore different communication elements and structures; (3) gain exposure to handoffs outside of healthcare; and (4) learn a structure for handoffs of patient care in hospitalized patients.

We used 3 video clips of handoffs from 2 football games to demonstrate the importance of practice, training, and 2‐way communications in handoffs. The first video clip showed a runner trying to make a spontaneous handoff while being tackled. The receiver was not expecting the handoff and was preoccupied with blocking another player. This attempted handoff resulted in a fumble, which we related to an adverse patient event.

The next 2 video clips showed 2 complex, seldom used, but well‐known football handoffsthe hook and lateral and the Statue of Liberty. Both handoffs were successfully executed presumably as a result of education, practice and the active participation of both players (handing off and receiving) in the process. We then related the teaching and practicing of complex communication to the Joint Commission on Accreditation of Healthcare Organizations (JCAHO; now simply the Joint Commission) data suggesting that most sentinel events have their root cause in communication and training failures.2

Basic communication elements and process structures were then explored using scenarios from everyday life and evidence from fields outside of medicine. We emphasized that structures for communication (modes, vehicles, and settings) must be chosen according to the occasion and that handoffs are common and important in all occupations. In discussing modes (verbal, written, or nonverbal), vehicles (paper, telephone, or e‐mail), and settings (face‐to face, virtual, or disconnected), we emphasized that the most effective structures for communication (verbal, face‐to face meetings, with written materials and other visual aids at the patient's bedside) were also the most time‐consuming (Figure 1). While our standard for resident handoffs is a face‐to‐face verbal interaction with preprinted written materials as an aid, we also emphasized that for complex patients (eg, mental status changes, concern for an acute abdomen) more robust communication is often needed. Accordingly, a more time‐consuming bedside handoff with simultaneous, focused physical exam and history‐taking by both oncoming and off‐going providers may be most appropriate.

Figure 1

As real‐life examples, we asked our students to communicate a happy birthday wish to their mother, who lives in another state. Almost uniformly, in addition to a written aid (birthday card), they choose the telephone as a vehicle for their verbal mode in a virtual setting with 2‐way communication possible. In contrast, when asked to propose marriage to a significant other in another state, students felt that a face‐to‐face meeting with verbal and nonverbal (ie, ring) modes was appropriate. This time‐consuming mode of communication was felt to be necessary to create a sentiment of importance and avert any possible miscommunication.

The didactic session concluded by demonstrating how to use standardized written and verbal templates for handoffs of the care of a hospitalized patient. We explore the differentiation between written and verbal handoffs in our discussion below.

Program Evaluation

We developed a 2‐part survey to evaluate the effectiveness of the Selective and to solicit feedback about the didactic and practicum portions of the course. The first part of the survey (Table 2) contained 16 items to assess the students' knowledge of, and attitudes toward handing off patient care, along with their comfort with the handoff process. Responses to this section were scored using a 5‐point Likert scale with 1 indicating strongly disagree and 5 indicating strongly agree. This part of the survey was administered both prior to and after the Selective.

The second part (Table 3) contained 12 items and was designed to evaluate the perceived usefulness of the different components of the class. This section was only administered at the end of the Selective. It utilized a 4‐point Likert scale with 1 indicating that the component was not useful at all, and 4 indicating that it was extremely useful. The first 6 items of the second section allowed students to evaluate the didactic portion of the handoff. The second 6 items allowed students to evaluate the practicum. Responses to all 12 items were then combined to determine an overall composite usefulness for the Selective.

The Selective was also evaluated qualitatively through the use of open‐ended, written comments that were solicited at the end of the survey. All surveys were administered anonymously.

Results

More students chose the Selective than we had capacity to accommodate (60 of a class of 150). The pre‐ and postcourse survey response rate was 56 of 60 (93%) and 58 of 60 (97%), respectively. After the Selective, the mean score in response to whether handoffs are well taught in medical school increased from 1.6 to 3.5 (P < 0.003). Our students' self‐perceived skills and knowledge about handoffs improved after the Selective (Table 2). The greatest changes in perceived knowledge occurred in questions regarding the what, how, and when of read‐backs, and the knowledge of standard verbal and written handoff structures. The responses to the survey elements which assessed our students' attitudes regarding the importance of standardization and whether they felt handoffs were safer with faculty supervision did not change after the Selective (Table 2).

A total of 92% of the students felt that the course was extremely useful or useful. The role‐playing activity was thought to be more helpful than the didactic, but 84% of the students still rated the didactic portion as useful or extremely useful (Table 3). The element which was the least well received in the didactic portion was the use of video clips to demonstrate successful and unsuccessful (fumbled) college football handoffs, although the majority (64%) of students still found it useful.

The major theme generated from the comments section of the survey was that the Selective should be a required course.

Discussion

We know of no previously published literature that has addressed teaching handoffs to medical students. Horwitz et al.15 developed a sign‐out curriculum for Internal Medicine residents and found that none of their house‐staff had any previous training in handoffs during medical school, consistent with the finding that only 8% of U.S. medical schools provided formal instruction on handoffs.3 Prior to taking the Selective, our students had no knowledge of verbal or written templates for patient handoffs, although both before and after the course they felt that standardization was an important component of the process.

A number of verbal structures for handing off patient care have been described in the literature and there is not a consensus as to which functions best. Perhaps the most cited verbal communication format is SBAR (ie, situation, background, assessment and recommendation).16, 17 This tool was developed by Leonard et al.18 specifically for use by nurses to provide 1‐way communication to physicians pertaining to a change in patient status. We considered teaching the SBAR approach to the students but felt that it did not provide a suitable structure for handoffs because the transfer of care is not generally an event‐based situation and the literature on handoffs indicates that an optimal verbal system includes 2‐way communication.

Additional mnemonics for handoffs found in the literature include SIGNOUT (ie, Sick or DNR, Identifying information, General hospital course, New events of the day, Overall health status, Upcoming possibilities with plan, and Tasks to complete),14 I PASS the BATON (ie, Introduction, Patient, Assessment, Situation, Safety, Background, Actions, Timing, Ownership, Next)19 and the SAIF‐IR system (see boxed text).14

We developed the SAIF‐IR mnemonic to maximize efficiency and effectiveness while differentiating the verbal portion of the handoff from the written and incorporating 2‐way communication into its structure. In the Summary statement, we emphasize that this is not a history of present illness. We ask our students to summarize, in 1 to 3 sentences, the patient's presentation and working diagnosis. When discussing patient issues, we ask our students to only verbalize Active issues, although the written template has inactive, chronic issues listed. Here, we also ask our students to express their level of concern for the active issues and patient in general. If‐then's and Follow‐ups are usually verbalized together. Based on the offgoing provider's knowledge of the patient, we encourage the offgoing provider to anticipate potential problems and advise the oncoming provider on potential responses. Much of this advice is difficult to express in the written format and thus may not be found on the written handoff when the verbal handoff occurs. We encourage oncoming providers to take notes on the preprinted handoff sheet as part of the handoff process.

Through Interactive questioning and Read‐backs, we train our students and house‐staff to use the active listening techniques used outside of healthcare, in settings such as nuclear power plants and National Aeronautics and Space Administration mission control, where poor handoff communication may also result in safety concerns and adverse events.20 Interactive questioning allows the oncoming provider to correct or clarify any information given by the off‐going provider. Read‐backs are a method of confirming follow‐up activity or contingency plans. Together, the SAIF‐IR mnemonic builds a 2‐way communication structure into the patient handoff with both offgoing and oncoming providers having predefined roles.

Much of the information on our written handoff (patient identifying information, medications, language preference, code status, admission date) is not verbalized unless it is part of the active issues or the if‐then, follow‐ups (ie, medication titration for a patient admitted with an acute coronary syndrome or cor status in a patient newly made comfort care). By not reading extraneous information, we seek to emphasize the Active issues as well as the If‐then, Follow‐ups. We feel this emphasis maximizes the effectiveness of the handoff, while the purposeful nonverbalization of written materials such as identifying information maximizes its efficiency. Future work may examine which verbal and written structures for patient handoffs most benefit patient care and workflow through standard communication.

While our students found the Handoff Selective to be useful and to improve their self‐perceived ability to perform handoffs, we were not able to determine whether our program affected downstream outcomes such as adverse events relating to failures in handoff communication. Additionally, since we only taught and evaluated our Selective at the University of Colorado Denver School of Medicine, the response of our students may not generalize to other medical schools. Multicentered, prospective, randomized controlled trials may determine whether handoff education programs are successful in reducing patient adverse events related to transfers of care.

While handoffs occur frequently and are increasingly recognized as a vulnerable time in patient care, little is known about how to effectively teach handoffs to medical students during their clinical years. We developed a formal course to teach the importance of handoffs and how the process should be conducted. Our students reported that the Handoff Selective we developed improved their knowledge about the process and their perception of their ability to perform handoffs in a time‐appropriate and effective manner. In response to the feedback we received from our students, the Handoff Selective is the only course in the ICC that has been made mandatory for all students.

Communication failures are well‐recognized as causes of medical errors.1, 2 Specifically, handoffs of patient care responsibilities, which are increasingly prevalent in academic medical centers,3 have been cited as the most frequent cause of teamwork breakdown resulting in the harmful medical errors found in malpractice claims.1 The Institute of Medicine has recently identified patient handoffs as the moment where patient care errors are most likely to occur.4 A survey of 125 U.S. medical schools, however, found that only 8% specifically taught students how to hand off patient care.3

In July 2003, the American Council of Graduate Medical Education (ACGME) mandated that residency programs decrease resident work hours to improve patient care and safety by reducing fatigue,5 and a recent Institute of Medicine report suggests that they be decreased even further.4 Studies examining outcomes during the first 2 years after reducing duty hours did not find reductions in risk‐adjusted mortality.68 One proposed explanation for this lack of improvement is that the reduction in fatigue‐related medical errors is being offset by discontinuity of care with due to the increased number of patient handoffs resulting from shortened duty hours,911 one recent study found that omission of key information during patient sign outs frequently resulted in adverse patient care outcomes.12

In 2007, the Joint Commission developed a new National Patient Safety Goal that requires organizations to improve communication between caregivers.13 We recently developed an approach by which Internal Medicine residents hand off patient care using a structured process, written and verbal templates, formal training about handoffs, and direct attending supervision.14 Because fourth‐year medical students perform the duties of interns when working as subinterns, we recognized that education about handoffs should occur prior to the time students became interns. Accordingly, we developed a course designed to teach patient handoffs to medical students at the transition between their third and fourth years of training.

Setting

The Handoff Selective was developed by faculty of Denver Health and the University of Colorado Denver School of Medicine.

Program Description

The Selective was first offered in April 2007 as part of an Integrated Clinician's Course (ICC), a 2‐week course for students beginning their fourth year, which starts in April at the University of Colorado. The ICC includes both mandatory and selective sessions that are focused on developing clinical skills and preparing them for their subinternships. The Handoff Selective was conducted in a computerized teaching laboratory, lasted a total of 2 hours and consisted of 2 parts. Each of the 5 Denver Health Hospital Medicine faculty members versed in handoff education taught 2 sessions of 6 to 8 students.

Part 1: Didactic

During the first hour of class, the faculty presented a lecture that summarized the relevant literature on handoffs and explained the importance of the topic. The objectives of the didactic were to: (1) understand the importance of handoffs; (2) explore different communication elements and structures; (3) gain exposure to handoffs outside of healthcare; and (4) learn a structure for handoffs of patient care in hospitalized patients.

We used 3 video clips of handoffs from 2 football games to demonstrate the importance of practice, training, and 2‐way communications in handoffs. The first video clip showed a runner trying to make a spontaneous handoff while being tackled. The receiver was not expecting the handoff and was preoccupied with blocking another player. This attempted handoff resulted in a fumble, which we related to an adverse patient event.

The next 2 video clips showed 2 complex, seldom used, but well‐known football handoffsthe hook and lateral and the Statue of Liberty. Both handoffs were successfully executed presumably as a result of education, practice and the active participation of both players (handing off and receiving) in the process. We then related the teaching and practicing of complex communication to the Joint Commission on Accreditation of Healthcare Organizations (JCAHO; now simply the Joint Commission) data suggesting that most sentinel events have their root cause in communication and training failures.2

Basic communication elements and process structures were then explored using scenarios from everyday life and evidence from fields outside of medicine. We emphasized that structures for communication (modes, vehicles, and settings) must be chosen according to the occasion and that handoffs are common and important in all occupations. In discussing modes (verbal, written, or nonverbal), vehicles (paper, telephone, or e‐mail), and settings (face‐to face, virtual, or disconnected), we emphasized that the most effective structures for communication (verbal, face‐to face meetings, with written materials and other visual aids at the patient's bedside) were also the most time‐consuming (Figure 1). While our standard for resident handoffs is a face‐to‐face verbal interaction with preprinted written materials as an aid, we also emphasized that for complex patients (eg, mental status changes, concern for an acute abdomen) more robust communication is often needed. Accordingly, a more time‐consuming bedside handoff with simultaneous, focused physical exam and history‐taking by both oncoming and off‐going providers may be most appropriate.

Figure 1

As real‐life examples, we asked our students to communicate a happy birthday wish to their mother, who lives in another state. Almost uniformly, in addition to a written aid (birthday card), they choose the telephone as a vehicle for their verbal mode in a virtual setting with 2‐way communication possible. In contrast, when asked to propose marriage to a significant other in another state, students felt that a face‐to‐face meeting with verbal and nonverbal (ie, ring) modes was appropriate. This time‐consuming mode of communication was felt to be necessary to create a sentiment of importance and avert any possible miscommunication.

The didactic session concluded by demonstrating how to use standardized written and verbal templates for handoffs of the care of a hospitalized patient. We explore the differentiation between written and verbal handoffs in our discussion below.

Program Evaluation

We developed a 2‐part survey to evaluate the effectiveness of the Selective and to solicit feedback about the didactic and practicum portions of the course. The first part of the survey (Table 2) contained 16 items to assess the students' knowledge of, and attitudes toward handing off patient care, along with their comfort with the handoff process. Responses to this section were scored using a 5‐point Likert scale with 1 indicating strongly disagree and 5 indicating strongly agree. This part of the survey was administered both prior to and after the Selective.

The second part (Table 3) contained 12 items and was designed to evaluate the perceived usefulness of the different components of the class. This section was only administered at the end of the Selective. It utilized a 4‐point Likert scale with 1 indicating that the component was not useful at all, and 4 indicating that it was extremely useful. The first 6 items of the second section allowed students to evaluate the didactic portion of the handoff. The second 6 items allowed students to evaluate the practicum. Responses to all 12 items were then combined to determine an overall composite usefulness for the Selective.

The Selective was also evaluated qualitatively through the use of open‐ended, written comments that were solicited at the end of the survey. All surveys were administered anonymously.

Results

More students chose the Selective than we had capacity to accommodate (60 of a class of 150). The pre‐ and postcourse survey response rate was 56 of 60 (93%) and 58 of 60 (97%), respectively. After the Selective, the mean score in response to whether handoffs are well taught in medical school increased from 1.6 to 3.5 (P < 0.003). Our students' self‐perceived skills and knowledge about handoffs improved after the Selective (Table 2). The greatest changes in perceived knowledge occurred in questions regarding the what, how, and when of read‐backs, and the knowledge of standard verbal and written handoff structures. The responses to the survey elements which assessed our students' attitudes regarding the importance of standardization and whether they felt handoffs were safer with faculty supervision did not change after the Selective (Table 2).

A total of 92% of the students felt that the course was extremely useful or useful. The role‐playing activity was thought to be more helpful than the didactic, but 84% of the students still rated the didactic portion as useful or extremely useful (Table 3). The element which was the least well received in the didactic portion was the use of video clips to demonstrate successful and unsuccessful (fumbled) college football handoffs, although the majority (64%) of students still found it useful.

The major theme generated from the comments section of the survey was that the Selective should be a required course.

Discussion

We know of no previously published literature that has addressed teaching handoffs to medical students. Horwitz et al.15 developed a sign‐out curriculum for Internal Medicine residents and found that none of their house‐staff had any previous training in handoffs during medical school, consistent with the finding that only 8% of U.S. medical schools provided formal instruction on handoffs.3 Prior to taking the Selective, our students had no knowledge of verbal or written templates for patient handoffs, although both before and after the course they felt that standardization was an important component of the process.

A number of verbal structures for handing off patient care have been described in the literature and there is not a consensus as to which functions best. Perhaps the most cited verbal communication format is SBAR (ie, situation, background, assessment and recommendation).16, 17 This tool was developed by Leonard et al.18 specifically for use by nurses to provide 1‐way communication to physicians pertaining to a change in patient status. We considered teaching the SBAR approach to the students but felt that it did not provide a suitable structure for handoffs because the transfer of care is not generally an event‐based situation and the literature on handoffs indicates that an optimal verbal system includes 2‐way communication.

Additional mnemonics for handoffs found in the literature include SIGNOUT (ie, Sick or DNR, Identifying information, General hospital course, New events of the day, Overall health status, Upcoming possibilities with plan, and Tasks to complete),14 I PASS the BATON (ie, Introduction, Patient, Assessment, Situation, Safety, Background, Actions, Timing, Ownership, Next)19 and the SAIF‐IR system (see boxed text).14

We developed the SAIF‐IR mnemonic to maximize efficiency and effectiveness while differentiating the verbal portion of the handoff from the written and incorporating 2‐way communication into its structure. In the Summary statement, we emphasize that this is not a history of present illness. We ask our students to summarize, in 1 to 3 sentences, the patient's presentation and working diagnosis. When discussing patient issues, we ask our students to only verbalize Active issues, although the written template has inactive, chronic issues listed. Here, we also ask our students to express their level of concern for the active issues and patient in general. If‐then's and Follow‐ups are usually verbalized together. Based on the offgoing provider's knowledge of the patient, we encourage the offgoing provider to anticipate potential problems and advise the oncoming provider on potential responses. Much of this advice is difficult to express in the written format and thus may not be found on the written handoff when the verbal handoff occurs. We encourage oncoming providers to take notes on the preprinted handoff sheet as part of the handoff process.

Through Interactive questioning and Read‐backs, we train our students and house‐staff to use the active listening techniques used outside of healthcare, in settings such as nuclear power plants and National Aeronautics and Space Administration mission control, where poor handoff communication may also result in safety concerns and adverse events.20 Interactive questioning allows the oncoming provider to correct or clarify any information given by the off‐going provider. Read‐backs are a method of confirming follow‐up activity or contingency plans. Together, the SAIF‐IR mnemonic builds a 2‐way communication structure into the patient handoff with both offgoing and oncoming providers having predefined roles.

Much of the information on our written handoff (patient identifying information, medications, language preference, code status, admission date) is not verbalized unless it is part of the active issues or the if‐then, follow‐ups (ie, medication titration for a patient admitted with an acute coronary syndrome or cor status in a patient newly made comfort care). By not reading extraneous information, we seek to emphasize the Active issues as well as the If‐then, Follow‐ups. We feel this emphasis maximizes the effectiveness of the handoff, while the purposeful nonverbalization of written materials such as identifying information maximizes its efficiency. Future work may examine which verbal and written structures for patient handoffs most benefit patient care and workflow through standard communication.

While our students found the Handoff Selective to be useful and to improve their self‐perceived ability to perform handoffs, we were not able to determine whether our program affected downstream outcomes such as adverse events relating to failures in handoff communication. Additionally, since we only taught and evaluated our Selective at the University of Colorado Denver School of Medicine, the response of our students may not generalize to other medical schools. Multicentered, prospective, randomized controlled trials may determine whether handoff education programs are successful in reducing patient adverse events related to transfers of care.

While handoffs occur frequently and are increasingly recognized as a vulnerable time in patient care, little is known about how to effectively teach handoffs to medical students during their clinical years. We developed a formal course to teach the importance of handoffs and how the process should be conducted. Our students reported that the Handoff Selective we developed improved their knowledge about the process and their perception of their ability to perform handoffs in a time‐appropriate and effective manner. In response to the feedback we received from our students, the Handoff Selective is the only course in the ICC that has been made mandatory for all students.

Communication failures are well‐recognized as causes of medical errors.1, 2 Specifically, handoffs of patient care responsibilities, which are increasingly prevalent in academic medical centers,3 have been cited as the most frequent cause of teamwork breakdown resulting in the harmful medical errors found in malpractice claims.1 The Institute of Medicine has recently identified patient handoffs as the moment where patient care errors are most likely to occur.4 A survey of 125 U.S. medical schools, however, found that only 8% specifically taught students how to hand off patient care.3

In July 2003, the American Council of Graduate Medical Education (ACGME) mandated that residency programs decrease resident work hours to improve patient care and safety by reducing fatigue,5 and a recent Institute of Medicine report suggests that they be decreased even further.4 Studies examining outcomes during the first 2 years after reducing duty hours did not find reductions in risk‐adjusted mortality.68 One proposed explanation for this lack of improvement is that the reduction in fatigue‐related medical errors is being offset by discontinuity of care with due to the increased number of patient handoffs resulting from shortened duty hours,911 one recent study found that omission of key information during patient sign outs frequently resulted in adverse patient care outcomes.12

In 2007, the Joint Commission developed a new National Patient Safety Goal that requires organizations to improve communication between caregivers.13 We recently developed an approach by which Internal Medicine residents hand off patient care using a structured process, written and verbal templates, formal training about handoffs, and direct attending supervision.14 Because fourth‐year medical students perform the duties of interns when working as subinterns, we recognized that education about handoffs should occur prior to the time students became interns. Accordingly, we developed a course designed to teach patient handoffs to medical students at the transition between their third and fourth years of training.

Setting

The Handoff Selective was developed by faculty of Denver Health and the University of Colorado Denver School of Medicine.

Program Description

The Selective was first offered in April 2007 as part of an Integrated Clinician's Course (ICC), a 2‐week course for students beginning their fourth year, which starts in April at the University of Colorado. The ICC includes both mandatory and selective sessions that are focused on developing clinical skills and preparing them for their subinternships. The Handoff Selective was conducted in a computerized teaching laboratory, lasted a total of 2 hours and consisted of 2 parts. Each of the 5 Denver Health Hospital Medicine faculty members versed in handoff education taught 2 sessions of 6 to 8 students.

Part 1: Didactic

During the first hour of class, the faculty presented a lecture that summarized the relevant literature on handoffs and explained the importance of the topic. The objectives of the didactic were to: (1) understand the importance of handoffs; (2) explore different communication elements and structures; (3) gain exposure to handoffs outside of healthcare; and (4) learn a structure for handoffs of patient care in hospitalized patients.

We used 3 video clips of handoffs from 2 football games to demonstrate the importance of practice, training, and 2‐way communications in handoffs. The first video clip showed a runner trying to make a spontaneous handoff while being tackled. The receiver was not expecting the handoff and was preoccupied with blocking another player. This attempted handoff resulted in a fumble, which we related to an adverse patient event.

The next 2 video clips showed 2 complex, seldom used, but well‐known football handoffsthe hook and lateral and the Statue of Liberty. Both handoffs were successfully executed presumably as a result of education, practice and the active participation of both players (handing off and receiving) in the process. We then related the teaching and practicing of complex communication to the Joint Commission on Accreditation of Healthcare Organizations (JCAHO; now simply the Joint Commission) data suggesting that most sentinel events have their root cause in communication and training failures.2

Basic communication elements and process structures were then explored using scenarios from everyday life and evidence from fields outside of medicine. We emphasized that structures for communication (modes, vehicles, and settings) must be chosen according to the occasion and that handoffs are common and important in all occupations. In discussing modes (verbal, written, or nonverbal), vehicles (paper, telephone, or e‐mail), and settings (face‐to face, virtual, or disconnected), we emphasized that the most effective structures for communication (verbal, face‐to face meetings, with written materials and other visual aids at the patient's bedside) were also the most time‐consuming (Figure 1). While our standard for resident handoffs is a face‐to‐face verbal interaction with preprinted written materials as an aid, we also emphasized that for complex patients (eg, mental status changes, concern for an acute abdomen) more robust communication is often needed. Accordingly, a more time‐consuming bedside handoff with simultaneous, focused physical exam and history‐taking by both oncoming and off‐going providers may be most appropriate.

Figure 1

As real‐life examples, we asked our students to communicate a happy birthday wish to their mother, who lives in another state. Almost uniformly, in addition to a written aid (birthday card), they choose the telephone as a vehicle for their verbal mode in a virtual setting with 2‐way communication possible. In contrast, when asked to propose marriage to a significant other in another state, students felt that a face‐to‐face meeting with verbal and nonverbal (ie, ring) modes was appropriate. This time‐consuming mode of communication was felt to be necessary to create a sentiment of importance and avert any possible miscommunication.

The didactic session concluded by demonstrating how to use standardized written and verbal templates for handoffs of the care of a hospitalized patient. We explore the differentiation between written and verbal handoffs in our discussion below.

Program Evaluation

We developed a 2‐part survey to evaluate the effectiveness of the Selective and to solicit feedback about the didactic and practicum portions of the course. The first part of the survey (Table 2) contained 16 items to assess the students' knowledge of, and attitudes toward handing off patient care, along with their comfort with the handoff process. Responses to this section were scored using a 5‐point Likert scale with 1 indicating strongly disagree and 5 indicating strongly agree. This part of the survey was administered both prior to and after the Selective.

The second part (Table 3) contained 12 items and was designed to evaluate the perceived usefulness of the different components of the class. This section was only administered at the end of the Selective. It utilized a 4‐point Likert scale with 1 indicating that the component was not useful at all, and 4 indicating that it was extremely useful. The first 6 items of the second section allowed students to evaluate the didactic portion of the handoff. The second 6 items allowed students to evaluate the practicum. Responses to all 12 items were then combined to determine an overall composite usefulness for the Selective.

The Selective was also evaluated qualitatively through the use of open‐ended, written comments that were solicited at the end of the survey. All surveys were administered anonymously.

Results

More students chose the Selective than we had capacity to accommodate (60 of a class of 150). The pre‐ and postcourse survey response rate was 56 of 60 (93%) and 58 of 60 (97%), respectively. After the Selective, the mean score in response to whether handoffs are well taught in medical school increased from 1.6 to 3.5 (P < 0.003). Our students' self‐perceived skills and knowledge about handoffs improved after the Selective (Table 2). The greatest changes in perceived knowledge occurred in questions regarding the what, how, and when of read‐backs, and the knowledge of standard verbal and written handoff structures. The responses to the survey elements which assessed our students' attitudes regarding the importance of standardization and whether they felt handoffs were safer with faculty supervision did not change after the Selective (Table 2).

A total of 92% of the students felt that the course was extremely useful or useful. The role‐playing activity was thought to be more helpful than the didactic, but 84% of the students still rated the didactic portion as useful or extremely useful (Table 3). The element which was the least well received in the didactic portion was the use of video clips to demonstrate successful and unsuccessful (fumbled) college football handoffs, although the majority (64%) of students still found it useful.

The major theme generated from the comments section of the survey was that the Selective should be a required course.

Discussion

We know of no previously published literature that has addressed teaching handoffs to medical students. Horwitz et al.15 developed a sign‐out curriculum for Internal Medicine residents and found that none of their house‐staff had any previous training in handoffs during medical school, consistent with the finding that only 8% of U.S. medical schools provided formal instruction on handoffs.3 Prior to taking the Selective, our students had no knowledge of verbal or written templates for patient handoffs, although both before and after the course they felt that standardization was an important component of the process.

A number of verbal structures for handing off patient care have been described in the literature and there is not a consensus as to which functions best. Perhaps the most cited verbal communication format is SBAR (ie, situation, background, assessment and recommendation).16, 17 This tool was developed by Leonard et al.18 specifically for use by nurses to provide 1‐way communication to physicians pertaining to a change in patient status. We considered teaching the SBAR approach to the students but felt that it did not provide a suitable structure for handoffs because the transfer of care is not generally an event‐based situation and the literature on handoffs indicates that an optimal verbal system includes 2‐way communication.

Additional mnemonics for handoffs found in the literature include SIGNOUT (ie, Sick or DNR, Identifying information, General hospital course, New events of the day, Overall health status, Upcoming possibilities with plan, and Tasks to complete),14 I PASS the BATON (ie, Introduction, Patient, Assessment, Situation, Safety, Background, Actions, Timing, Ownership, Next)19 and the SAIF‐IR system (see boxed text).14

We developed the SAIF‐IR mnemonic to maximize efficiency and effectiveness while differentiating the verbal portion of the handoff from the written and incorporating 2‐way communication into its structure. In the Summary statement, we emphasize that this is not a history of present illness. We ask our students to summarize, in 1 to 3 sentences, the patient's presentation and working diagnosis. When discussing patient issues, we ask our students to only verbalize Active issues, although the written template has inactive, chronic issues listed. Here, we also ask our students to express their level of concern for the active issues and patient in general. If‐then's and Follow‐ups are usually verbalized together. Based on the offgoing provider's knowledge of the patient, we encourage the offgoing provider to anticipate potential problems and advise the oncoming provider on potential responses. Much of this advice is difficult to express in the written format and thus may not be found on the written handoff when the verbal handoff occurs. We encourage oncoming providers to take notes on the preprinted handoff sheet as part of the handoff process.

Through Interactive questioning and Read‐backs, we train our students and house‐staff to use the active listening techniques used outside of healthcare, in settings such as nuclear power plants and National Aeronautics and Space Administration mission control, where poor handoff communication may also result in safety concerns and adverse events.20 Interactive questioning allows the oncoming provider to correct or clarify any information given by the off‐going provider. Read‐backs are a method of confirming follow‐up activity or contingency plans. Together, the SAIF‐IR mnemonic builds a 2‐way communication structure into the patient handoff with both offgoing and oncoming providers having predefined roles.

Much of the information on our written handoff (patient identifying information, medications, language preference, code status, admission date) is not verbalized unless it is part of the active issues or the if‐then, follow‐ups (ie, medication titration for a patient admitted with an acute coronary syndrome or cor status in a patient newly made comfort care). By not reading extraneous information, we seek to emphasize the Active issues as well as the If‐then, Follow‐ups. We feel this emphasis maximizes the effectiveness of the handoff, while the purposeful nonverbalization of written materials such as identifying information maximizes its efficiency. Future work may examine which verbal and written structures for patient handoffs most benefit patient care and workflow through standard communication.

While our students found the Handoff Selective to be useful and to improve their self‐perceived ability to perform handoffs, we were not able to determine whether our program affected downstream outcomes such as adverse events relating to failures in handoff communication. Additionally, since we only taught and evaluated our Selective at the University of Colorado Denver School of Medicine, the response of our students may not generalize to other medical schools. Multicentered, prospective, randomized controlled trials may determine whether handoff education programs are successful in reducing patient adverse events related to transfers of care.

While handoffs occur frequently and are increasingly recognized as a vulnerable time in patient care, little is known about how to effectively teach handoffs to medical students during their clinical years. We developed a formal course to teach the importance of handoffs and how the process should be conducted. Our students reported that the Handoff Selective we developed improved their knowledge about the process and their perception of their ability to perform handoffs in a time‐appropriate and effective manner. In response to the feedback we received from our students, the Handoff Selective is the only course in the ICC that has been made mandatory for all students.