Vineet M. Arora, MD, MA

Determining when residents are independently prepared to perform clinical care tasks safely is not easy or understood. Educators have struggled to identify robust ways to evaluate trainees and their preparedness to treat patients while unsupervised. Trust allows the trainee to experience increasing levels of participation and responsibility in the workplace in a way that builds competence for future practice. The breadth of knowledge and skills required to become a competent and safe physician, coupled with the busy workload confound this challenge. Notably, a technically proficient trainee may not have the clinical judgment to treat patients without supervision.

The Accreditation Council of Graduate Medical Education (ACGME) has previously outlined 6 core competencies for residency training: patient care, medical knowledge, practice‐based learning and improvement, interpersonal and communication skills, professionalism, and systems‐based practice.[1] A systematic literature review suggests that traditional trainee evaluation tools are difficult to use and unreliable in measuring the competencies independently from one another, whereas certain competencies are consistently difficult to quantify in a reliable and valid way.[2] The evaluation of trainees' clinical performance despite efforts to create objective tools remain strongly influenced by subjective measures and continues to be highly variable among different evaluators.[3] Objectively measuring resident autonomy and readiness to supervise junior colleagues remains imprecise.[4]

The ACGME's Next Accreditation System (NAS) incorporates educational milestones as part of the reporting of resident training outcomes.[5] The milestones allow for the translation of the core competencies into integrative and observable abilities. Furthermore, the milestone categories are stratified into tiers to allow progress to be measured longitudinally and by task complexity using a novel assessment strategy.

The development of trust between supervisors and trainees is a critical step in decisions to allow increased responsibility and the provision of autonomous decision making, which is an important aspect of physician training. Identifying the factors that influence the supervisors' evaluation of resident competency and capability is at the crux of trainee maturation as well as patient safety.[4] Trust, defined as believability and discernment by attendings of resident physicians, plays a large role in attending evaluations of residents during their clinical rotations.[3] Trust impacts the decisions of successful performance of entrustable professional activities (EPAs), or those tasks that require mastery prior to completion of training milestones.[6] A study of entrustment decisions made by attending anesthesiologists identified the factors that contribute to the amount of autonomy given to residents, such as trainee trustworthiness, medical knowledge, and level of training.[4] The aim of our study, building on this study, was 2‐fold: (1) use deductive qualitative analysis to apply this framework to existing resident and attending data, and (2) define the categories within this framework and describe how internal medicine attending and resident physician perceptions of trust can impact clinical decision making and patient care.

METHODS

We are reporting on a secondary data analysis of interview transcripts from a study conducted on the inpatient general medicine service at the University of Chicago, an academic tertiary care medical center. The methods for data collection and full consent have been outlined previously.[7, 8, 9] The institutional review board of the University of Chicago approved this study.

Briefly, between January 2006 and November 2006, all eligible internal medicine resident physicians, postgraduate year (PGY)‐2 or PGY‐3, and attending physicians, either generalists or hospitalists, were privately interviewed within 1 week of their final call night on the inpatient general medicine rotation to assess decision making and clinical supervision during the rotation. All interviews were conducted by 1 investigator (J.F.), and discussions were audio taped and transcribed for analysis. Interviews were conducted at the conclusion of the rotation to prevent any influence on resident and attending behavior during the rotation.

The critical incident technique, a procedure used for collecting direct observations of human behavior that have critical significance on the decision‐making process, was used to solicit examples of ineffective supervision, inquiring about 2 to 3 important clinical decisions made on the most recent call night, with probes to identify issues of trust, autonomy, and decision making.[10] A critical incident can be described as one that makes a significant contribution, either positively or negatively, on the process.

Appreciative inquiry, a technique that aims to uncover the best things about the clinical encounter being explored, was used to solicit examples of effective supervision. Probes are used to identify factors, either personal or situational, that influenced the withholding or provision of resident autonomy during periods of clinical care delivery.[11]

All identifiable information was removed from the interview transcripts to protect participant and patient confidentiality. Deductive qualitative analysis was performed using the conceptual EPA framework, which describes several factors that influence the attending physicians' decisions to deem a resident trustworthy to independently fulfill a specific clinical task.[4] These factors include (1) the nature of the task, (2) the qualities of the supervisor, (3) the qualities of the trainee and the quality of the relationship between the supervisor and the trainee, and (4) the circumstances surrounding the clinical task.

The deidentified, anonymous transcripts were reviewed by 2 investigators (K.J.C., J.M.F.) and analyzed using the constant comparative methods to deductively map the content to the existing framework and generate novel sub themes.[12, 13, 14] Novel categories within each of the domains were inductively generated. Two reviewers (K.J.C., J.M.F.) independently applied the themes to a randomly selected 10% portion of the interview transcripts to assess the inter‐rater reliability. The inter‐rater agreement was assessed using the generalized kappa statistic. The discrepancies between reviewers regarding assignment of codes were resolved via discussion and third party adjudication until consensus was achieved on thematic structure. The codes were then applied to the entire dataset.

RESULTS

Between January 2006 and November 2006, 46 of 50 (88%) attending physicians and 44 of 50 (92%) resident physicians were interviewed following the conclusion of their general medicine inpatient rotation. Of attending physicians, 55% were male, 45% were female, and 38% were academic faculty hospitalists. Of the residents who completed interviews, 47% were male, 53% were female, 52% were PGY‐2, and 45% were PGY‐3.

A total of 535 mentions of trust were abstracted from the transcripts. The 4 major domains that influence trusttrainee factors (Table 1), supervisor factors (Table 2), task factors (Table 3), and systems factors (Table 4)were deductively coded with several emerging novel categories and subthemes. The domains were consistent across the postgraduate year of trainee. No differences in themes were noted, other than those explicitly stated, between the postgraduate years.

DISCUSSION

Clinical supervisors must hold the quality of care constant while balancing the amount of supervision and autonomy provided to learners in procedural tasks and clinical decision making. We found that the development of trust is multifactorial and highly contextual. It occurs under the broad constructs of task, supervisor, trainee, and environmental factors, and is well described in prior work. We also demonstrate that often what determines these broader factors is highly subjective, frequently independent of objective measures of trainee performance. Many decisions are based on personal characteristics, such as the perception of honesty, disposition, perceived confidence or perceived overconfidence of the trainee, prior experience, and expressed future field of specialty.

Our findings are consistent with prior research, but go further in describing and demonstrating the existence and innovative use of factors, other than clinical knowledge and skill, in the formation of a multidimensional construct of trust. Kennedy et al. identified 4 dimensions of trust knowledge and skill, discernment, conscientiousness, and truthfulness[15]and demonstrated that supervising physicians rely on specific processes to assess trainee trustworthiness, specifically the use of double checks and language cues. This is consistent with our results, which demonstrate that many attending physicians independently verify information, such as laboratory findings, to inform their perceptions of trainee honesty, attention to detail, and ability to follow orders reliably. Furthermore, our subthemes of communication and the demonstration of logical clinical reasoning correspond to Kennedy's use of language cues.[15] We found that language cues are used as markers of trustworthiness, particularly early on in the rotation, as a litmus test to gauge the trainee's integrity and ability to assess and treat patients unsupervised.

To date, much has been written about the importance of direct observation in the evaluation of trainees.[16, 17, 18, 19] Our results demonstrate that supervising clinicians use a multifactorial, highly nuanced, and subjective process despite validated performance‐based assessment methods, such as the objective structured clinical exam or mini‐clinical evaluation exercise, to assess competence and grant entrustement.[3] Several factors utilized to determine trustworthiness in addition to direct observation are subjective in nature, specifically the trainee's prior experience and expressed career choice.

It is encouraging that attending physicians make use of direct observations to inform decisions of entrustment, albeit in an informal and unstructured way. They also seem to take into account the context and setting in which the observation occurs, and consider both the environmental factors as well as factors that relate to the task itself.[20] For example, attendings and residents reported that team dynamics played a large role in influencing trust decisions. We also found that attending physicians rely on indirect observation and will inquire among their colleagues and other senior residents to gain information about their trainees abilities and integrity. Evaluation tools that facilitate sharing of trainees' level of preparedness, prior feedback, and experience could facilitate the determination of readiness to complete EPAs as well as the reporting of achieved milestones in accordance with the ACGME NAS.

Sharing knowledge about trainees among attendings is common and of increasing importance in the context of attending physicians' shortened exposure to trainees due to the residency work‐hour restrictions and growing productivity pressures. In our study, attending physicians described work‐hour restrictions as detrimental to trainee trustworthiness, either in the context of decreased accountability for patient care or as intrinsic to the nature of forced absences that kept trainees from fully participating in daily ward activities and knowing their patients. Attending physicians felt that trainees did not know their patients well enough to be able to make independent decisions about care. The increased transition to a shift‐based structure of inpatient medicine may result in increasingly less time for direct observation and make it more difficult for attendings to justify their decisions about engendering trust. In addition, the increased fragmentation that is noted in training secondary to the work‐hour regulations may in fact have consequences on the development of clinical skill and decision making, such that increased attention to the need for supervision and longer lead to entrustment may be needed in certain circumstances. Attendings need guidance on how to improve their ability to observe trainees in the context of the new work environment, and how to role model decision making more effectively in the compressed time exposure to housestaff.

Our study has several limitations. The organizational structure and culture of our institution are unique to 1 academic setting. This may undermine our ability to generalize these research findings and analysis to the population at large.[21] In addition, recall bias may have played into the interpretation of the interview content given the timing with which they were performed after the conclusion of the rotation. The study interviews took place in 2006, and it is reasonable to believe that some perceptions concerning duty‐hour restrictions and competency‐based graduate medical education have changed. However, from our ongoing research over the past 5 years[4] and our personal experience with entrustment factors, we believe that the participants' perceptions of trust and competency are valid and have largely remained unchanged, given the similarity in findings to the accepted ten Cate framework. In addition, this work was done following the first iteration of the work‐hour regulations but prior to the implementation of explicit supervisory levels, so it may indeed represent a truer state of the supervisory relationship before external regulations were applied. Finally, this work represents an internal medicine residency training program and may not be generalizable to other specialties that posses different cultural factors that impact the decision for entrustment. However, the congruence of our data with that of the original work of ten Cate, which was done in gynecology,[6] and that of Sterkenberg et al. in anesthesiology,[4] supports our key factors being ubiquitous to all training programs.

In conclusion, we provide new insights into subjective factors that inform the perceptions of trust and entrustment decisions by supervising physicians, specifically subjective trainee characteristics, team dynamics, and informal observation. There was agreement among attendings about which elements of competence are considered most important in their entrustment decisions related to trainee, supervisor, task, and environmental factors. Rather than undervaluing the use of personal factors in the determination of trust, we believe that acknowledgement and appreciation of these factors may be important to give supervisors more confidence and better tools to assess resident physicians, and to understand how their personality traits relate to and impact their professional competence. Our findings are relevant for the development of assessment instruments to evaluate whether medical graduates are ready for safe practice without supervision.

Determining when residents are independently prepared to perform clinical care tasks safely is not easy or understood. Educators have struggled to identify robust ways to evaluate trainees and their preparedness to treat patients while unsupervised. Trust allows the trainee to experience increasing levels of participation and responsibility in the workplace in a way that builds competence for future practice. The breadth of knowledge and skills required to become a competent and safe physician, coupled with the busy workload confound this challenge. Notably, a technically proficient trainee may not have the clinical judgment to treat patients without supervision.

The Accreditation Council of Graduate Medical Education (ACGME) has previously outlined 6 core competencies for residency training: patient care, medical knowledge, practice‐based learning and improvement, interpersonal and communication skills, professionalism, and systems‐based practice.[1] A systematic literature review suggests that traditional trainee evaluation tools are difficult to use and unreliable in measuring the competencies independently from one another, whereas certain competencies are consistently difficult to quantify in a reliable and valid way.[2] The evaluation of trainees' clinical performance despite efforts to create objective tools remain strongly influenced by subjective measures and continues to be highly variable among different evaluators.[3] Objectively measuring resident autonomy and readiness to supervise junior colleagues remains imprecise.[4]

The ACGME's Next Accreditation System (NAS) incorporates educational milestones as part of the reporting of resident training outcomes.[5] The milestones allow for the translation of the core competencies into integrative and observable abilities. Furthermore, the milestone categories are stratified into tiers to allow progress to be measured longitudinally and by task complexity using a novel assessment strategy.

The development of trust between supervisors and trainees is a critical step in decisions to allow increased responsibility and the provision of autonomous decision making, which is an important aspect of physician training. Identifying the factors that influence the supervisors' evaluation of resident competency and capability is at the crux of trainee maturation as well as patient safety.[4] Trust, defined as believability and discernment by attendings of resident physicians, plays a large role in attending evaluations of residents during their clinical rotations.[3] Trust impacts the decisions of successful performance of entrustable professional activities (EPAs), or those tasks that require mastery prior to completion of training milestones.[6] A study of entrustment decisions made by attending anesthesiologists identified the factors that contribute to the amount of autonomy given to residents, such as trainee trustworthiness, medical knowledge, and level of training.[4] The aim of our study, building on this study, was 2‐fold: (1) use deductive qualitative analysis to apply this framework to existing resident and attending data, and (2) define the categories within this framework and describe how internal medicine attending and resident physician perceptions of trust can impact clinical decision making and patient care.

METHODS

We are reporting on a secondary data analysis of interview transcripts from a study conducted on the inpatient general medicine service at the University of Chicago, an academic tertiary care medical center. The methods for data collection and full consent have been outlined previously.[7, 8, 9] The institutional review board of the University of Chicago approved this study.

Briefly, between January 2006 and November 2006, all eligible internal medicine resident physicians, postgraduate year (PGY)‐2 or PGY‐3, and attending physicians, either generalists or hospitalists, were privately interviewed within 1 week of their final call night on the inpatient general medicine rotation to assess decision making and clinical supervision during the rotation. All interviews were conducted by 1 investigator (J.F.), and discussions were audio taped and transcribed for analysis. Interviews were conducted at the conclusion of the rotation to prevent any influence on resident and attending behavior during the rotation.

The critical incident technique, a procedure used for collecting direct observations of human behavior that have critical significance on the decision‐making process, was used to solicit examples of ineffective supervision, inquiring about 2 to 3 important clinical decisions made on the most recent call night, with probes to identify issues of trust, autonomy, and decision making.[10] A critical incident can be described as one that makes a significant contribution, either positively or negatively, on the process.

Appreciative inquiry, a technique that aims to uncover the best things about the clinical encounter being explored, was used to solicit examples of effective supervision. Probes are used to identify factors, either personal or situational, that influenced the withholding or provision of resident autonomy during periods of clinical care delivery.[11]

All identifiable information was removed from the interview transcripts to protect participant and patient confidentiality. Deductive qualitative analysis was performed using the conceptual EPA framework, which describes several factors that influence the attending physicians' decisions to deem a resident trustworthy to independently fulfill a specific clinical task.[4] These factors include (1) the nature of the task, (2) the qualities of the supervisor, (3) the qualities of the trainee and the quality of the relationship between the supervisor and the trainee, and (4) the circumstances surrounding the clinical task.

The deidentified, anonymous transcripts were reviewed by 2 investigators (K.J.C., J.M.F.) and analyzed using the constant comparative methods to deductively map the content to the existing framework and generate novel sub themes.[12, 13, 14] Novel categories within each of the domains were inductively generated. Two reviewers (K.J.C., J.M.F.) independently applied the themes to a randomly selected 10% portion of the interview transcripts to assess the inter‐rater reliability. The inter‐rater agreement was assessed using the generalized kappa statistic. The discrepancies between reviewers regarding assignment of codes were resolved via discussion and third party adjudication until consensus was achieved on thematic structure. The codes were then applied to the entire dataset.

RESULTS

Between January 2006 and November 2006, 46 of 50 (88%) attending physicians and 44 of 50 (92%) resident physicians were interviewed following the conclusion of their general medicine inpatient rotation. Of attending physicians, 55% were male, 45% were female, and 38% were academic faculty hospitalists. Of the residents who completed interviews, 47% were male, 53% were female, 52% were PGY‐2, and 45% were PGY‐3.

A total of 535 mentions of trust were abstracted from the transcripts. The 4 major domains that influence trusttrainee factors (Table 1), supervisor factors (Table 2), task factors (Table 3), and systems factors (Table 4)were deductively coded with several emerging novel categories and subthemes. The domains were consistent across the postgraduate year of trainee. No differences in themes were noted, other than those explicitly stated, between the postgraduate years.

DISCUSSION

Clinical supervisors must hold the quality of care constant while balancing the amount of supervision and autonomy provided to learners in procedural tasks and clinical decision making. We found that the development of trust is multifactorial and highly contextual. It occurs under the broad constructs of task, supervisor, trainee, and environmental factors, and is well described in prior work. We also demonstrate that often what determines these broader factors is highly subjective, frequently independent of objective measures of trainee performance. Many decisions are based on personal characteristics, such as the perception of honesty, disposition, perceived confidence or perceived overconfidence of the trainee, prior experience, and expressed future field of specialty.

Our findings are consistent with prior research, but go further in describing and demonstrating the existence and innovative use of factors, other than clinical knowledge and skill, in the formation of a multidimensional construct of trust. Kennedy et al. identified 4 dimensions of trust knowledge and skill, discernment, conscientiousness, and truthfulness[15]and demonstrated that supervising physicians rely on specific processes to assess trainee trustworthiness, specifically the use of double checks and language cues. This is consistent with our results, which demonstrate that many attending physicians independently verify information, such as laboratory findings, to inform their perceptions of trainee honesty, attention to detail, and ability to follow orders reliably. Furthermore, our subthemes of communication and the demonstration of logical clinical reasoning correspond to Kennedy's use of language cues.[15] We found that language cues are used as markers of trustworthiness, particularly early on in the rotation, as a litmus test to gauge the trainee's integrity and ability to assess and treat patients unsupervised.

To date, much has been written about the importance of direct observation in the evaluation of trainees.[16, 17, 18, 19] Our results demonstrate that supervising clinicians use a multifactorial, highly nuanced, and subjective process despite validated performance‐based assessment methods, such as the objective structured clinical exam or mini‐clinical evaluation exercise, to assess competence and grant entrustement.[3] Several factors utilized to determine trustworthiness in addition to direct observation are subjective in nature, specifically the trainee's prior experience and expressed career choice.

It is encouraging that attending physicians make use of direct observations to inform decisions of entrustment, albeit in an informal and unstructured way. They also seem to take into account the context and setting in which the observation occurs, and consider both the environmental factors as well as factors that relate to the task itself.[20] For example, attendings and residents reported that team dynamics played a large role in influencing trust decisions. We also found that attending physicians rely on indirect observation and will inquire among their colleagues and other senior residents to gain information about their trainees abilities and integrity. Evaluation tools that facilitate sharing of trainees' level of preparedness, prior feedback, and experience could facilitate the determination of readiness to complete EPAs as well as the reporting of achieved milestones in accordance with the ACGME NAS.

Sharing knowledge about trainees among attendings is common and of increasing importance in the context of attending physicians' shortened exposure to trainees due to the residency work‐hour restrictions and growing productivity pressures. In our study, attending physicians described work‐hour restrictions as detrimental to trainee trustworthiness, either in the context of decreased accountability for patient care or as intrinsic to the nature of forced absences that kept trainees from fully participating in daily ward activities and knowing their patients. Attending physicians felt that trainees did not know their patients well enough to be able to make independent decisions about care. The increased transition to a shift‐based structure of inpatient medicine may result in increasingly less time for direct observation and make it more difficult for attendings to justify their decisions about engendering trust. In addition, the increased fragmentation that is noted in training secondary to the work‐hour regulations may in fact have consequences on the development of clinical skill and decision making, such that increased attention to the need for supervision and longer lead to entrustment may be needed in certain circumstances. Attendings need guidance on how to improve their ability to observe trainees in the context of the new work environment, and how to role model decision making more effectively in the compressed time exposure to housestaff.

Our study has several limitations. The organizational structure and culture of our institution are unique to 1 academic setting. This may undermine our ability to generalize these research findings and analysis to the population at large.[21] In addition, recall bias may have played into the interpretation of the interview content given the timing with which they were performed after the conclusion of the rotation. The study interviews took place in 2006, and it is reasonable to believe that some perceptions concerning duty‐hour restrictions and competency‐based graduate medical education have changed. However, from our ongoing research over the past 5 years[4] and our personal experience with entrustment factors, we believe that the participants' perceptions of trust and competency are valid and have largely remained unchanged, given the similarity in findings to the accepted ten Cate framework. In addition, this work was done following the first iteration of the work‐hour regulations but prior to the implementation of explicit supervisory levels, so it may indeed represent a truer state of the supervisory relationship before external regulations were applied. Finally, this work represents an internal medicine residency training program and may not be generalizable to other specialties that posses different cultural factors that impact the decision for entrustment. However, the congruence of our data with that of the original work of ten Cate, which was done in gynecology,[6] and that of Sterkenberg et al. in anesthesiology,[4] supports our key factors being ubiquitous to all training programs.

In conclusion, we provide new insights into subjective factors that inform the perceptions of trust and entrustment decisions by supervising physicians, specifically subjective trainee characteristics, team dynamics, and informal observation. There was agreement among attendings about which elements of competence are considered most important in their entrustment decisions related to trainee, supervisor, task, and environmental factors. Rather than undervaluing the use of personal factors in the determination of trust, we believe that acknowledgement and appreciation of these factors may be important to give supervisors more confidence and better tools to assess resident physicians, and to understand how their personality traits relate to and impact their professional competence. Our findings are relevant for the development of assessment instruments to evaluate whether medical graduates are ready for safe practice without supervision.

Determining when residents are independently prepared to perform clinical care tasks safely is not easy or understood. Educators have struggled to identify robust ways to evaluate trainees and their preparedness to treat patients while unsupervised. Trust allows the trainee to experience increasing levels of participation and responsibility in the workplace in a way that builds competence for future practice. The breadth of knowledge and skills required to become a competent and safe physician, coupled with the busy workload confound this challenge. Notably, a technically proficient trainee may not have the clinical judgment to treat patients without supervision.

The Accreditation Council of Graduate Medical Education (ACGME) has previously outlined 6 core competencies for residency training: patient care, medical knowledge, practice‐based learning and improvement, interpersonal and communication skills, professionalism, and systems‐based practice.[1] A systematic literature review suggests that traditional trainee evaluation tools are difficult to use and unreliable in measuring the competencies independently from one another, whereas certain competencies are consistently difficult to quantify in a reliable and valid way.[2] The evaluation of trainees' clinical performance despite efforts to create objective tools remain strongly influenced by subjective measures and continues to be highly variable among different evaluators.[3] Objectively measuring resident autonomy and readiness to supervise junior colleagues remains imprecise.[4]

The ACGME's Next Accreditation System (NAS) incorporates educational milestones as part of the reporting of resident training outcomes.[5] The milestones allow for the translation of the core competencies into integrative and observable abilities. Furthermore, the milestone categories are stratified into tiers to allow progress to be measured longitudinally and by task complexity using a novel assessment strategy.

The development of trust between supervisors and trainees is a critical step in decisions to allow increased responsibility and the provision of autonomous decision making, which is an important aspect of physician training. Identifying the factors that influence the supervisors' evaluation of resident competency and capability is at the crux of trainee maturation as well as patient safety.[4] Trust, defined as believability and discernment by attendings of resident physicians, plays a large role in attending evaluations of residents during their clinical rotations.[3] Trust impacts the decisions of successful performance of entrustable professional activities (EPAs), or those tasks that require mastery prior to completion of training milestones.[6] A study of entrustment decisions made by attending anesthesiologists identified the factors that contribute to the amount of autonomy given to residents, such as trainee trustworthiness, medical knowledge, and level of training.[4] The aim of our study, building on this study, was 2‐fold: (1) use deductive qualitative analysis to apply this framework to existing resident and attending data, and (2) define the categories within this framework and describe how internal medicine attending and resident physician perceptions of trust can impact clinical decision making and patient care.

METHODS

We are reporting on a secondary data analysis of interview transcripts from a study conducted on the inpatient general medicine service at the University of Chicago, an academic tertiary care medical center. The methods for data collection and full consent have been outlined previously.[7, 8, 9] The institutional review board of the University of Chicago approved this study.

Briefly, between January 2006 and November 2006, all eligible internal medicine resident physicians, postgraduate year (PGY)‐2 or PGY‐3, and attending physicians, either generalists or hospitalists, were privately interviewed within 1 week of their final call night on the inpatient general medicine rotation to assess decision making and clinical supervision during the rotation. All interviews were conducted by 1 investigator (J.F.), and discussions were audio taped and transcribed for analysis. Interviews were conducted at the conclusion of the rotation to prevent any influence on resident and attending behavior during the rotation.

The critical incident technique, a procedure used for collecting direct observations of human behavior that have critical significance on the decision‐making process, was used to solicit examples of ineffective supervision, inquiring about 2 to 3 important clinical decisions made on the most recent call night, with probes to identify issues of trust, autonomy, and decision making.[10] A critical incident can be described as one that makes a significant contribution, either positively or negatively, on the process.

Appreciative inquiry, a technique that aims to uncover the best things about the clinical encounter being explored, was used to solicit examples of effective supervision. Probes are used to identify factors, either personal or situational, that influenced the withholding or provision of resident autonomy during periods of clinical care delivery.[11]

All identifiable information was removed from the interview transcripts to protect participant and patient confidentiality. Deductive qualitative analysis was performed using the conceptual EPA framework, which describes several factors that influence the attending physicians' decisions to deem a resident trustworthy to independently fulfill a specific clinical task.[4] These factors include (1) the nature of the task, (2) the qualities of the supervisor, (3) the qualities of the trainee and the quality of the relationship between the supervisor and the trainee, and (4) the circumstances surrounding the clinical task.

The deidentified, anonymous transcripts were reviewed by 2 investigators (K.J.C., J.M.F.) and analyzed using the constant comparative methods to deductively map the content to the existing framework and generate novel sub themes.[12, 13, 14] Novel categories within each of the domains were inductively generated. Two reviewers (K.J.C., J.M.F.) independently applied the themes to a randomly selected 10% portion of the interview transcripts to assess the inter‐rater reliability. The inter‐rater agreement was assessed using the generalized kappa statistic. The discrepancies between reviewers regarding assignment of codes were resolved via discussion and third party adjudication until consensus was achieved on thematic structure. The codes were then applied to the entire dataset.

RESULTS

Between January 2006 and November 2006, 46 of 50 (88%) attending physicians and 44 of 50 (92%) resident physicians were interviewed following the conclusion of their general medicine inpatient rotation. Of attending physicians, 55% were male, 45% were female, and 38% were academic faculty hospitalists. Of the residents who completed interviews, 47% were male, 53% were female, 52% were PGY‐2, and 45% were PGY‐3.

A total of 535 mentions of trust were abstracted from the transcripts. The 4 major domains that influence trusttrainee factors (Table 1), supervisor factors (Table 2), task factors (Table 3), and systems factors (Table 4)were deductively coded with several emerging novel categories and subthemes. The domains were consistent across the postgraduate year of trainee. No differences in themes were noted, other than those explicitly stated, between the postgraduate years.

DISCUSSION

Clinical supervisors must hold the quality of care constant while balancing the amount of supervision and autonomy provided to learners in procedural tasks and clinical decision making. We found that the development of trust is multifactorial and highly contextual. It occurs under the broad constructs of task, supervisor, trainee, and environmental factors, and is well described in prior work. We also demonstrate that often what determines these broader factors is highly subjective, frequently independent of objective measures of trainee performance. Many decisions are based on personal characteristics, such as the perception of honesty, disposition, perceived confidence or perceived overconfidence of the trainee, prior experience, and expressed future field of specialty.

Our findings are consistent with prior research, but go further in describing and demonstrating the existence and innovative use of factors, other than clinical knowledge and skill, in the formation of a multidimensional construct of trust. Kennedy et al. identified 4 dimensions of trust knowledge and skill, discernment, conscientiousness, and truthfulness[15]and demonstrated that supervising physicians rely on specific processes to assess trainee trustworthiness, specifically the use of double checks and language cues. This is consistent with our results, which demonstrate that many attending physicians independently verify information, such as laboratory findings, to inform their perceptions of trainee honesty, attention to detail, and ability to follow orders reliably. Furthermore, our subthemes of communication and the demonstration of logical clinical reasoning correspond to Kennedy's use of language cues.[15] We found that language cues are used as markers of trustworthiness, particularly early on in the rotation, as a litmus test to gauge the trainee's integrity and ability to assess and treat patients unsupervised.

To date, much has been written about the importance of direct observation in the evaluation of trainees.[16, 17, 18, 19] Our results demonstrate that supervising clinicians use a multifactorial, highly nuanced, and subjective process despite validated performance‐based assessment methods, such as the objective structured clinical exam or mini‐clinical evaluation exercise, to assess competence and grant entrustement.[3] Several factors utilized to determine trustworthiness in addition to direct observation are subjective in nature, specifically the trainee's prior experience and expressed career choice.

It is encouraging that attending physicians make use of direct observations to inform decisions of entrustment, albeit in an informal and unstructured way. They also seem to take into account the context and setting in which the observation occurs, and consider both the environmental factors as well as factors that relate to the task itself.[20] For example, attendings and residents reported that team dynamics played a large role in influencing trust decisions. We also found that attending physicians rely on indirect observation and will inquire among their colleagues and other senior residents to gain information about their trainees abilities and integrity. Evaluation tools that facilitate sharing of trainees' level of preparedness, prior feedback, and experience could facilitate the determination of readiness to complete EPAs as well as the reporting of achieved milestones in accordance with the ACGME NAS.

Sharing knowledge about trainees among attendings is common and of increasing importance in the context of attending physicians' shortened exposure to trainees due to the residency work‐hour restrictions and growing productivity pressures. In our study, attending physicians described work‐hour restrictions as detrimental to trainee trustworthiness, either in the context of decreased accountability for patient care or as intrinsic to the nature of forced absences that kept trainees from fully participating in daily ward activities and knowing their patients. Attending physicians felt that trainees did not know their patients well enough to be able to make independent decisions about care. The increased transition to a shift‐based structure of inpatient medicine may result in increasingly less time for direct observation and make it more difficult for attendings to justify their decisions about engendering trust. In addition, the increased fragmentation that is noted in training secondary to the work‐hour regulations may in fact have consequences on the development of clinical skill and decision making, such that increased attention to the need for supervision and longer lead to entrustment may be needed in certain circumstances. Attendings need guidance on how to improve their ability to observe trainees in the context of the new work environment, and how to role model decision making more effectively in the compressed time exposure to housestaff.

Our study has several limitations. The organizational structure and culture of our institution are unique to 1 academic setting. This may undermine our ability to generalize these research findings and analysis to the population at large.[21] In addition, recall bias may have played into the interpretation of the interview content given the timing with which they were performed after the conclusion of the rotation. The study interviews took place in 2006, and it is reasonable to believe that some perceptions concerning duty‐hour restrictions and competency‐based graduate medical education have changed. However, from our ongoing research over the past 5 years[4] and our personal experience with entrustment factors, we believe that the participants' perceptions of trust and competency are valid and have largely remained unchanged, given the similarity in findings to the accepted ten Cate framework. In addition, this work was done following the first iteration of the work‐hour regulations but prior to the implementation of explicit supervisory levels, so it may indeed represent a truer state of the supervisory relationship before external regulations were applied. Finally, this work represents an internal medicine residency training program and may not be generalizable to other specialties that posses different cultural factors that impact the decision for entrustment. However, the congruence of our data with that of the original work of ten Cate, which was done in gynecology,[6] and that of Sterkenberg et al. in anesthesiology,[4] supports our key factors being ubiquitous to all training programs.

In conclusion, we provide new insights into subjective factors that inform the perceptions of trust and entrustment decisions by supervising physicians, specifically subjective trainee characteristics, team dynamics, and informal observation. There was agreement among attendings about which elements of competence are considered most important in their entrustment decisions related to trainee, supervisor, task, and environmental factors. Rather than undervaluing the use of personal factors in the determination of trust, we believe that acknowledgement and appreciation of these factors may be important to give supervisors more confidence and better tools to assess resident physicians, and to understand how their personality traits relate to and impact their professional competence. Our findings are relevant for the development of assessment instruments to evaluate whether medical graduates are ready for safe practice without supervision.

Postgraduate medical education has traditionally relied on a training model of progressive independence, where housestaff learn patient care through increasing autonomy and decreasing levels of supervision.1 While this framework has little empirical backing, it is grounded in sound educational theory from similar disciplines and endorsed by medical associations.1, 2 The Accreditation Council for Graduate Medical Education (ACGME) recently implemented regulations requiring that first‐year residents have a qualified supervisor physically present or immediately available at all times.3 Previously, oversight by an offsite supervisor (for example, an attending physician at home) was considered adequate. These new regulations, although motivated by patient safety imperatives,4 have elicited concerns that increased supervision may lead to decreased housestaff autonomy and an increased reliance on supervisors for clinical guidance.5 Such changes could ultimately produce less qualified practitioners by the completion of training.

Critics of the current training model point to a patient safety mechanism where housestaff must take responsibility for requesting attending‐level help when situations arise that surpass their skill level.5 For resident physicians, however, the decision to request support is often complex and dependent not only on the clinical question, but also on unique and variable trainee and supervisor factors.6 Survey data from 1999, prior to the current training regulations, showed that increased faculty presence improved resident reports of educational value, quality of patient care, and autonomy.7 A recent survey, performed after the initiation of overnight attending supervision at an academic medical center, demonstrated perceived improvements in educational value and patient‐level outcomes by both faculty and housestaff.8 Whether increased supervision and resident autonomy can coexist remains undetermined.

Overnight rotations for residents (commonly referred to as night float) are often times of little direct or indirect supervision. A recent systematic review of clinical supervision practices for housestaff in all fields found scarce literature on overnight supervision practices.9 There remains limited and conflicting data regarding the quality of patient care provided by the resident night float,10 as well as evidence revealing a low perceived educational value of night rotations when compared with non‐night float rotations.11 Yet in 2006, more than three‐quarters of all internal medicine programs employed night float rotations.12 In response to ACGME guidelines mandating decreased shift lengths with continued restrictions on overall duty hours, it appears likely even more training programs will implement night float systems.

The presence of overnight hospitalists (also known as nocturnists) is growing within the academic setting, yet their role in relation to trainees is either poorly defined13 or independent of housestaff.14 To better understand the impact of increasing levels of supervision on residency training, we investigated housestaff perceptions of education, autonomy, and clinical decision‐making before and after implementation of an in‐hospital, overnight attending physician (nocturnist).

METHODS

The study was conducted at a 570‐bed academic, tertiary care medical center affiliated with an internal medicine residency program of 170 housestaff. At our institution, all first year residents perform a week of intern night float consisting of overnight cross‐coverage of general medicine patients on the floor, step‐down, and intensive care units (ICUs). Second and third year residents each complete 4 to 6 days of resident night float each year at this hospital. They are responsible for assisting the intern night float with cross‐coverage, in addition to admitting general medicine patients to the floor, step‐down unit, and intensive care units. Every night at our medical center, 1 intern night float and 1 resident night float are on duty in the hospital; this is in addition to a resident from the on‐call medicine team and a resident working in the ICU. Prior to July 2010, no internal medicine attending physicians were physically present in the hospital at night. Oversight for the intern and resident night float was provided by the attending physician for the on‐call resident ward team, who was at home and available by pager. The night float housestaff were instructed to contact the responsible attending physician only when a major change in clinical status occurred for hospitalized or newly admitted patients, though this expectation was neither standardized nor monitored.

We established a nocturnist program at the start of the 2010 academic year. The position was staffed by hospitalists from within the Division of Hospital Medicine without the use of moonlighters. Two‐thirds of shifts were filled by 3 dedicated nocturnists with remaining staffing provided by junior hospitalist faculty. The dedicated nocturnists had recently completed their internal medicine residency at our institution. Shift length was 12 hours and dedicated nocturnists worked, on average, 10 shifts per month. The nocturnist filled a critical overnight safety role through mandatory bedside staffing of newly admitted ICU patients within 2 hours of admission, discussion in person or via telephone of newly admitted step‐down unit patients within 6 hours of admission, and direct or indirect supervision of the care of any patients undergoing a major change in clinical status. The overnight hospitalist was also available for clinical questions and to assist housestaff with triaging of overnight admissions. After nocturnist implementation, overnight housestaff received direct supervision or had immediate access to direct supervision, while prior to the nocturnist, residents had access only to indirect supervision.

In addition, the nocturnist admitted medicine patients after 1 _AM in a 1:1 ratio with the admitting night float resident, performed medical consults, and provided coverage of non‐teaching medicine services. While actual volume numbers were not obtained, the estimated average of resident admissions per night was 2 to 3, and the number of nocturnist admissions was 1 to 2. The nocturnist also met nightly with night float housestaff for half‐hour didactics focusing on the management of common overnight clinical scenarios. The role of the new nocturnist was described to all housestaff in orientation materials given prior to their night float rotation and their general medicine ward rotation.

We administered pre‐rolling surveys and post‐rolling surveys of internal medicine intern and resident physicians who underwent the night float rotation at our hospital during the 2010 to 2011 academic year. Surveys examined housestaff perceptions of the night float rotation with regard to supervisory roles, educational and clinical value, and clinical decision‐making prior to and after implementation of the nocturnist. Surveys were designed by the study investigators based on prior literature,1, 510 personal experience, and housestaff suggestion, and were refined during works‐in‐progress meetings. Surveys were composed of Likert‐style questions asking housestaff to rate their level of agreement (15, strongly disagree to strongly agree) with statements regarding the supervisory and educational experience of the night float rotation, and to judge their frequency of contact (15, never to always/nightly) with an attending physician for specific clinical scenarios. The clinical scenarios described situations dealing with attendingresident communication around transfers of care, diagnostic evaluation, therapeutic interventions, and adverse events. Scenarios were taken from previous literature describing supervision preferences of faculty and residents during times of critical clinical decision‐making.15

One week prior to the beginning their night float rotation for the 20102011 academic year, housestaff were sent an e‐mail request to complete an online survey asking about their night float rotation during the prior academic year, when no nocturnist was present. One week after completion of their night float rotation for the 20102011 academic year, housestaff received an e‐mail with a link to a post‐survey asking about their recently completed, nocturnist‐supervised, night float rotation. First year residents received only a post‐survey at the completion of their night float rotation, as they would be unable to reflect on prior experience.

Informed consent was imbedded within the e‐mail survey request. Survey requests were sent by a fellow within the Division of Hospital Medicine with a brief message cosigned by an associate program director of the residency program. We did not collect unique identifiers from respondents in order to offer additional assurances to the participants that the survey was anonymous. There was no incentive offered for completion of the survey. Survey data were anonymous and downloaded to a database by a third party. Data were analyzed using Microsoft Excel, and pre‐responses and post‐responses compared using a Student t test. The study was approved by the medical center's Institutional Review Board.

RESULTS

Rates of response for pre‐surveys and post‐surveys were 57% (43 respondents) and 51% (53 respondents), respectively. Due to response rates and in order to convey accurately the perceptions of the training program as a whole, we collapsed responses of the pre‐surveys and post‐surveys based on level of training. After implementation of the overnight attending, we observed a significant increase in the perceived clinical value of the night float rotation (3.95 vs 4.27, P = 0.01) as well as in the adequacy of overnight supervision (3.65 vs 4.30, P < 0.0001; Table 1). There was no reported change in housestaff decision‐making autonomy (4.35 vs 4.45, P = 0.44). In addition, we noted a nonsignificant trend towards an increased perception of the night float rotation as a valuable educational experience (3.83 vs 4.04, P = 0.24). After implementation of the nocturnist, more resident physicians agreed that overnight supervision by an attending positively impacted patient outcomes (3.79 vs 4.30, P = 0.002).

After implementation of the nocturnist, night float providers demonstrated increased rates of contacting an attending physician overnight (Table 2). There were significantly greater rates of attending contact for transfers from outside facilities (2.00 vs 3.20, P = 0.006) and during times of adverse events (2.51 vs 3.25, P = 0.04). We observed a reported increase in attending contact prior to ordering invasive diagnostic procedures (1.75 vs 2.76, P = 0.004) and noninvasive diagnostic procedures (1.09 vs 1.31, P = 0.03), as well as prior to initiation of intravenous antibiotics (1.11 vs 1.47, P = 0.007) and vasopressors (1.52 vs 2.40, P = 0.004).

After initiating the program, the nocturnist became the most commonly contacted overnight provider by the night float housestaff (Table 3). We observed a decrease in peer to peer contact between the night float housestaff and the on‐call overnight resident after implementation of the nocturnist (2.67 vs 2.04, P = 0.006).

Attending presence led to increased agreement that there was a defined overnight attending to contact (2.97 vs 1.96, P < 0.0001) and a decreased fear of waking an attending overnight for assistance (3.26 vs 2.72, P = 0.03). Increased attending availability, however, did not change resident physician's fear of revealing knowledge gaps, their desire to make decisions independently, or their belief that contacting an attending would not change a patient's outcome (Table 4).

DISCUSSION

The ACGME's new duty hour regulations require that supervision for first‐year residents be provided by a qualified physician (advanced resident, fellow, or attending physician) who is physically present at the hospital. Our study demonstrates that increased direct overnight supervision provided by an in‐house nocturnist enhanced the clinical value of the night float rotation and the perceived quality of patient care. In our study, increased attending supervision did not reduce perceived decision‐making autonomy, and in fact led to increased rates of attending contact during times of critical clinical decision‐making. Such results may help assuage fears that recent regulations mandating enhanced attending supervision will produce less capable practitioners, and offers reassurance that such changes are positively impacting patient care.

Many academic institutions are implementing nocturnists, although their precise roles and responsibilities are still being defined. Our nocturnist program was explicitly designed with housestaff supervision as a core responsibility, with the goal of improving patient safety and housestaff education overnight. We found that availability barriers to attending contact were logically decreased with in‐house faculty presence. Potentially harmful attitudes, however, around requesting support (such as fear of revealing knowledge gaps or the desire to make decisions independently) remained. Furthermore, despite statistically significant increases in contact between faculty and residents at times of critical decision‐making, overall rates of attending contact for diagnostic and therapeutic interventions remained low. It is unknown from our study or previous research, however, what level of contact is appropriate or ideal for many clinical scenarios.

Additionally, we described a novel role of an academic nocturnist at a tertiary care teaching hospital and offered a potential template for the development of academic nocturnists at similar institutions seeking to increase direct overnight supervision. Such roles have not been previously well defined in the literature. Based on our experience, the nocturnist's role was manageable and well utilized by housestaff, particularly for assistance with critically ill patients and overnight triaging. We believe there are a number of factors associated with the success of this role. First, clear guidelines were presented to housestaff and nocturnists regarding expectations for supervision (for example, staffing ICU admissions within 2 hours). These guidelines likely contributed to the increased attending contact observed during critical clinical decision‐making, as well as the perceived improved patient outcomes by our housestaff. Second, the nocturnists were expected to be an integral part of the overnight care team. In many systems, the nocturnists act completely independently of the housestaff teams, creating an additional barrier to contact and communication. In our system, because of clear guidelines and their integral role in staffing overnight admissions, the nocturnists were an essential partner in care for the housestaff. Third, most of the nocturnists had recently completed their residency training at this institution. Although our survey does not directly address this, we believe their knowledge of the hospital, appreciation of the role of the intern and the resident within our system, and understanding of the need to preserve housestaff autonomy were essential to building a successful nocturnist role. Lastly, the nocturnists were not only expected to supervise and staff new admissions, but were also given a teaching expectation. We believe they were viewed by housestaff as qualified teaching attendings, similar to the daytime hospitalist. These findings may provide guidelines for other institutions seeking to balance overnight hospitalist supervision with preserving resident's ability to make autonomous decisions.

There are several limitations to our study. The findings represent the experience of internal medicine housestaff at a single academic, tertiary care medical center and may not be reflective of other institutions or specialties. We asked housestaff to recall night float experiences from the prior year, which may have introduced recall bias, though responses were obtained before participants underwent the new curriculum. Maturation of housestaff over time could have led to changes in perceived autonomy, value of the night float rotation, and rates of attending contact independent of nocturnist implementation. In addition, there may have been unaccounted changes to other elements of the residency program, hospital, or patient volume between rotations. The implementation of the nocturnist, however, was the only major change to our training program that academic year, and there were no significant changes in patient volume, structure of the teaching or non‐resident services, or other policies around resident supervision.

It is possible that the nocturnist may have contributed to reports of increased clinical value and perceived quality of patient care simply by decreasing overnight workload for housestaff, and enhanced supervision and teaching may have played a lesser role. Even if this were true, optimizing resident workload is in itself an important goal for teaching hospitals and residency programs alike in order to maximize patient safety. Inclusion of intern post‐rotation surveys may have influenced data; though, we had no reason to suspect the surveyed interns would respond in a different manner than prior resident groups. The responses of both junior and senior housestaff were pooled; while this potentially weighted the results in favor of higher responding groups, we felt that it conveyed the residents' accurate sentiments on the program. Finally, while we compared two models of overnight supervision, we reported only housestaff perceptions of education, autonomy, patient outcomes, and supervisory contact, and not direct measures of knowledge or patient care. Further research will be required to define the relationship between supervision practices and patient‐level clinical outcomes.

The new ACGME regulations around resident supervision, as well as the broader movement to improve the safety and quality of care, require residency programs to negotiate a delicate balance between providing high‐quality patient care while preserving graduated independence in clinical training. Our study demonstrates that increased overnight supervision by nocturnists with well‐defined supervisory and teaching roles can preserve housestaff autonomy, improve the clinical experience for trainees, increase access to support during times of critical decision‐making, and potentially lead to improved patient outcomes.

Postgraduate medical education has traditionally relied on a training model of progressive independence, where housestaff learn patient care through increasing autonomy and decreasing levels of supervision.1 While this framework has little empirical backing, it is grounded in sound educational theory from similar disciplines and endorsed by medical associations.1, 2 The Accreditation Council for Graduate Medical Education (ACGME) recently implemented regulations requiring that first‐year residents have a qualified supervisor physically present or immediately available at all times.3 Previously, oversight by an offsite supervisor (for example, an attending physician at home) was considered adequate. These new regulations, although motivated by patient safety imperatives,4 have elicited concerns that increased supervision may lead to decreased housestaff autonomy and an increased reliance on supervisors for clinical guidance.5 Such changes could ultimately produce less qualified practitioners by the completion of training.

Critics of the current training model point to a patient safety mechanism where housestaff must take responsibility for requesting attending‐level help when situations arise that surpass their skill level.5 For resident physicians, however, the decision to request support is often complex and dependent not only on the clinical question, but also on unique and variable trainee and supervisor factors.6 Survey data from 1999, prior to the current training regulations, showed that increased faculty presence improved resident reports of educational value, quality of patient care, and autonomy.7 A recent survey, performed after the initiation of overnight attending supervision at an academic medical center, demonstrated perceived improvements in educational value and patient‐level outcomes by both faculty and housestaff.8 Whether increased supervision and resident autonomy can coexist remains undetermined.

Overnight rotations for residents (commonly referred to as night float) are often times of little direct or indirect supervision. A recent systematic review of clinical supervision practices for housestaff in all fields found scarce literature on overnight supervision practices.9 There remains limited and conflicting data regarding the quality of patient care provided by the resident night float,10 as well as evidence revealing a low perceived educational value of night rotations when compared with non‐night float rotations.11 Yet in 2006, more than three‐quarters of all internal medicine programs employed night float rotations.12 In response to ACGME guidelines mandating decreased shift lengths with continued restrictions on overall duty hours, it appears likely even more training programs will implement night float systems.

The presence of overnight hospitalists (also known as nocturnists) is growing within the academic setting, yet their role in relation to trainees is either poorly defined13 or independent of housestaff.14 To better understand the impact of increasing levels of supervision on residency training, we investigated housestaff perceptions of education, autonomy, and clinical decision‐making before and after implementation of an in‐hospital, overnight attending physician (nocturnist).

METHODS

The study was conducted at a 570‐bed academic, tertiary care medical center affiliated with an internal medicine residency program of 170 housestaff. At our institution, all first year residents perform a week of intern night float consisting of overnight cross‐coverage of general medicine patients on the floor, step‐down, and intensive care units (ICUs). Second and third year residents each complete 4 to 6 days of resident night float each year at this hospital. They are responsible for assisting the intern night float with cross‐coverage, in addition to admitting general medicine patients to the floor, step‐down unit, and intensive care units. Every night at our medical center, 1 intern night float and 1 resident night float are on duty in the hospital; this is in addition to a resident from the on‐call medicine team and a resident working in the ICU. Prior to July 2010, no internal medicine attending physicians were physically present in the hospital at night. Oversight for the intern and resident night float was provided by the attending physician for the on‐call resident ward team, who was at home and available by pager. The night float housestaff were instructed to contact the responsible attending physician only when a major change in clinical status occurred for hospitalized or newly admitted patients, though this expectation was neither standardized nor monitored.

We established a nocturnist program at the start of the 2010 academic year. The position was staffed by hospitalists from within the Division of Hospital Medicine without the use of moonlighters. Two‐thirds of shifts were filled by 3 dedicated nocturnists with remaining staffing provided by junior hospitalist faculty. The dedicated nocturnists had recently completed their internal medicine residency at our institution. Shift length was 12 hours and dedicated nocturnists worked, on average, 10 shifts per month. The nocturnist filled a critical overnight safety role through mandatory bedside staffing of newly admitted ICU patients within 2 hours of admission, discussion in person or via telephone of newly admitted step‐down unit patients within 6 hours of admission, and direct or indirect supervision of the care of any patients undergoing a major change in clinical status. The overnight hospitalist was also available for clinical questions and to assist housestaff with triaging of overnight admissions. After nocturnist implementation, overnight housestaff received direct supervision or had immediate access to direct supervision, while prior to the nocturnist, residents had access only to indirect supervision.

In addition, the nocturnist admitted medicine patients after 1 _AM in a 1:1 ratio with the admitting night float resident, performed medical consults, and provided coverage of non‐teaching medicine services. While actual volume numbers were not obtained, the estimated average of resident admissions per night was 2 to 3, and the number of nocturnist admissions was 1 to 2. The nocturnist also met nightly with night float housestaff for half‐hour didactics focusing on the management of common overnight clinical scenarios. The role of the new nocturnist was described to all housestaff in orientation materials given prior to their night float rotation and their general medicine ward rotation.

We administered pre‐rolling surveys and post‐rolling surveys of internal medicine intern and resident physicians who underwent the night float rotation at our hospital during the 2010 to 2011 academic year. Surveys examined housestaff perceptions of the night float rotation with regard to supervisory roles, educational and clinical value, and clinical decision‐making prior to and after implementation of the nocturnist. Surveys were designed by the study investigators based on prior literature,1, 510 personal experience, and housestaff suggestion, and were refined during works‐in‐progress meetings. Surveys were composed of Likert‐style questions asking housestaff to rate their level of agreement (15, strongly disagree to strongly agree) with statements regarding the supervisory and educational experience of the night float rotation, and to judge their frequency of contact (15, never to always/nightly) with an attending physician for specific clinical scenarios. The clinical scenarios described situations dealing with attendingresident communication around transfers of care, diagnostic evaluation, therapeutic interventions, and adverse events. Scenarios were taken from previous literature describing supervision preferences of faculty and residents during times of critical clinical decision‐making.15

One week prior to the beginning their night float rotation for the 20102011 academic year, housestaff were sent an e‐mail request to complete an online survey asking about their night float rotation during the prior academic year, when no nocturnist was present. One week after completion of their night float rotation for the 20102011 academic year, housestaff received an e‐mail with a link to a post‐survey asking about their recently completed, nocturnist‐supervised, night float rotation. First year residents received only a post‐survey at the completion of their night float rotation, as they would be unable to reflect on prior experience.

Informed consent was imbedded within the e‐mail survey request. Survey requests were sent by a fellow within the Division of Hospital Medicine with a brief message cosigned by an associate program director of the residency program. We did not collect unique identifiers from respondents in order to offer additional assurances to the participants that the survey was anonymous. There was no incentive offered for completion of the survey. Survey data were anonymous and downloaded to a database by a third party. Data were analyzed using Microsoft Excel, and pre‐responses and post‐responses compared using a Student t test. The study was approved by the medical center's Institutional Review Board.

RESULTS

Rates of response for pre‐surveys and post‐surveys were 57% (43 respondents) and 51% (53 respondents), respectively. Due to response rates and in order to convey accurately the perceptions of the training program as a whole, we collapsed responses of the pre‐surveys and post‐surveys based on level of training. After implementation of the overnight attending, we observed a significant increase in the perceived clinical value of the night float rotation (3.95 vs 4.27, P = 0.01) as well as in the adequacy of overnight supervision (3.65 vs 4.30, P < 0.0001; Table 1). There was no reported change in housestaff decision‐making autonomy (4.35 vs 4.45, P = 0.44). In addition, we noted a nonsignificant trend towards an increased perception of the night float rotation as a valuable educational experience (3.83 vs 4.04, P = 0.24). After implementation of the nocturnist, more resident physicians agreed that overnight supervision by an attending positively impacted patient outcomes (3.79 vs 4.30, P = 0.002).

After implementation of the nocturnist, night float providers demonstrated increased rates of contacting an attending physician overnight (Table 2). There were significantly greater rates of attending contact for transfers from outside facilities (2.00 vs 3.20, P = 0.006) and during times of adverse events (2.51 vs 3.25, P = 0.04). We observed a reported increase in attending contact prior to ordering invasive diagnostic procedures (1.75 vs 2.76, P = 0.004) and noninvasive diagnostic procedures (1.09 vs 1.31, P = 0.03), as well as prior to initiation of intravenous antibiotics (1.11 vs 1.47, P = 0.007) and vasopressors (1.52 vs 2.40, P = 0.004).

After initiating the program, the nocturnist became the most commonly contacted overnight provider by the night float housestaff (Table 3). We observed a decrease in peer to peer contact between the night float housestaff and the on‐call overnight resident after implementation of the nocturnist (2.67 vs 2.04, P = 0.006).

Attending presence led to increased agreement that there was a defined overnight attending to contact (2.97 vs 1.96, P < 0.0001) and a decreased fear of waking an attending overnight for assistance (3.26 vs 2.72, P = 0.03). Increased attending availability, however, did not change resident physician's fear of revealing knowledge gaps, their desire to make decisions independently, or their belief that contacting an attending would not change a patient's outcome (Table 4).

DISCUSSION

The ACGME's new duty hour regulations require that supervision for first‐year residents be provided by a qualified physician (advanced resident, fellow, or attending physician) who is physically present at the hospital. Our study demonstrates that increased direct overnight supervision provided by an in‐house nocturnist enhanced the clinical value of the night float rotation and the perceived quality of patient care. In our study, increased attending supervision did not reduce perceived decision‐making autonomy, and in fact led to increased rates of attending contact during times of critical clinical decision‐making. Such results may help assuage fears that recent regulations mandating enhanced attending supervision will produce less capable practitioners, and offers reassurance that such changes are positively impacting patient care.

Many academic institutions are implementing nocturnists, although their precise roles and responsibilities are still being defined. Our nocturnist program was explicitly designed with housestaff supervision as a core responsibility, with the goal of improving patient safety and housestaff education overnight. We found that availability barriers to attending contact were logically decreased with in‐house faculty presence. Potentially harmful attitudes, however, around requesting support (such as fear of revealing knowledge gaps or the desire to make decisions independently) remained. Furthermore, despite statistically significant increases in contact between faculty and residents at times of critical decision‐making, overall rates of attending contact for diagnostic and therapeutic interventions remained low. It is unknown from our study or previous research, however, what level of contact is appropriate or ideal for many clinical scenarios.

Additionally, we described a novel role of an academic nocturnist at a tertiary care teaching hospital and offered a potential template for the development of academic nocturnists at similar institutions seeking to increase direct overnight supervision. Such roles have not been previously well defined in the literature. Based on our experience, the nocturnist's role was manageable and well utilized by housestaff, particularly for assistance with critically ill patients and overnight triaging. We believe there are a number of factors associated with the success of this role. First, clear guidelines were presented to housestaff and nocturnists regarding expectations for supervision (for example, staffing ICU admissions within 2 hours). These guidelines likely contributed to the increased attending contact observed during critical clinical decision‐making, as well as the perceived improved patient outcomes by our housestaff. Second, the nocturnists were expected to be an integral part of the overnight care team. In many systems, the nocturnists act completely independently of the housestaff teams, creating an additional barrier to contact and communication. In our system, because of clear guidelines and their integral role in staffing overnight admissions, the nocturnists were an essential partner in care for the housestaff. Third, most of the nocturnists had recently completed their residency training at this institution. Although our survey does not directly address this, we believe their knowledge of the hospital, appreciation of the role of the intern and the resident within our system, and understanding of the need to preserve housestaff autonomy were essential to building a successful nocturnist role. Lastly, the nocturnists were not only expected to supervise and staff new admissions, but were also given a teaching expectation. We believe they were viewed by housestaff as qualified teaching attendings, similar to the daytime hospitalist. These findings may provide guidelines for other institutions seeking to balance overnight hospitalist supervision with preserving resident's ability to make autonomous decisions.

There are several limitations to our study. The findings represent the experience of internal medicine housestaff at a single academic, tertiary care medical center and may not be reflective of other institutions or specialties. We asked housestaff to recall night float experiences from the prior year, which may have introduced recall bias, though responses were obtained before participants underwent the new curriculum. Maturation of housestaff over time could have led to changes in perceived autonomy, value of the night float rotation, and rates of attending contact independent of nocturnist implementation. In addition, there may have been unaccounted changes to other elements of the residency program, hospital, or patient volume between rotations. The implementation of the nocturnist, however, was the only major change to our training program that academic year, and there were no significant changes in patient volume, structure of the teaching or non‐resident services, or other policies around resident supervision.

It is possible that the nocturnist may have contributed to reports of increased clinical value and perceived quality of patient care simply by decreasing overnight workload for housestaff, and enhanced supervision and teaching may have played a lesser role. Even if this were true, optimizing resident workload is in itself an important goal for teaching hospitals and residency programs alike in order to maximize patient safety. Inclusion of intern post‐rotation surveys may have influenced data; though, we had no reason to suspect the surveyed interns would respond in a different manner than prior resident groups. The responses of both junior and senior housestaff were pooled; while this potentially weighted the results in favor of higher responding groups, we felt that it conveyed the residents' accurate sentiments on the program. Finally, while we compared two models of overnight supervision, we reported only housestaff perceptions of education, autonomy, patient outcomes, and supervisory contact, and not direct measures of knowledge or patient care. Further research will be required to define the relationship between supervision practices and patient‐level clinical outcomes.

The new ACGME regulations around resident supervision, as well as the broader movement to improve the safety and quality of care, require residency programs to negotiate a delicate balance between providing high‐quality patient care while preserving graduated independence in clinical training. Our study demonstrates that increased overnight supervision by nocturnists with well‐defined supervisory and teaching roles can preserve housestaff autonomy, improve the clinical experience for trainees, increase access to support during times of critical decision‐making, and potentially lead to improved patient outcomes.

Postgraduate medical education has traditionally relied on a training model of progressive independence, where housestaff learn patient care through increasing autonomy and decreasing levels of supervision.1 While this framework has little empirical backing, it is grounded in sound educational theory from similar disciplines and endorsed by medical associations.1, 2 The Accreditation Council for Graduate Medical Education (ACGME) recently implemented regulations requiring that first‐year residents have a qualified supervisor physically present or immediately available at all times.3 Previously, oversight by an offsite supervisor (for example, an attending physician at home) was considered adequate. These new regulations, although motivated by patient safety imperatives,4 have elicited concerns that increased supervision may lead to decreased housestaff autonomy and an increased reliance on supervisors for clinical guidance.5 Such changes could ultimately produce less qualified practitioners by the completion of training.

Critics of the current training model point to a patient safety mechanism where housestaff must take responsibility for requesting attending‐level help when situations arise that surpass their skill level.5 For resident physicians, however, the decision to request support is often complex and dependent not only on the clinical question, but also on unique and variable trainee and supervisor factors.6 Survey data from 1999, prior to the current training regulations, showed that increased faculty presence improved resident reports of educational value, quality of patient care, and autonomy.7 A recent survey, performed after the initiation of overnight attending supervision at an academic medical center, demonstrated perceived improvements in educational value and patient‐level outcomes by both faculty and housestaff.8 Whether increased supervision and resident autonomy can coexist remains undetermined.

Overnight rotations for residents (commonly referred to as night float) are often times of little direct or indirect supervision. A recent systematic review of clinical supervision practices for housestaff in all fields found scarce literature on overnight supervision practices.9 There remains limited and conflicting data regarding the quality of patient care provided by the resident night float,10 as well as evidence revealing a low perceived educational value of night rotations when compared with non‐night float rotations.11 Yet in 2006, more than three‐quarters of all internal medicine programs employed night float rotations.12 In response to ACGME guidelines mandating decreased shift lengths with continued restrictions on overall duty hours, it appears likely even more training programs will implement night float systems.

The presence of overnight hospitalists (also known as nocturnists) is growing within the academic setting, yet their role in relation to trainees is either poorly defined13 or independent of housestaff.14 To better understand the impact of increasing levels of supervision on residency training, we investigated housestaff perceptions of education, autonomy, and clinical decision‐making before and after implementation of an in‐hospital, overnight attending physician (nocturnist).

METHODS

The study was conducted at a 570‐bed academic, tertiary care medical center affiliated with an internal medicine residency program of 170 housestaff. At our institution, all first year residents perform a week of intern night float consisting of overnight cross‐coverage of general medicine patients on the floor, step‐down, and intensive care units (ICUs). Second and third year residents each complete 4 to 6 days of resident night float each year at this hospital. They are responsible for assisting the intern night float with cross‐coverage, in addition to admitting general medicine patients to the floor, step‐down unit, and intensive care units. Every night at our medical center, 1 intern night float and 1 resident night float are on duty in the hospital; this is in addition to a resident from the on‐call medicine team and a resident working in the ICU. Prior to July 2010, no internal medicine attending physicians were physically present in the hospital at night. Oversight for the intern and resident night float was provided by the attending physician for the on‐call resident ward team, who was at home and available by pager. The night float housestaff were instructed to contact the responsible attending physician only when a major change in clinical status occurred for hospitalized or newly admitted patients, though this expectation was neither standardized nor monitored.

We established a nocturnist program at the start of the 2010 academic year. The position was staffed by hospitalists from within the Division of Hospital Medicine without the use of moonlighters. Two‐thirds of shifts were filled by 3 dedicated nocturnists with remaining staffing provided by junior hospitalist faculty. The dedicated nocturnists had recently completed their internal medicine residency at our institution. Shift length was 12 hours and dedicated nocturnists worked, on average, 10 shifts per month. The nocturnist filled a critical overnight safety role through mandatory bedside staffing of newly admitted ICU patients within 2 hours of admission, discussion in person or via telephone of newly admitted step‐down unit patients within 6 hours of admission, and direct or indirect supervision of the care of any patients undergoing a major change in clinical status. The overnight hospitalist was also available for clinical questions and to assist housestaff with triaging of overnight admissions. After nocturnist implementation, overnight housestaff received direct supervision or had immediate access to direct supervision, while prior to the nocturnist, residents had access only to indirect supervision.

In addition, the nocturnist admitted medicine patients after 1 _AM in a 1:1 ratio with the admitting night float resident, performed medical consults, and provided coverage of non‐teaching medicine services. While actual volume numbers were not obtained, the estimated average of resident admissions per night was 2 to 3, and the number of nocturnist admissions was 1 to 2. The nocturnist also met nightly with night float housestaff for half‐hour didactics focusing on the management of common overnight clinical scenarios. The role of the new nocturnist was described to all housestaff in orientation materials given prior to their night float rotation and their general medicine ward rotation.

We administered pre‐rolling surveys and post‐rolling surveys of internal medicine intern and resident physicians who underwent the night float rotation at our hospital during the 2010 to 2011 academic year. Surveys examined housestaff perceptions of the night float rotation with regard to supervisory roles, educational and clinical value, and clinical decision‐making prior to and after implementation of the nocturnist. Surveys were designed by the study investigators based on prior literature,1, 510 personal experience, and housestaff suggestion, and were refined during works‐in‐progress meetings. Surveys were composed of Likert‐style questions asking housestaff to rate their level of agreement (15, strongly disagree to strongly agree) with statements regarding the supervisory and educational experience of the night float rotation, and to judge their frequency of contact (15, never to always/nightly) with an attending physician for specific clinical scenarios. The clinical scenarios described situations dealing with attendingresident communication around transfers of care, diagnostic evaluation, therapeutic interventions, and adverse events. Scenarios were taken from previous literature describing supervision preferences of faculty and residents during times of critical clinical decision‐making.15

One week prior to the beginning their night float rotation for the 20102011 academic year, housestaff were sent an e‐mail request to complete an online survey asking about their night float rotation during the prior academic year, when no nocturnist was present. One week after completion of their night float rotation for the 20102011 academic year, housestaff received an e‐mail with a link to a post‐survey asking about their recently completed, nocturnist‐supervised, night float rotation. First year residents received only a post‐survey at the completion of their night float rotation, as they would be unable to reflect on prior experience.

Informed consent was imbedded within the e‐mail survey request. Survey requests were sent by a fellow within the Division of Hospital Medicine with a brief message cosigned by an associate program director of the residency program. We did not collect unique identifiers from respondents in order to offer additional assurances to the participants that the survey was anonymous. There was no incentive offered for completion of the survey. Survey data were anonymous and downloaded to a database by a third party. Data were analyzed using Microsoft Excel, and pre‐responses and post‐responses compared using a Student t test. The study was approved by the medical center's Institutional Review Board.

RESULTS

Rates of response for pre‐surveys and post‐surveys were 57% (43 respondents) and 51% (53 respondents), respectively. Due to response rates and in order to convey accurately the perceptions of the training program as a whole, we collapsed responses of the pre‐surveys and post‐surveys based on level of training. After implementation of the overnight attending, we observed a significant increase in the perceived clinical value of the night float rotation (3.95 vs 4.27, P = 0.01) as well as in the adequacy of overnight supervision (3.65 vs 4.30, P < 0.0001; Table 1). There was no reported change in housestaff decision‐making autonomy (4.35 vs 4.45, P = 0.44). In addition, we noted a nonsignificant trend towards an increased perception of the night float rotation as a valuable educational experience (3.83 vs 4.04, P = 0.24). After implementation of the nocturnist, more resident physicians agreed that overnight supervision by an attending positively impacted patient outcomes (3.79 vs 4.30, P = 0.002).

After implementation of the nocturnist, night float providers demonstrated increased rates of contacting an attending physician overnight (Table 2). There were significantly greater rates of attending contact for transfers from outside facilities (2.00 vs 3.20, P = 0.006) and during times of adverse events (2.51 vs 3.25, P = 0.04). We observed a reported increase in attending contact prior to ordering invasive diagnostic procedures (1.75 vs 2.76, P = 0.004) and noninvasive diagnostic procedures (1.09 vs 1.31, P = 0.03), as well as prior to initiation of intravenous antibiotics (1.11 vs 1.47, P = 0.007) and vasopressors (1.52 vs 2.40, P = 0.004).

After initiating the program, the nocturnist became the most commonly contacted overnight provider by the night float housestaff (Table 3). We observed a decrease in peer to peer contact between the night float housestaff and the on‐call overnight resident after implementation of the nocturnist (2.67 vs 2.04, P = 0.006).

Attending presence led to increased agreement that there was a defined overnight attending to contact (2.97 vs 1.96, P < 0.0001) and a decreased fear of waking an attending overnight for assistance (3.26 vs 2.72, P = 0.03). Increased attending availability, however, did not change resident physician's fear of revealing knowledge gaps, their desire to make decisions independently, or their belief that contacting an attending would not change a patient's outcome (Table 4).

DISCUSSION

The ACGME's new duty hour regulations require that supervision for first‐year residents be provided by a qualified physician (advanced resident, fellow, or attending physician) who is physically present at the hospital. Our study demonstrates that increased direct overnight supervision provided by an in‐house nocturnist enhanced the clinical value of the night float rotation and the perceived quality of patient care. In our study, increased attending supervision did not reduce perceived decision‐making autonomy, and in fact led to increased rates of attending contact during times of critical clinical decision‐making. Such results may help assuage fears that recent regulations mandating enhanced attending supervision will produce less capable practitioners, and offers reassurance that such changes are positively impacting patient care.

Many academic institutions are implementing nocturnists, although their precise roles and responsibilities are still being defined. Our nocturnist program was explicitly designed with housestaff supervision as a core responsibility, with the goal of improving patient safety and housestaff education overnight. We found that availability barriers to attending contact were logically decreased with in‐house faculty presence. Potentially harmful attitudes, however, around requesting support (such as fear of revealing knowledge gaps or the desire to make decisions independently) remained. Furthermore, despite statistically significant increases in contact between faculty and residents at times of critical decision‐making, overall rates of attending contact for diagnostic and therapeutic interventions remained low. It is unknown from our study or previous research, however, what level of contact is appropriate or ideal for many clinical scenarios.

Additionally, we described a novel role of an academic nocturnist at a tertiary care teaching hospital and offered a potential template for the development of academic nocturnists at similar institutions seeking to increase direct overnight supervision. Such roles have not been previously well defined in the literature. Based on our experience, the nocturnist's role was manageable and well utilized by housestaff, particularly for assistance with critically ill patients and overnight triaging. We believe there are a number of factors associated with the success of this role. First, clear guidelines were presented to housestaff and nocturnists regarding expectations for supervision (for example, staffing ICU admissions within 2 hours). These guidelines likely contributed to the increased attending contact observed during critical clinical decision‐making, as well as the perceived improved patient outcomes by our housestaff. Second, the nocturnists were expected to be an integral part of the overnight care team. In many systems, the nocturnists act completely independently of the housestaff teams, creating an additional barrier to contact and communication. In our system, because of clear guidelines and their integral role in staffing overnight admissions, the nocturnists were an essential partner in care for the housestaff. Third, most of the nocturnists had recently completed their residency training at this institution. Although our survey does not directly address this, we believe their knowledge of the hospital, appreciation of the role of the intern and the resident within our system, and understanding of the need to preserve housestaff autonomy were essential to building a successful nocturnist role. Lastly, the nocturnists were not only expected to supervise and staff new admissions, but were also given a teaching expectation. We believe they were viewed by housestaff as qualified teaching attendings, similar to the daytime hospitalist. These findings may provide guidelines for other institutions seeking to balance overnight hospitalist supervision with preserving resident's ability to make autonomous decisions.

There are several limitations to our study. The findings represent the experience of internal medicine housestaff at a single academic, tertiary care medical center and may not be reflective of other institutions or specialties. We asked housestaff to recall night float experiences from the prior year, which may have introduced recall bias, though responses were obtained before participants underwent the new curriculum. Maturation of housestaff over time could have led to changes in perceived autonomy, value of the night float rotation, and rates of attending contact independent of nocturnist implementation. In addition, there may have been unaccounted changes to other elements of the residency program, hospital, or patient volume between rotations. The implementation of the nocturnist, however, was the only major change to our training program that academic year, and there were no significant changes in patient volume, structure of the teaching or non‐resident services, or other policies around resident supervision.

It is possible that the nocturnist may have contributed to reports of increased clinical value and perceived quality of patient care simply by decreasing overnight workload for housestaff, and enhanced supervision and teaching may have played a lesser role. Even if this were true, optimizing resident workload is in itself an important goal for teaching hospitals and residency programs alike in order to maximize patient safety. Inclusion of intern post‐rotation surveys may have influenced data; though, we had no reason to suspect the surveyed interns would respond in a different manner than prior resident groups. The responses of both junior and senior housestaff were pooled; while this potentially weighted the results in favor of higher responding groups, we felt that it conveyed the residents' accurate sentiments on the program. Finally, while we compared two models of overnight supervision, we reported only housestaff perceptions of education, autonomy, patient outcomes, and supervisory contact, and not direct measures of knowledge or patient care. Further research will be required to define the relationship between supervision practices and patient‐level clinical outcomes.

The new ACGME regulations around resident supervision, as well as the broader movement to improve the safety and quality of care, require residency programs to negotiate a delicate balance between providing high‐quality patient care while preserving graduated independence in clinical training. Our study demonstrates that increased overnight supervision by nocturnists with well‐defined supervisory and teaching roles can preserve housestaff autonomy, improve the clinical experience for trainees, increase access to support during times of critical decision‐making, and potentially lead to improved patient outcomes.

Recently, there has been an increased focus on improving communication during care transitions for older patients as they leave the hospital. One reason for this focus is the increasing utilization of hospitalists, or hospital‐based physicians, caring for patients in the United States.1 As a result, many primary care physicians (PCPs) no longer care for their patients while in the hospital and may not be informed of their patients' hospitalization.2 Additionally, with an emphasis on shorter lengths of hospital stay, more extensive post‐discharge follow‐up is often warranted for patients, which often becomes the responsibility of a patient's PCP. Recently 6 societies (American College of Physicians, Society of General Internal Medicine, Society of Hospital Medicine, American Geriatrics Society, American College of Emergency Physicians, and Society of Academic Emergency Medicine) have recommended that a patient's PCP is notified during all steps in care transitions and that patient‐centered approaches are employed.3 Despite the increased need for improved inpatient‐ambulatory care transitions, the communication between hospitalists and PCPs has been characterized as being poor and ineffective.4 Prior studies have shown that PCPs are not aware of test results that require follow‐up, may not receive timely or high quality discharge materials, and have an overall poor perception of the quality of communication.46 Ensuring adequate communication is considered important due to the increased risk of adverse events that patients experience after discharge from the hospital.79 Furthermore, recent studies have shown that patients are often able to identify and report adverse events that would not be detected by medical record review alone.10, 11 Eliciting patient perspectives on their experiences after discharge and their expectations of communication between PCPs and hospital physicians can help clinical teams design more patient‐centered solutions for care transitions.

The aim of this study is to report older patients' experiences with problems after hospital discharge and their understanding and expectation of communication between hospital physicians and their PCP. We also explored the relationship between patient experiences and whether their PCPs were aware of their hospitalization.

Methods

Results

Of the 114 eligible patients recruited between February and July 2008, 64 patient interviews were completed (56%). The average patient age was 73 years. Most patients were female (69%), African American (70%), live at home (75%), and have a PCP located at the University of Chicago (70%). There were also several who were low income (23% below a median yearly income of $15,000), and did not attend any college (52%). These patients had an average length of stay of 5.3 days, nearly half (48%) having been hospitalized in the past year, and 6 patients (9%) required a proxy to complete the interview (Table 1). There were no significant differences between responders and nonresponders with respect to race, gender, education, income, admission in the past 12 months, residence, PCP location, mental status, length of stay, or discharge status. Responders were more likely to be older than nonresponders (73 years [95% confidence interval {CI} 6976 years] vs. 63 years for nonresponders [95% CI 5769 years]; [P < 0.01]).

Forty‐two percent (27) of patients reported experiencing a post‐discharge problem. These 27 patients reported 42 distinct problems, each of which fell into 1 of 5 broad categories (Table 2). The most common of these were patients having difficulty obtaining follow‐up tests or appointments. These patients either had delay in getting, or were unable to get, follow‐up appointments, or follow‐up tests and test results. There were also many patients who needed reevaluation and thus, were either readmitted to the hospital or had to return to the Emergency Department. Another major category was those who had problems getting medication or therapy. For example, one of (the patients) treatment medswas very hard to find and it delayed us giving her her meds. Others reported they were not properly prepared for discharge. Most of these patients did not receive proper discharge materials which then caused other issues. As one proxy reported, The services were supposed to be provided for (the patient) through her social worker, no one has been informed to her being discharged or her being sent home. We have not gotten any services. Lastly, a few patients reported having hospital complications, such as post‐procedural complications, or questions, such as diagnosis questions.0

Patients were often uncertain of whether and how communication between the inpatient physician and PCP (Table 3) took place. One patient said, I don't know what the procedure is as far as giving him the message. Does she fax it to him? I don't know She told me that she was going to call and inform him on everything that happened. I don't know anything from there. The second most commonly expressed perception was from patients who assumed good communication had taken place between his or her physicians. This assumption was grounded in a belief that good communication naturally occurred between physicians. For example 1 patient expressed: (doctors) let the other doctors in too. That's the way to take care of stuff. Lastly, many patients expressed the feeling that their physicians were obligated to communicate with each other. As 1 patient reported, I think that they should have let (my PCP) know that I was in the hospital.

Two new themes emerged from the inductive analysis (Table 4). Forty‐five percent of patients reported experiencing negative emotions. These negative emotions were most often expressed as frustration or confusion. For example, 1 patient expressed confusion by saying, When I usually have lab work done I have prescription signedmaybe they changed the way of doing it. Now the pharmacy called me. But I'm supposed to have a note or something. Patients who reported a post‐discharge problem were more likely to report negative emotions (67% vs. 26%, P < 0.01). Feelings of empowerment were reported by 31% of patients. Empowerment was expressed most often as the patient being proactive in communicating with the PCP. One patient reported, We informed (my PCP) and we filled in all of the information that we wanted him to know about. Empowerment was also expressed as being proactive in advocating for communication between the inpatient team and the PCP (Table 3). Some patients expressed feeling empowered through the support of a third party, such as a home nurse. In addition, patients who have a third party advocate are more likely to report being empowered. Empowerment was expressed by 26% of patients with no third party advocate compared with 71% of patients with a third party advocate (P = 0.02).

From our sample of patients who completed a 2‐week post‐discharge interview, we were able to obtain PCP surveys for 40 (63%) of these patients (Figure 1). Thirty percent (12) of PCPs reported being unaware of the hospitalization. In all but 4 cases, PCPs had communicated with the medical team during hospitalization. Examining the association between PCP knowledge and patient reported post‐discharge problems showed that patients whose PCPs were not aware of the hospitalization were 2 times more likely to report a post‐discharge problem. A post‐discharge problem was reported by 67% of patients whose PCP was not aware of the hospitalization, while a post‐discharge problem was reported by 32% of patients whose PCP was aware (P < 0.05). Six patients reported returning to the ED or being readmitted. Four patients (33%) of PCPs who were unaware of hospitalization reported returning for reevaluation whereas 7% (n = 2) of patients whose PCP was aware of hospitalization reported returning for evaluation (P = 0.055). Interestingly, patients whose PCPs were not aware of the hospitalization reported feeling more empowered (58%) than those patients whose PCP were aware of the hospitalization (21%, P = 0.03). Because of possible confounding (patient report of problems post‐discharge problems may be affected by PCP awareness of hospitalization), we examined whether patients whose PCPs were aware of their hospitalization differed from those that did not. Patients whose PCPs were aware of their hospitalization were often older (75 vs. 69 years old), white (80% white vs. 65% nonwhite) and female (75% female vs. 54% male). While this small sample size prohibits examining for statistical significance, the magnitude of these differences suggests the need for a larger study to examine patient predictors of PCP awareness of hospitalization.

Figure 1

Discussion

In this sample of frail, older hospitalized patients, nearly half reported at least 1 post‐discharge problem. Most patients have perceptions of what communication did or did not take place between their physicians. While most do not understand the communication process, many expect good communication to occur, and feel that physicians are obligated to communicate with each other. However, patients' perceptions of communication highlight that patient expectations are far from the actual practice in some cases. Nearly half of patients reported feeling negative emotions, such as confusion and frustration, and patients were more likely to experience negative emotions when they also reported a post‐discharge problem. One‐third of patients reported feeling empowered. Empowerment was associated with having a third party who helped advocate for them. Paradoxically, patients whose PCP were not aware of their hospitalization were more likely to feel empowered. Lastly, more patients reported a post‐discharge problem when their PCP was not aware of the hospitalization.

Because this is predominantly a qualitative observational study, it is important to consider the mechanism for these findings since we cannot assume causal relationships. The association of negative emotions, like confusion and frustration, with post‐discharge problems could be explained due to additional stress of the problem itself or that a distressed frame of mind is associated with reporting more problems that may have been overlooked otherwise. In addition, the association between patient empowerment and lack of PCP awareness could be due to the fact that patients are forced to assume a more proactive role in contacting their PCP if they feel that their PCP was not aware. It is equally possible that PCP communication is selectively initiated by hospital physicians when the patients are least empowered. For example, our comparison of demographics for patients whose PCP was aware versus those that were not do suggest that patient characteristics might play a role in whether a patient's PCP is contacted. The association between a third party advocate and patient empowerment is likely explained as the third party is able to keep the patient informed and empowered.

This study has implications for efforts to design a more patient‐centered care transition for hospitalized older patients. First, patients and their proxies should be advocates for good communication to avoid the risks of care transitions. Prior interventions such as use of coaches to boost patient empowerment have had positive results for hospitalized older patients. Moreover, hospitals should keep in mind that problems after discharge are common and are linked to negative emotions, which may lower patient satisfaction or increase liability risk. Similarly, these findings also highlight the importance of keeping PCPs aware of patient hospitalization. For example, PCPs that are aware of hospitalization are better prepared to properly follow‐up on medications, tests, and appointments. The PCP can also help to better prepare the patient for discharge and ease the transition for the patient.

There are several limitations to our study. First and foremost, our small sample size limits our ability to examine statistical significance. This study was part of a short planning grant to design interventions to improve communication with PCPs during hospitalization. Efforts are currently underway to design a communication solution and educational intervention to highlight the importance of contacting PCPs during hospitalization. Because these patients were hospitalized on the teaching service, the resident with the guidance of the teaching attending is responsible for communicating with the PCP. The teaching attending was either a generalist, hospitalist, or specialist who routinely had no a priori relationship with patients prior to the hospitalization. Only 53% of patients were reached by telephone which raises the concern for nonresponse bias. Our low response rate highlights the challenge of doing this type of work with recently discharge patients in low income, underserved areas. In comparing responders and nonresponders, the only difference between the 2 groups was that responders were more likely to be older. One possible reason for this difference may be that older people are more likely to be at home and easier to contact over the phone. Similarly, since data were collected through interviews and adverse events were discussed, these results are subject to recall bias. Efforts were made to reduce this by calling within 2 to 3 weeks after discharge. Lastly, these findings are limited by generalizability. All the patients included in this study were from the University of Chicago Medical Center, which serves largely underserved, African American patients. The experiences of these patients may be unique to this site. In addition, we only studied patients who had a PCP, excluding a population of patients that are at inherent risk due to lack of a coordinating physician to guide ongoing care.

In conclusion, this study suggests that many frail, older patients reported experiencing a post‐discharge problem and patients whose PCPs did not know about their admission were more likely to report a post‐discharge problem. Systematic interventions to improve communications with PCPs during patient care transitions in and out of the hospital are needed.

Recently, there has been an increased focus on improving communication during care transitions for older patients as they leave the hospital. One reason for this focus is the increasing utilization of hospitalists, or hospital‐based physicians, caring for patients in the United States.1 As a result, many primary care physicians (PCPs) no longer care for their patients while in the hospital and may not be informed of their patients' hospitalization.2 Additionally, with an emphasis on shorter lengths of hospital stay, more extensive post‐discharge follow‐up is often warranted for patients, which often becomes the responsibility of a patient's PCP. Recently 6 societies (American College of Physicians, Society of General Internal Medicine, Society of Hospital Medicine, American Geriatrics Society, American College of Emergency Physicians, and Society of Academic Emergency Medicine) have recommended that a patient's PCP is notified during all steps in care transitions and that patient‐centered approaches are employed.3 Despite the increased need for improved inpatient‐ambulatory care transitions, the communication between hospitalists and PCPs has been characterized as being poor and ineffective.4 Prior studies have shown that PCPs are not aware of test results that require follow‐up, may not receive timely or high quality discharge materials, and have an overall poor perception of the quality of communication.46 Ensuring adequate communication is considered important due to the increased risk of adverse events that patients experience after discharge from the hospital.79 Furthermore, recent studies have shown that patients are often able to identify and report adverse events that would not be detected by medical record review alone.10, 11 Eliciting patient perspectives on their experiences after discharge and their expectations of communication between PCPs and hospital physicians can help clinical teams design more patient‐centered solutions for care transitions.

The aim of this study is to report older patients' experiences with problems after hospital discharge and their understanding and expectation of communication between hospital physicians and their PCP. We also explored the relationship between patient experiences and whether their PCPs were aware of their hospitalization.

Methods

Results

Of the 114 eligible patients recruited between February and July 2008, 64 patient interviews were completed (56%). The average patient age was 73 years. Most patients were female (69%), African American (70%), live at home (75%), and have a PCP located at the University of Chicago (70%). There were also several who were low income (23% below a median yearly income of $15,000), and did not attend any college (52%). These patients had an average length of stay of 5.3 days, nearly half (48%) having been hospitalized in the past year, and 6 patients (9%) required a proxy to complete the interview (Table 1). There were no significant differences between responders and nonresponders with respect to race, gender, education, income, admission in the past 12 months, residence, PCP location, mental status, length of stay, or discharge status. Responders were more likely to be older than nonresponders (73 years [95% confidence interval {CI} 6976 years] vs. 63 years for nonresponders [95% CI 5769 years]; [P < 0.01]).

Forty‐two percent (27) of patients reported experiencing a post‐discharge problem. These 27 patients reported 42 distinct problems, each of which fell into 1 of 5 broad categories (Table 2). The most common of these were patients having difficulty obtaining follow‐up tests or appointments. These patients either had delay in getting, or were unable to get, follow‐up appointments, or follow‐up tests and test results. There were also many patients who needed reevaluation and thus, were either readmitted to the hospital or had to return to the Emergency Department. Another major category was those who had problems getting medication or therapy. For example, one of (the patients) treatment medswas very hard to find and it delayed us giving her her meds. Others reported they were not properly prepared for discharge. Most of these patients did not receive proper discharge materials which then caused other issues. As one proxy reported, The services were supposed to be provided for (the patient) through her social worker, no one has been informed to her being discharged or her being sent home. We have not gotten any services. Lastly, a few patients reported having hospital complications, such as post‐procedural complications, or questions, such as diagnosis questions.0

Patients were often uncertain of whether and how communication between the inpatient physician and PCP (Table 3) took place. One patient said, I don't know what the procedure is as far as giving him the message. Does she fax it to him? I don't know She told me that she was going to call and inform him on everything that happened. I don't know anything from there. The second most commonly expressed perception was from patients who assumed good communication had taken place between his or her physicians. This assumption was grounded in a belief that good communication naturally occurred between physicians. For example 1 patient expressed: (doctors) let the other doctors in too. That's the way to take care of stuff. Lastly, many patients expressed the feeling that their physicians were obligated to communicate with each other. As 1 patient reported, I think that they should have let (my PCP) know that I was in the hospital.

Two new themes emerged from the inductive analysis (Table 4). Forty‐five percent of patients reported experiencing negative emotions. These negative emotions were most often expressed as frustration or confusion. For example, 1 patient expressed confusion by saying, When I usually have lab work done I have prescription signedmaybe they changed the way of doing it. Now the pharmacy called me. But I'm supposed to have a note or something. Patients who reported a post‐discharge problem were more likely to report negative emotions (67% vs. 26%, P < 0.01). Feelings of empowerment were reported by 31% of patients. Empowerment was expressed most often as the patient being proactive in communicating with the PCP. One patient reported, We informed (my PCP) and we filled in all of the information that we wanted him to know about. Empowerment was also expressed as being proactive in advocating for communication between the inpatient team and the PCP (Table 3). Some patients expressed feeling empowered through the support of a third party, such as a home nurse. In addition, patients who have a third party advocate are more likely to report being empowered. Empowerment was expressed by 26% of patients with no third party advocate compared with 71% of patients with a third party advocate (P = 0.02).

From our sample of patients who completed a 2‐week post‐discharge interview, we were able to obtain PCP surveys for 40 (63%) of these patients (Figure 1). Thirty percent (12) of PCPs reported being unaware of the hospitalization. In all but 4 cases, PCPs had communicated with the medical team during hospitalization. Examining the association between PCP knowledge and patient reported post‐discharge problems showed that patients whose PCPs were not aware of the hospitalization were 2 times more likely to report a post‐discharge problem. A post‐discharge problem was reported by 67% of patients whose PCP was not aware of the hospitalization, while a post‐discharge problem was reported by 32% of patients whose PCP was aware (P < 0.05). Six patients reported returning to the ED or being readmitted. Four patients (33%) of PCPs who were unaware of hospitalization reported returning for reevaluation whereas 7% (n = 2) of patients whose PCP was aware of hospitalization reported returning for evaluation (P = 0.055). Interestingly, patients whose PCPs were not aware of the hospitalization reported feeling more empowered (58%) than those patients whose PCP were aware of the hospitalization (21%, P = 0.03). Because of possible confounding (patient report of problems post‐discharge problems may be affected by PCP awareness of hospitalization), we examined whether patients whose PCPs were aware of their hospitalization differed from those that did not. Patients whose PCPs were aware of their hospitalization were often older (75 vs. 69 years old), white (80% white vs. 65% nonwhite) and female (75% female vs. 54% male). While this small sample size prohibits examining for statistical significance, the magnitude of these differences suggests the need for a larger study to examine patient predictors of PCP awareness of hospitalization.

Figure 1

Discussion

In this sample of frail, older hospitalized patients, nearly half reported at least 1 post‐discharge problem. Most patients have perceptions of what communication did or did not take place between their physicians. While most do not understand the communication process, many expect good communication to occur, and feel that physicians are obligated to communicate with each other. However, patients' perceptions of communication highlight that patient expectations are far from the actual practice in some cases. Nearly half of patients reported feeling negative emotions, such as confusion and frustration, and patients were more likely to experience negative emotions when they also reported a post‐discharge problem. One‐third of patients reported feeling empowered. Empowerment was associated with having a third party who helped advocate for them. Paradoxically, patients whose PCP were not aware of their hospitalization were more likely to feel empowered. Lastly, more patients reported a post‐discharge problem when their PCP was not aware of the hospitalization.

Because this is predominantly a qualitative observational study, it is important to consider the mechanism for these findings since we cannot assume causal relationships. The association of negative emotions, like confusion and frustration, with post‐discharge problems could be explained due to additional stress of the problem itself or that a distressed frame of mind is associated with reporting more problems that may have been overlooked otherwise. In addition, the association between patient empowerment and lack of PCP awareness could be due to the fact that patients are forced to assume a more proactive role in contacting their PCP if they feel that their PCP was not aware. It is equally possible that PCP communication is selectively initiated by hospital physicians when the patients are least empowered. For example, our comparison of demographics for patients whose PCP was aware versus those that were not do suggest that patient characteristics might play a role in whether a patient's PCP is contacted. The association between a third party advocate and patient empowerment is likely explained as the third party is able to keep the patient informed and empowered.

This study has implications for efforts to design a more patient‐centered care transition for hospitalized older patients. First, patients and their proxies should be advocates for good communication to avoid the risks of care transitions. Prior interventions such as use of coaches to boost patient empowerment have had positive results for hospitalized older patients. Moreover, hospitals should keep in mind that problems after discharge are common and are linked to negative emotions, which may lower patient satisfaction or increase liability risk. Similarly, these findings also highlight the importance of keeping PCPs aware of patient hospitalization. For example, PCPs that are aware of hospitalization are better prepared to properly follow‐up on medications, tests, and appointments. The PCP can also help to better prepare the patient for discharge and ease the transition for the patient.

There are several limitations to our study. First and foremost, our small sample size limits our ability to examine statistical significance. This study was part of a short planning grant to design interventions to improve communication with PCPs during hospitalization. Efforts are currently underway to design a communication solution and educational intervention to highlight the importance of contacting PCPs during hospitalization. Because these patients were hospitalized on the teaching service, the resident with the guidance of the teaching attending is responsible for communicating with the PCP. The teaching attending was either a generalist, hospitalist, or specialist who routinely had no a priori relationship with patients prior to the hospitalization. Only 53% of patients were reached by telephone which raises the concern for nonresponse bias. Our low response rate highlights the challenge of doing this type of work with recently discharge patients in low income, underserved areas. In comparing responders and nonresponders, the only difference between the 2 groups was that responders were more likely to be older. One possible reason for this difference may be that older people are more likely to be at home and easier to contact over the phone. Similarly, since data were collected through interviews and adverse events were discussed, these results are subject to recall bias. Efforts were made to reduce this by calling within 2 to 3 weeks after discharge. Lastly, these findings are limited by generalizability. All the patients included in this study were from the University of Chicago Medical Center, which serves largely underserved, African American patients. The experiences of these patients may be unique to this site. In addition, we only studied patients who had a PCP, excluding a population of patients that are at inherent risk due to lack of a coordinating physician to guide ongoing care.

In conclusion, this study suggests that many frail, older patients reported experiencing a post‐discharge problem and patients whose PCPs did not know about their admission were more likely to report a post‐discharge problem. Systematic interventions to improve communications with PCPs during patient care transitions in and out of the hospital are needed.

Recently, there has been an increased focus on improving communication during care transitions for older patients as they leave the hospital. One reason for this focus is the increasing utilization of hospitalists, or hospital‐based physicians, caring for patients in the United States.1 As a result, many primary care physicians (PCPs) no longer care for their patients while in the hospital and may not be informed of their patients' hospitalization.2 Additionally, with an emphasis on shorter lengths of hospital stay, more extensive post‐discharge follow‐up is often warranted for patients, which often becomes the responsibility of a patient's PCP. Recently 6 societies (American College of Physicians, Society of General Internal Medicine, Society of Hospital Medicine, American Geriatrics Society, American College of Emergency Physicians, and Society of Academic Emergency Medicine) have recommended that a patient's PCP is notified during all steps in care transitions and that patient‐centered approaches are employed.3 Despite the increased need for improved inpatient‐ambulatory care transitions, the communication between hospitalists and PCPs has been characterized as being poor and ineffective.4 Prior studies have shown that PCPs are not aware of test results that require follow‐up, may not receive timely or high quality discharge materials, and have an overall poor perception of the quality of communication.46 Ensuring adequate communication is considered important due to the increased risk of adverse events that patients experience after discharge from the hospital.79 Furthermore, recent studies have shown that patients are often able to identify and report adverse events that would not be detected by medical record review alone.10, 11 Eliciting patient perspectives on their experiences after discharge and their expectations of communication between PCPs and hospital physicians can help clinical teams design more patient‐centered solutions for care transitions.

The aim of this study is to report older patients' experiences with problems after hospital discharge and their understanding and expectation of communication between hospital physicians and their PCP. We also explored the relationship between patient experiences and whether their PCPs were aware of their hospitalization.

Methods

Results

Of the 114 eligible patients recruited between February and July 2008, 64 patient interviews were completed (56%). The average patient age was 73 years. Most patients were female (69%), African American (70%), live at home (75%), and have a PCP located at the University of Chicago (70%). There were also several who were low income (23% below a median yearly income of $15,000), and did not attend any college (52%). These patients had an average length of stay of 5.3 days, nearly half (48%) having been hospitalized in the past year, and 6 patients (9%) required a proxy to complete the interview (Table 1). There were no significant differences between responders and nonresponders with respect to race, gender, education, income, admission in the past 12 months, residence, PCP location, mental status, length of stay, or discharge status. Responders were more likely to be older than nonresponders (73 years [95% confidence interval {CI} 6976 years] vs. 63 years for nonresponders [95% CI 5769 years]; [P < 0.01]).

Forty‐two percent (27) of patients reported experiencing a post‐discharge problem. These 27 patients reported 42 distinct problems, each of which fell into 1 of 5 broad categories (Table 2). The most common of these were patients having difficulty obtaining follow‐up tests or appointments. These patients either had delay in getting, or were unable to get, follow‐up appointments, or follow‐up tests and test results. There were also many patients who needed reevaluation and thus, were either readmitted to the hospital or had to return to the Emergency Department. Another major category was those who had problems getting medication or therapy. For example, one of (the patients) treatment medswas very hard to find and it delayed us giving her her meds. Others reported they were not properly prepared for discharge. Most of these patients did not receive proper discharge materials which then caused other issues. As one proxy reported, The services were supposed to be provided for (the patient) through her social worker, no one has been informed to her being discharged or her being sent home. We have not gotten any services. Lastly, a few patients reported having hospital complications, such as post‐procedural complications, or questions, such as diagnosis questions.0

Patients were often uncertain of whether and how communication between the inpatient physician and PCP (Table 3) took place. One patient said, I don't know what the procedure is as far as giving him the message. Does she fax it to him? I don't know She told me that she was going to call and inform him on everything that happened. I don't know anything from there. The second most commonly expressed perception was from patients who assumed good communication had taken place between his or her physicians. This assumption was grounded in a belief that good communication naturally occurred between physicians. For example 1 patient expressed: (doctors) let the other doctors in too. That's the way to take care of stuff. Lastly, many patients expressed the feeling that their physicians were obligated to communicate with each other. As 1 patient reported, I think that they should have let (my PCP) know that I was in the hospital.

Two new themes emerged from the inductive analysis (Table 4). Forty‐five percent of patients reported experiencing negative emotions. These negative emotions were most often expressed as frustration or confusion. For example, 1 patient expressed confusion by saying, When I usually have lab work done I have prescription signedmaybe they changed the way of doing it. Now the pharmacy called me. But I'm supposed to have a note or something. Patients who reported a post‐discharge problem were more likely to report negative emotions (67% vs. 26%, P < 0.01). Feelings of empowerment were reported by 31% of patients. Empowerment was expressed most often as the patient being proactive in communicating with the PCP. One patient reported, We informed (my PCP) and we filled in all of the information that we wanted him to know about. Empowerment was also expressed as being proactive in advocating for communication between the inpatient team and the PCP (Table 3). Some patients expressed feeling empowered through the support of a third party, such as a home nurse. In addition, patients who have a third party advocate are more likely to report being empowered. Empowerment was expressed by 26% of patients with no third party advocate compared with 71% of patients with a third party advocate (P = 0.02).

From our sample of patients who completed a 2‐week post‐discharge interview, we were able to obtain PCP surveys for 40 (63%) of these patients (Figure 1). Thirty percent (12) of PCPs reported being unaware of the hospitalization. In all but 4 cases, PCPs had communicated with the medical team during hospitalization. Examining the association between PCP knowledge and patient reported post‐discharge problems showed that patients whose PCPs were not aware of the hospitalization were 2 times more likely to report a post‐discharge problem. A post‐discharge problem was reported by 67% of patients whose PCP was not aware of the hospitalization, while a post‐discharge problem was reported by 32% of patients whose PCP was aware (P < 0.05). Six patients reported returning to the ED or being readmitted. Four patients (33%) of PCPs who were unaware of hospitalization reported returning for reevaluation whereas 7% (n = 2) of patients whose PCP was aware of hospitalization reported returning for evaluation (P = 0.055). Interestingly, patients whose PCPs were not aware of the hospitalization reported feeling more empowered (58%) than those patients whose PCP were aware of the hospitalization (21%, P = 0.03). Because of possible confounding (patient report of problems post‐discharge problems may be affected by PCP awareness of hospitalization), we examined whether patients whose PCPs were aware of their hospitalization differed from those that did not. Patients whose PCPs were aware of their hospitalization were often older (75 vs. 69 years old), white (80% white vs. 65% nonwhite) and female (75% female vs. 54% male). While this small sample size prohibits examining for statistical significance, the magnitude of these differences suggests the need for a larger study to examine patient predictors of PCP awareness of hospitalization.

Figure 1

Discussion

In this sample of frail, older hospitalized patients, nearly half reported at least 1 post‐discharge problem. Most patients have perceptions of what communication did or did not take place between their physicians. While most do not understand the communication process, many expect good communication to occur, and feel that physicians are obligated to communicate with each other. However, patients' perceptions of communication highlight that patient expectations are far from the actual practice in some cases. Nearly half of patients reported feeling negative emotions, such as confusion and frustration, and patients were more likely to experience negative emotions when they also reported a post‐discharge problem. One‐third of patients reported feeling empowered. Empowerment was associated with having a third party who helped advocate for them. Paradoxically, patients whose PCP were not aware of their hospitalization were more likely to feel empowered. Lastly, more patients reported a post‐discharge problem when their PCP was not aware of the hospitalization.

Because this is predominantly a qualitative observational study, it is important to consider the mechanism for these findings since we cannot assume causal relationships. The association of negative emotions, like confusion and frustration, with post‐discharge problems could be explained due to additional stress of the problem itself or that a distressed frame of mind is associated with reporting more problems that may have been overlooked otherwise. In addition, the association between patient empowerment and lack of PCP awareness could be due to the fact that patients are forced to assume a more proactive role in contacting their PCP if they feel that their PCP was not aware. It is equally possible that PCP communication is selectively initiated by hospital physicians when the patients are least empowered. For example, our comparison of demographics for patients whose PCP was aware versus those that were not do suggest that patient characteristics might play a role in whether a patient's PCP is contacted. The association between a third party advocate and patient empowerment is likely explained as the third party is able to keep the patient informed and empowered.

This study has implications for efforts to design a more patient‐centered care transition for hospitalized older patients. First, patients and their proxies should be advocates for good communication to avoid the risks of care transitions. Prior interventions such as use of coaches to boost patient empowerment have had positive results for hospitalized older patients. Moreover, hospitals should keep in mind that problems after discharge are common and are linked to negative emotions, which may lower patient satisfaction or increase liability risk. Similarly, these findings also highlight the importance of keeping PCPs aware of patient hospitalization. For example, PCPs that are aware of hospitalization are better prepared to properly follow‐up on medications, tests, and appointments. The PCP can also help to better prepare the patient for discharge and ease the transition for the patient.

There are several limitations to our study. First and foremost, our small sample size limits our ability to examine statistical significance. This study was part of a short planning grant to design interventions to improve communication with PCPs during hospitalization. Efforts are currently underway to design a communication solution and educational intervention to highlight the importance of contacting PCPs during hospitalization. Because these patients were hospitalized on the teaching service, the resident with the guidance of the teaching attending is responsible for communicating with the PCP. The teaching attending was either a generalist, hospitalist, or specialist who routinely had no a priori relationship with patients prior to the hospitalization. Only 53% of patients were reached by telephone which raises the concern for nonresponse bias. Our low response rate highlights the challenge of doing this type of work with recently discharge patients in low income, underserved areas. In comparing responders and nonresponders, the only difference between the 2 groups was that responders were more likely to be older. One possible reason for this difference may be that older people are more likely to be at home and easier to contact over the phone. Similarly, since data were collected through interviews and adverse events were discussed, these results are subject to recall bias. Efforts were made to reduce this by calling within 2 to 3 weeks after discharge. Lastly, these findings are limited by generalizability. All the patients included in this study were from the University of Chicago Medical Center, which serves largely underserved, African American patients. The experiences of these patients may be unique to this site. In addition, we only studied patients who had a PCP, excluding a population of patients that are at inherent risk due to lack of a coordinating physician to guide ongoing care.

In conclusion, this study suggests that many frail, older patients reported experiencing a post‐discharge problem and patients whose PCPs did not know about their admission were more likely to report a post‐discharge problem. Systematic interventions to improve communications with PCPs during patient care transitions in and out of the hospital are needed.

A growing number of reports indicate that communication failures among physicians at transitions of care are critical to patient safety.16 The practice of physician handoffs at shift and service changes are variable, with no standardized protocol shown to be effective at ensuring complete transmission of information.7 In 2006, the Joint Commission set a National Patient Safety Goal to implement a standardized approach to hand off communications.8 Hospitalists stand to be impacted by this decision due to the frequency of care transitions that are inherent in hospital practice. The Society of Hospital Medicine (SHM) recognizes safe transitions of care as a core competency of hospitalists and is actively exploring standardization of the process.9 While recent attention has focused on improved communication during shift changes, little data exists to guide handoffs among hospitalists at service changes.

Good service change communication is an essential skill of hospital medicine because frequent service handoffs are often unavoidable in hospitalist practices that seek to balance the demand for around‐the‐clock coverage for inpatients and the need to create sustainable schedules to avoid physician burnout.10 But the tradeoff between fewer hours worked and discontinuity of care is well recognized.7 Increasingly fragmented care without corresponding improvements in handoff communication may exacerbate the problem. This study aims to characterize communication practices among hospitalists during service changes and to describe adverse and near miss events that may occur as a result of poor handoffs during these vulnerable care transitions.

Methods

Results

Service Change Communication

Fifty‐six of 60 (93%) surveys evaluating service changes were completed and returned. All (17) eligible hospitalists participated. All but 1 completed survey indicated that some form of handoff communication took place between the incoming and the outgoing hospitalists. The median time category spent on service change communications was 6 to 15 minutes. Forty‐eight of 55 (87%) respondents who participated in handoff communication reported communicating on the day prior to the transition day, while the remainder communicated 2 or 3 days prior to, or on the transition day. Most communicated verbally, either by telephone (75%) or face to face (16%); 10% of respondents who did not speak with the outgoing physician received e‐mail as the main method of communication. The distribution of time spent on the service change communication is summarized in Figure 1A.

Figure 1

Discussion

To our knowledge, this is the first study of service changes among hospitalists. The results suggest that hospitalists in an academic medical center face obstacles to effective communication during service changes. A significant number of handoffs were described by hospitalists as incomplete and that missing information were associated with negative outcomes at the patient level. Reports of incomplete handoffs were associated with uncertainty by incoming physicians about the plan of care for patients and with the need to spend more time dealing with issues arising from this uncertainty. Although most of the effects on patients were near‐misses and not adverse events, the details elicited in this study reveal the threats to patient safety that arise from ineffective communication.

Interestingly, verbal communication was not associated with better transmission of information in this study. One reason for this may be the almost universal use of verbal communication in the service change handoffs among hospitalists at the UCMC. The value of verbal communication is supported by other studies that suggest the benefits of verbal exchanges combined with typed information sheets.15

In our study, hospitalists spent a significant amount of time resolving issues that arose from incomplete communication at service change. The need to retrieve missing information from charts and electronic medical records is to be expected, even if the handoffs were complete, but the use of patients and their family as redundant sources of information may lead to delay and stakeholder dissatisfaction. Likewise, consulting physicians were sometimes frustrated by not having their recommendations passed on during hospitalist service changes and of being asked to repeat their recommendations to each new incoming hospitalist. Moreover, many patients and consulting physicians were not informed about upcoming service changes by hospitalists. Informing stakeholders of staffing changes may be an important component of handoffs that requires attention.4, 16

The frequent communication between the outgoing and incoming hospitalists, even after their original handoff communication, points to the possible benefit of an overlap period during which outgoing physicians remain available to fill gaps in information. The willingness of outgoing hospitalists in this study to initiate this interaction reveals an opportunity for an intervention and is contrary to existing concerns that hospitalists, as opposed to primary care physicians, absolve themselves of patient responsibilities when their shift is completed.17, 18

Ensuring that handoff communication is concise and systematic is essential to improving the quality of care provided by hospitalists. An all‐inclusive transmission of unprocessed information, no matter how detailed, does not improve completeness of communication. Instead, we find that the complete transmission of patient information consists of both the discussion of the salient factual information about the case and the outgoing physician's assessment and future plan. A new strategy to improve completeness of service change communication may involve the use of a checklist to ensure a comprehensive review of critical details, as well as the use of narratives to tie together a coherent plan.

Alternative cutpoints for the dichotomized Likert categories for uncertainty about the plan and completeness of the handoff were explored. For example, it is also reasonable to interpret the response somewhat certain, referring to the plan of care on the transition day, as belonging to the dichotomized category uncertain as opposed to certain. A broader definition of uncertain increased the number of responses in that category but the variable's associations with other item responses were not significantly different from the results presented. We chose the symmetrical dichotomization cutpoint to ensure similar number of answers in each category.

There are several limitations with this study. First, the study was limited to self‐reported data without confirmation by direct observation. Additionally, responses to survey questions that ask participants to recollect details of a past handoff communication are subject to recall bias. We tried to minimize this bias effect by adhering to a schedule that surveyed hospitalists almost exactly at 48 hours into their rotation. However, there may still be hindsight bias about the respondents' perceived completeness of the handoffs based on the events of those 48 hours.19 In addition, a service of difficult patients requiring more of the hospitalist's time could influence his or her perception of a poor handoff through reverse causation. The study is not immune to a Hawthorne effect during the 8 months study period.20 This was a single‐center study examining 1 clinical service. The small sample size limits the depth of our analysis, but this is the first work to describe this phenomenon and although the data is not definitive, it may stimulate further work in the area. Although our study focused on completeness as the sole measure of handoff adequacy, additional measures may be explored in future studies. Finally, our findings may not be generalizable because of the unique features of the UCMC's hospitalist program, such as the specialized patient population. An examination of other practice settings is indicated for future studies.

Significant variability exists in the methods used to conduct service changes. Although a previous qualitative study of handoffs at our institution yielded a theme of poor communication around specific individuals,21 consistently poor communicators did not emerge as a theme in this qualitative analysis. We believe that episodes of incomplete communication are not always attributable to individual deficits and suggest that solutions to the communication problem exist at the systems level. The development and implementation of future interventions to improve hospitalist service changes may incorporate some of the elements suggested here.

A growing number of reports indicate that communication failures among physicians at transitions of care are critical to patient safety.16 The practice of physician handoffs at shift and service changes are variable, with no standardized protocol shown to be effective at ensuring complete transmission of information.7 In 2006, the Joint Commission set a National Patient Safety Goal to implement a standardized approach to hand off communications.8 Hospitalists stand to be impacted by this decision due to the frequency of care transitions that are inherent in hospital practice. The Society of Hospital Medicine (SHM) recognizes safe transitions of care as a core competency of hospitalists and is actively exploring standardization of the process.9 While recent attention has focused on improved communication during shift changes, little data exists to guide handoffs among hospitalists at service changes.

Good service change communication is an essential skill of hospital medicine because frequent service handoffs are often unavoidable in hospitalist practices that seek to balance the demand for around‐the‐clock coverage for inpatients and the need to create sustainable schedules to avoid physician burnout.10 But the tradeoff between fewer hours worked and discontinuity of care is well recognized.7 Increasingly fragmented care without corresponding improvements in handoff communication may exacerbate the problem. This study aims to characterize communication practices among hospitalists during service changes and to describe adverse and near miss events that may occur as a result of poor handoffs during these vulnerable care transitions.

Methods

Results

Service Change Communication

Fifty‐six of 60 (93%) surveys evaluating service changes were completed and returned. All (17) eligible hospitalists participated. All but 1 completed survey indicated that some form of handoff communication took place between the incoming and the outgoing hospitalists. The median time category spent on service change communications was 6 to 15 minutes. Forty‐eight of 55 (87%) respondents who participated in handoff communication reported communicating on the day prior to the transition day, while the remainder communicated 2 or 3 days prior to, or on the transition day. Most communicated verbally, either by telephone (75%) or face to face (16%); 10% of respondents who did not speak with the outgoing physician received e‐mail as the main method of communication. The distribution of time spent on the service change communication is summarized in Figure 1A.

Figure 1

Discussion

To our knowledge, this is the first study of service changes among hospitalists. The results suggest that hospitalists in an academic medical center face obstacles to effective communication during service changes. A significant number of handoffs were described by hospitalists as incomplete and that missing information were associated with negative outcomes at the patient level. Reports of incomplete handoffs were associated with uncertainty by incoming physicians about the plan of care for patients and with the need to spend more time dealing with issues arising from this uncertainty. Although most of the effects on patients were near‐misses and not adverse events, the details elicited in this study reveal the threats to patient safety that arise from ineffective communication.

Interestingly, verbal communication was not associated with better transmission of information in this study. One reason for this may be the almost universal use of verbal communication in the service change handoffs among hospitalists at the UCMC. The value of verbal communication is supported by other studies that suggest the benefits of verbal exchanges combined with typed information sheets.15

In our study, hospitalists spent a significant amount of time resolving issues that arose from incomplete communication at service change. The need to retrieve missing information from charts and electronic medical records is to be expected, even if the handoffs were complete, but the use of patients and their family as redundant sources of information may lead to delay and stakeholder dissatisfaction. Likewise, consulting physicians were sometimes frustrated by not having their recommendations passed on during hospitalist service changes and of being asked to repeat their recommendations to each new incoming hospitalist. Moreover, many patients and consulting physicians were not informed about upcoming service changes by hospitalists. Informing stakeholders of staffing changes may be an important component of handoffs that requires attention.4, 16

The frequent communication between the outgoing and incoming hospitalists, even after their original handoff communication, points to the possible benefit of an overlap period during which outgoing physicians remain available to fill gaps in information. The willingness of outgoing hospitalists in this study to initiate this interaction reveals an opportunity for an intervention and is contrary to existing concerns that hospitalists, as opposed to primary care physicians, absolve themselves of patient responsibilities when their shift is completed.17, 18

Ensuring that handoff communication is concise and systematic is essential to improving the quality of care provided by hospitalists. An all‐inclusive transmission of unprocessed information, no matter how detailed, does not improve completeness of communication. Instead, we find that the complete transmission of patient information consists of both the discussion of the salient factual information about the case and the outgoing physician's assessment and future plan. A new strategy to improve completeness of service change communication may involve the use of a checklist to ensure a comprehensive review of critical details, as well as the use of narratives to tie together a coherent plan.

Alternative cutpoints for the dichotomized Likert categories for uncertainty about the plan and completeness of the handoff were explored. For example, it is also reasonable to interpret the response somewhat certain, referring to the plan of care on the transition day, as belonging to the dichotomized category uncertain as opposed to certain. A broader definition of uncertain increased the number of responses in that category but the variable's associations with other item responses were not significantly different from the results presented. We chose the symmetrical dichotomization cutpoint to ensure similar number of answers in each category.

There are several limitations with this study. First, the study was limited to self‐reported data without confirmation by direct observation. Additionally, responses to survey questions that ask participants to recollect details of a past handoff communication are subject to recall bias. We tried to minimize this bias effect by adhering to a schedule that surveyed hospitalists almost exactly at 48 hours into their rotation. However, there may still be hindsight bias about the respondents' perceived completeness of the handoffs based on the events of those 48 hours.19 In addition, a service of difficult patients requiring more of the hospitalist's time could influence his or her perception of a poor handoff through reverse causation. The study is not immune to a Hawthorne effect during the 8 months study period.20 This was a single‐center study examining 1 clinical service. The small sample size limits the depth of our analysis, but this is the first work to describe this phenomenon and although the data is not definitive, it may stimulate further work in the area. Although our study focused on completeness as the sole measure of handoff adequacy, additional measures may be explored in future studies. Finally, our findings may not be generalizable because of the unique features of the UCMC's hospitalist program, such as the specialized patient population. An examination of other practice settings is indicated for future studies.

Significant variability exists in the methods used to conduct service changes. Although a previous qualitative study of handoffs at our institution yielded a theme of poor communication around specific individuals,21 consistently poor communicators did not emerge as a theme in this qualitative analysis. We believe that episodes of incomplete communication are not always attributable to individual deficits and suggest that solutions to the communication problem exist at the systems level. The development and implementation of future interventions to improve hospitalist service changes may incorporate some of the elements suggested here.

A growing number of reports indicate that communication failures among physicians at transitions of care are critical to patient safety.16 The practice of physician handoffs at shift and service changes are variable, with no standardized protocol shown to be effective at ensuring complete transmission of information.7 In 2006, the Joint Commission set a National Patient Safety Goal to implement a standardized approach to hand off communications.8 Hospitalists stand to be impacted by this decision due to the frequency of care transitions that are inherent in hospital practice. The Society of Hospital Medicine (SHM) recognizes safe transitions of care as a core competency of hospitalists and is actively exploring standardization of the process.9 While recent attention has focused on improved communication during shift changes, little data exists to guide handoffs among hospitalists at service changes.

Good service change communication is an essential skill of hospital medicine because frequent service handoffs are often unavoidable in hospitalist practices that seek to balance the demand for around‐the‐clock coverage for inpatients and the need to create sustainable schedules to avoid physician burnout.10 But the tradeoff between fewer hours worked and discontinuity of care is well recognized.7 Increasingly fragmented care without corresponding improvements in handoff communication may exacerbate the problem. This study aims to characterize communication practices among hospitalists during service changes and to describe adverse and near miss events that may occur as a result of poor handoffs during these vulnerable care transitions.

Methods

Results

Service Change Communication

Fifty‐six of 60 (93%) surveys evaluating service changes were completed and returned. All (17) eligible hospitalists participated. All but 1 completed survey indicated that some form of handoff communication took place between the incoming and the outgoing hospitalists. The median time category spent on service change communications was 6 to 15 minutes. Forty‐eight of 55 (87%) respondents who participated in handoff communication reported communicating on the day prior to the transition day, while the remainder communicated 2 or 3 days prior to, or on the transition day. Most communicated verbally, either by telephone (75%) or face to face (16%); 10% of respondents who did not speak with the outgoing physician received e‐mail as the main method of communication. The distribution of time spent on the service change communication is summarized in Figure 1A.

Figure 1

Discussion

To our knowledge, this is the first study of service changes among hospitalists. The results suggest that hospitalists in an academic medical center face obstacles to effective communication during service changes. A significant number of handoffs were described by hospitalists as incomplete and that missing information were associated with negative outcomes at the patient level. Reports of incomplete handoffs were associated with uncertainty by incoming physicians about the plan of care for patients and with the need to spend more time dealing with issues arising from this uncertainty. Although most of the effects on patients were near‐misses and not adverse events, the details elicited in this study reveal the threats to patient safety that arise from ineffective communication.

Interestingly, verbal communication was not associated with better transmission of information in this study. One reason for this may be the almost universal use of verbal communication in the service change handoffs among hospitalists at the UCMC. The value of verbal communication is supported by other studies that suggest the benefits of verbal exchanges combined with typed information sheets.15

In our study, hospitalists spent a significant amount of time resolving issues that arose from incomplete communication at service change. The need to retrieve missing information from charts and electronic medical records is to be expected, even if the handoffs were complete, but the use of patients and their family as redundant sources of information may lead to delay and stakeholder dissatisfaction. Likewise, consulting physicians were sometimes frustrated by not having their recommendations passed on during hospitalist service changes and of being asked to repeat their recommendations to each new incoming hospitalist. Moreover, many patients and consulting physicians were not informed about upcoming service changes by hospitalists. Informing stakeholders of staffing changes may be an important component of handoffs that requires attention.4, 16

The frequent communication between the outgoing and incoming hospitalists, even after their original handoff communication, points to the possible benefit of an overlap period during which outgoing physicians remain available to fill gaps in information. The willingness of outgoing hospitalists in this study to initiate this interaction reveals an opportunity for an intervention and is contrary to existing concerns that hospitalists, as opposed to primary care physicians, absolve themselves of patient responsibilities when their shift is completed.17, 18

Ensuring that handoff communication is concise and systematic is essential to improving the quality of care provided by hospitalists. An all‐inclusive transmission of unprocessed information, no matter how detailed, does not improve completeness of communication. Instead, we find that the complete transmission of patient information consists of both the discussion of the salient factual information about the case and the outgoing physician's assessment and future plan. A new strategy to improve completeness of service change communication may involve the use of a checklist to ensure a comprehensive review of critical details, as well as the use of narratives to tie together a coherent plan.

Alternative cutpoints for the dichotomized Likert categories for uncertainty about the plan and completeness of the handoff were explored. For example, it is also reasonable to interpret the response somewhat certain, referring to the plan of care on the transition day, as belonging to the dichotomized category uncertain as opposed to certain. A broader definition of uncertain increased the number of responses in that category but the variable's associations with other item responses were not significantly different from the results presented. We chose the symmetrical dichotomization cutpoint to ensure similar number of answers in each category.

There are several limitations with this study. First, the study was limited to self‐reported data without confirmation by direct observation. Additionally, responses to survey questions that ask participants to recollect details of a past handoff communication are subject to recall bias. We tried to minimize this bias effect by adhering to a schedule that surveyed hospitalists almost exactly at 48 hours into their rotation. However, there may still be hindsight bias about the respondents' perceived completeness of the handoffs based on the events of those 48 hours.19 In addition, a service of difficult patients requiring more of the hospitalist's time could influence his or her perception of a poor handoff through reverse causation. The study is not immune to a Hawthorne effect during the 8 months study period.20 This was a single‐center study examining 1 clinical service. The small sample size limits the depth of our analysis, but this is the first work to describe this phenomenon and although the data is not definitive, it may stimulate further work in the area. Although our study focused on completeness as the sole measure of handoff adequacy, additional measures may be explored in future studies. Finally, our findings may not be generalizable because of the unique features of the UCMC's hospitalist program, such as the specialized patient population. An examination of other practice settings is indicated for future studies.

Significant variability exists in the methods used to conduct service changes. Although a previous qualitative study of handoffs at our institution yielded a theme of poor communication around specific individuals,21 consistently poor communicators did not emerge as a theme in this qualitative analysis. We believe that episodes of incomplete communication are not always attributable to individual deficits and suggest that solutions to the communication problem exist at the systems level. The development and implementation of future interventions to improve hospitalist service changes may incorporate some of the elements suggested here.

Handoffs during hospitalization from one provider to another represent critical transition points in patient care.1 In‐hospital handoffs are a frequent occurrence, with 1 teaching hospital reporting 4000 handoffs daily for a total of 1.6 million per year.2

Incomplete or poor‐quality handoffs have been implicated as a source of adverse events and near misses in hospitalized patients.35 Standardizing the handoff process may improve patient safety during care transitions.6 In 2006, the Joint Commission issued a National Patient Safety Goal that requires care providers to adopt a standardized approach for handoff communications, including an opportunity to ask and respond to questions about a patient's care.7 The reductions in resident work hours by the Accreditation Council for Graduate Medical Education (ACGME) has also resulted in a greater number and greater scrutiny of handoffs in teaching hospitals.8, 9

In response to these issues, and because handoffs are a core competency for hospitalists, the Society of Hospital Medicine (SHM)convened a task force.10 Our goal was to develop a set of recommendations for handoffs that would be applicable in both community and academic settings; among physicians (hospitalists, internists, subspecialists, residents), nurse practitioners, and physicians assistants; and across roles including serving as the primary provider of hospital care, comanager, or consultant. This work focuses on handoffs that occur at shift change and service change.11 Shift changes are transitions of care between an outgoing provider and an incoming provider that occur at the end of the outgoing provider's continuous on‐duty period. Service changesa special type of shift changeare transitions of care between an outgoing provider and an incoming provider that occur when an outgoing provider is leaving a rotation or period of consecutive daily care for patients on the same service.

For this initiative, transfers of care in which the patient is moving from one patient area to another (eg, Emergency Department to inpatient floor, or floor to intensive care unit [ICU]) were excluded since they likely require unique consideration given their cross‐disciplinary and multispecialty nature. Likewise, transitions of care at hospital admission and discharge were also excluded because recommendations for discharge are already summarized in 2 complementary reports.12, 13

To develop recommendations for handoffs at routine shift change and service changes, the Handoff Task Force performed a systematic review of the literature to develop initial recommendations, obtained feedback from hospital‐based clinicians in addition to a panel of handoff experts, and finalized handoff recommendations, as well as a proposed research agenda, for the SHM.

Methods

The SHM Healthcare Quality and Patient Safety (HQPS) Committee convened the Handoff Task Force, which was comprised of 6 geographically diverse, predominantly academic hospitalists with backgrounds in education, patient safety, health communication, evidence‐based medicine, and handoffs. The Task Force then engaged a panel of 4 content experts selected for their work on handoffs in the fields of nursing, information technology, human factors engineering, and hospital medicine. Similar to clinical guideline development by professional societies, the Task Force used a combination of evidence‐based review and expert opinions to propose recommendations.

Results

Literature Review

Of the 374 articles identified by the electronic search of PubMed and the AHRQ Patient Safety Network, 109 were retrieved for detailed review, and 10 of these met the criteria for inclusion (Figure 1). Of these studies, 3 were derived from nursing literature and the remaining were tests of technology solutions or structured templates (Table 1).1618, 20, 22, 3842 No studies examined hospitalist handoffs. All eligible studies concerned shift change. There were no studies of service change. Only 1 study was a randomized controlled trial; the rest were pre‐post studies with historical controls or a controlled simulation. All reports were single‐site studies. Most outcomes were staff‐related or system‐related, with only 2 studies using patient outcomes.

Table 1.Characteristics of Studies Included in Review

Author (Year)	Study Design	Intervention	Setting and Study Population	Target	Outcomes
Abbreviations: IM, internal medicine; IS, ; UW, University of Washington.
Nursing
Kelly22 (2005)	Pre‐post	Change to walk‐round handover (at bedside) from baseline (control)	12‐bed rehab unit with 18 nurses and 10 patients	Staff, patient	11/18 nurses felt more or much more informed and involved; 8/10 patients felt more involved
Pothier et al.20 (2005)	Controlled simulation	Compared pure verbal to verbal with note‐taking to verbal plus typed content	Handover of 12 simulated patients over 5 cycles	System (data loss)	Minimal data loss with typed content, compared to 31% data retained with note‐taking, and no data retained with verbal only
Wallum38 (1995)	Pre‐post	Change from oral handover (baseline) to written template read with exchange	20 nurses in a geriatric dementia ward	Staff	83% of nurses felt care plans followed better; 88% knew care plans better
Technology or structured template
Cheah et al.39 (2005)	Pre‐post	Electronic template with free‐text entry compared to baseline	14 UK Surgery residents	Staff	100% (14) of residents rated electronic system as desirable, but 7 (50%) reported that information was not updated
Lee et al.40 (1996)	Pre‐post	Standardized signout card for interns to transmit information during handoffs compared to handwritten (baseline)	Inpatient cardiology service at IM residency program in Minnesota with 19 new interns over a 3‐month period	Staff	Intervention interns (n = 10) reported poor sign‐out less often than controls (n = 9) [intervention 8 nights (5.8%) vs. control 17 nights (14.9%); P = 0.016]
Kannry and Moore18 (1999)	Pre‐post	Compared web‐based signout program to usual system (baseline)	An academic teaching hospital in New York (34 patients admitted in 1997; 40 patients admitted in 1998)	System	Improved provider identification (86% web signout vs. 57% hospital census)
Petersen et al.17 (1998)	Pre‐post	4 months of computerized signouts compared to baseline period (control)	3747 patients admitted to the medical service at an academic teaching hospital	Patient	Preventable adverse events (ADE) decreased (1.7% to 1.2%, P < 0.10); risk of cross‐cover physician for ADE eliminated
Ram and Block41 (1993)	Pre‐post	Compared handwritten (baseline) to computer‐generated	Family medicine residents at 2 academic teaching hospitals [Buffalo (n = 16) and Pittsburgh (n = 16)]	Staff	Higher satisfaction after electronic signout, but complaints with burden of data entry and need to keep information updated
Van Eaton et al.42 (2004)	Pre‐post	Use of UW Cores links sign‐out to list for rounds and IS data	28 surgical and medical residents at 2 teaching hospitals	System	At 6 months, 66% of patients entered in system (adoption)
Van Eaton et al.16 (2005)	Prospective, randomized, crossover study.	Compared UW Cores* integrated system compared to usual system	14 inpatient resident teams (6 surgery, 8 IM) at 2 teaching hospitals for 5 months	Staff, system	50% reduction in the perceived time spent copying data [from 24% to 12% (P < 0.0001)] and number of patients missed on rounds (2.5 vs. 5 patients/team/month, P = 0.0001); improved signout quality (69.6% agree or strongly agree); and improved continuity of care (66.1% agree or strongly agree)

Figure 1

Overall, the literature presented supports the use of a verbal handoff supplemented with written documentation in a structured format or technology solution. The 2 most rigorous studies were led by Van Eaton et al.16 and Petersen et al.17 and focused on evaluating technology solutions. Van Eaton et al.16 performed a randomized controlled trial of a locally created rounding template with 161 surgical residents. This template downloads certain information (lab values and recent vital signs) from the hospital system into a sign‐out sheet and allows residents to enter notes about diagnoses, allergies, medications and to‐do items. When implemented, the investigators found the number of patients missed on rounds decreased by 50%. Residents reported an increase of 40% in the amount of time available to pre‐round, due largely to not having to copy data such as vital signs. They reported a decrease in rounding time by 3 hours per week, and this was perceived as helping them meet the ACGME 80 hours work rules. Lastly, the residents reported a higher quality of sign‐outs from their peers and perceived an overall improvement in continuity of care. Petersen and colleagues implemented a computerized sign‐out (auto‐imported medications, name, room number) in an internal medicine residency to improve continuity of care during cross‐coverage and decrease adverse events.17 Prior to the intervention, the frequency of preventable adverse events was 1.7% and it was significantly associated with cross‐coverage. Preventable adverse events were identified using a confidential self‐report system that was also validated by clinician review. After the intervention, the frequency of preventable adverse events dropped to 1.2% (P < 0.1), and cross‐coverage was no longer associated with preventable adverse events. In other studies, technological solutions also improved provider identification and staff communication.18, 19 Together, these technology‐based intervention studies suggest that a computerized sign‐out with auto‐imported fields has the ability to improve physician efficiency and also improve inpatient care (reduction in number of patients missed on rounds, decrease in preventable adverse events).

Studies from nursing demonstrated that supplementing a verbal exchange with written information improved transfer of information, compared to verbal exchange alone.20 One of these studies rated the transfer of information using videotaped simulated handoff cases.21 Last, 1 nursing study that more directly involved patients in the handoff process resulted in improved nursing knowledge and greater patient empowerment (Table 1).22

White papers or consensus statements originated from international and national consortia in patient safety including the Australian Council for Safety and Quality in Healthcare,23 the Junior Doctors Committee of the British Medical Association,24 University Health Consortium,25 the Department of Defense Patient Safety Program,26 and The Joint Commission.27 Several common themes were prevalent in all white papers. First, there exists a need to train new personnel on how to perform an effective handoff. Second, efforts should be undertaken to ensure adequate time for handoffs and reduce interruptions during handoffs. Third, several of the papers supported verbal exchange that facilitates interactive questioning, focuses on ill patients, and delineates actions to be taken. Lastly, content should be updated to ensure transfer of the latest clinical information.

Final Handoff Recommendations

The final handoff recommendations are shown in Figure 2. The recommendations were designed to be consistent with the overall finding of the literature review, which supports the use of a verbal handoff supplemented with written documentation or a technological solution in a structured format. With the exception of 1 recommendation that is specific to service changes, all recommendations are designed to refer to shift changes and service changes. One overarching recommendation refers to the need for a formally recognized handoff plan at a shift change or change of service. The remaining 12 recommendations are divided into 4 that refer to hospitalist groups or programs, 3 that refer to verbal exchange, and 5 that refer to content exchange. The distinction is an important one because program‐level recommendations require organizational support and buy‐in to promote clinician participation and adherence. The 4 program recommendations also form the necessary framework for the remaining recommendations. For example, the second program recommendation describes the need for a standardized template or technology solution for accessing and recording patient information during the handoff. After a program adopts such a mechanism for exchanging patient information, the specific details for use and maintenance are outlined in greater detail in content exchange recommendations.

Figure 2

Because of the limited trials of handoff strategies, none of the recommendations are supported with level of evidence A (multiple numerous randomized controlled trials). In fact, with the exception of using a template or technology solution which was supported with level of evidence B, all handoff recommendations were supported with C level of evidence. The recommendations, however, were rated as Class I (effective) because there were no conflicting expert opinions or studies (Figure 2).

Discussion

In summary, our review of the literature supports the use of face‐to‐face verbal handoffs that are aided by the use of structured template to guide exchange of information. Furthermore, the development of these recommendations is the first effort of its kind for hospitalist handoffs and a movement towards standardizing the handoff process. While these recommendations are meant to provide structure to the hospitalist handoff process, the use and implementation by individual hospitalist programs may require more specific detail than these recommendations provide. Local modifications can allow for improved acceptance and adoption by practicing hospitalists. These recommendations can also help guide teaching efforts for academic hospitalists who are responsible for supervising residents.

The limitations of these recommendations related to lack of evidence in this field. Studies suffered from small size, poor description of methods, and a paucity of controlled interventions. The described technology solutions are not standardized or commercially available. Only 1 study included patient outcomes.28 There are no multicenter studies, studies of hospitalist handoffs, or studies to guide inclusion of specific content. Randomized controlled trials, interrupted time series analyses, and other rigorous study designs are needed in both teaching and non‐teaching settings to evaluate these recommendations and other approaches to improving handoffs. Ideally, these studies would occur through multicenter collaboratives and with human factors researchers familiar with mixed methods approaches to evaluate how and why interventions work.29 Efforts should focus on developing surrogate measures that are sensitive to handoff quality and related to important patient outcomes. The results of future studies should be used to refine the present recommendations. Locating new literature could be facilitated through the introduction of Medical Subject Heading for the term handoff by the National Library of Medicine. After completing this systematic review and developing the handoff recommendations described here, a few other noteworthy articles have been published on this topic, to which we refer interested readers. Several of these studies demonstrate that standardizing content and process during medical or surgical intern sign‐out improves resident confidence with handoffs,30 resident perceptions of accuracy and completeness of signout,31 and perceptions of patient safety.32 Another prospective audiotape study of 12 days of resident signout of clinical information demonstrated that poor quality oral sign‐outs was associated with an increased risk of post‐call resident reported signout‐related problems.5 Lastly, 1 nursing study demonstrated improved staff reports of safety, efficiency, and teamwork after a change from verbal reporting in an isolated room to bedside handover.33 Overall, these additional studies continue to support the current recommendations presented in this paper and do not significantly impact the conclusions of our literature review.

While lacking specific content domain recommendations, this report can be used as a starting point to guide development of self and peer assessment of hospitalist handoff quality. Development and validation of such assessments is especially important and can be incorporated into efforts to certify hospitalists through the recently approved certificate of focused practice in hospital medicine by the American Board of Internal Medicine (ABIM). Initiatives by several related organizations may help guide these effortsThe Joint Commission, the ABIM's Stepping Up to the Plate (SUTTP) Alliance, the Institute for Healthcare Improvement, the Information Transfer and Communication Practices (ITCP) Project for surgical care transitions, and the Hospital at Night (H@N) Program sponsored by the United Kingdom's National Health Service.3437 Professional medical organizations can also serve as powerful mediators of change in this area, not only by raising the visibility of handoffs, but also by mobilizing research funding. Patients and their caregivers may also play an important role in increasing awareness and education in this area. Future efforts should target handoffs not addressed in this initiative, such as transfers from emergency departments to inpatient care units, or between ICUs and the medical floor.

Conclusion

With the growth of hospital medicine and the increased acuity of inpatients, improving handoffs becomes an important part of ensuring patient safety. The goal of the SHM Handoffs Task Force was to begin to standardize handoffs at change of shift and change of servicea fundamental activity of hospitalists. These recommendations build on the limited literature in surgery, nursing, and medical informatics and provide a starting point for promoting safe and seamless in‐hospital handoffs for practitioners of Hospital Medicine.

Handoffs during hospitalization from one provider to another represent critical transition points in patient care.1 In‐hospital handoffs are a frequent occurrence, with 1 teaching hospital reporting 4000 handoffs daily for a total of 1.6 million per year.2

Incomplete or poor‐quality handoffs have been implicated as a source of adverse events and near misses in hospitalized patients.35 Standardizing the handoff process may improve patient safety during care transitions.6 In 2006, the Joint Commission issued a National Patient Safety Goal that requires care providers to adopt a standardized approach for handoff communications, including an opportunity to ask and respond to questions about a patient's care.7 The reductions in resident work hours by the Accreditation Council for Graduate Medical Education (ACGME) has also resulted in a greater number and greater scrutiny of handoffs in teaching hospitals.8, 9

In response to these issues, and because handoffs are a core competency for hospitalists, the Society of Hospital Medicine (SHM)convened a task force.10 Our goal was to develop a set of recommendations for handoffs that would be applicable in both community and academic settings; among physicians (hospitalists, internists, subspecialists, residents), nurse practitioners, and physicians assistants; and across roles including serving as the primary provider of hospital care, comanager, or consultant. This work focuses on handoffs that occur at shift change and service change.11 Shift changes are transitions of care between an outgoing provider and an incoming provider that occur at the end of the outgoing provider's continuous on‐duty period. Service changesa special type of shift changeare transitions of care between an outgoing provider and an incoming provider that occur when an outgoing provider is leaving a rotation or period of consecutive daily care for patients on the same service.

For this initiative, transfers of care in which the patient is moving from one patient area to another (eg, Emergency Department to inpatient floor, or floor to intensive care unit [ICU]) were excluded since they likely require unique consideration given their cross‐disciplinary and multispecialty nature. Likewise, transitions of care at hospital admission and discharge were also excluded because recommendations for discharge are already summarized in 2 complementary reports.12, 13

To develop recommendations for handoffs at routine shift change and service changes, the Handoff Task Force performed a systematic review of the literature to develop initial recommendations, obtained feedback from hospital‐based clinicians in addition to a panel of handoff experts, and finalized handoff recommendations, as well as a proposed research agenda, for the SHM.

Methods

The SHM Healthcare Quality and Patient Safety (HQPS) Committee convened the Handoff Task Force, which was comprised of 6 geographically diverse, predominantly academic hospitalists with backgrounds in education, patient safety, health communication, evidence‐based medicine, and handoffs. The Task Force then engaged a panel of 4 content experts selected for their work on handoffs in the fields of nursing, information technology, human factors engineering, and hospital medicine. Similar to clinical guideline development by professional societies, the Task Force used a combination of evidence‐based review and expert opinions to propose recommendations.

Results

Literature Review

Of the 374 articles identified by the electronic search of PubMed and the AHRQ Patient Safety Network, 109 were retrieved for detailed review, and 10 of these met the criteria for inclusion (Figure 1). Of these studies, 3 were derived from nursing literature and the remaining were tests of technology solutions or structured templates (Table 1).1618, 20, 22, 3842 No studies examined hospitalist handoffs. All eligible studies concerned shift change. There were no studies of service change. Only 1 study was a randomized controlled trial; the rest were pre‐post studies with historical controls or a controlled simulation. All reports were single‐site studies. Most outcomes were staff‐related or system‐related, with only 2 studies using patient outcomes.

Table 1.Characteristics of Studies Included in Review

Author (Year)	Study Design	Intervention	Setting and Study Population	Target	Outcomes
Abbreviations: IM, internal medicine; IS, ; UW, University of Washington.
Nursing
Kelly22 (2005)	Pre‐post	Change to walk‐round handover (at bedside) from baseline (control)	12‐bed rehab unit with 18 nurses and 10 patients	Staff, patient	11/18 nurses felt more or much more informed and involved; 8/10 patients felt more involved
Pothier et al.20 (2005)	Controlled simulation	Compared pure verbal to verbal with note‐taking to verbal plus typed content	Handover of 12 simulated patients over 5 cycles	System (data loss)	Minimal data loss with typed content, compared to 31% data retained with note‐taking, and no data retained with verbal only
Wallum38 (1995)	Pre‐post	Change from oral handover (baseline) to written template read with exchange	20 nurses in a geriatric dementia ward	Staff	83% of nurses felt care plans followed better; 88% knew care plans better
Technology or structured template
Cheah et al.39 (2005)	Pre‐post	Electronic template with free‐text entry compared to baseline	14 UK Surgery residents	Staff	100% (14) of residents rated electronic system as desirable, but 7 (50%) reported that information was not updated
Lee et al.40 (1996)	Pre‐post	Standardized signout card for interns to transmit information during handoffs compared to handwritten (baseline)	Inpatient cardiology service at IM residency program in Minnesota with 19 new interns over a 3‐month period	Staff	Intervention interns (n = 10) reported poor sign‐out less often than controls (n = 9) [intervention 8 nights (5.8%) vs. control 17 nights (14.9%); P = 0.016]
Kannry and Moore18 (1999)	Pre‐post	Compared web‐based signout program to usual system (baseline)	An academic teaching hospital in New York (34 patients admitted in 1997; 40 patients admitted in 1998)	System	Improved provider identification (86% web signout vs. 57% hospital census)
Petersen et al.17 (1998)	Pre‐post	4 months of computerized signouts compared to baseline period (control)	3747 patients admitted to the medical service at an academic teaching hospital	Patient	Preventable adverse events (ADE) decreased (1.7% to 1.2%, P < 0.10); risk of cross‐cover physician for ADE eliminated
Ram and Block41 (1993)	Pre‐post	Compared handwritten (baseline) to computer‐generated	Family medicine residents at 2 academic teaching hospitals [Buffalo (n = 16) and Pittsburgh (n = 16)]	Staff	Higher satisfaction after electronic signout, but complaints with burden of data entry and need to keep information updated
Van Eaton et al.42 (2004)	Pre‐post	Use of UW Cores links sign‐out to list for rounds and IS data	28 surgical and medical residents at 2 teaching hospitals	System	At 6 months, 66% of patients entered in system (adoption)
Van Eaton et al.16 (2005)	Prospective, randomized, crossover study.	Compared UW Cores* integrated system compared to usual system	14 inpatient resident teams (6 surgery, 8 IM) at 2 teaching hospitals for 5 months	Staff, system	50% reduction in the perceived time spent copying data [from 24% to 12% (P < 0.0001)] and number of patients missed on rounds (2.5 vs. 5 patients/team/month, P = 0.0001); improved signout quality (69.6% agree or strongly agree); and improved continuity of care (66.1% agree or strongly agree)

Figure 1

Overall, the literature presented supports the use of a verbal handoff supplemented with written documentation in a structured format or technology solution. The 2 most rigorous studies were led by Van Eaton et al.16 and Petersen et al.17 and focused on evaluating technology solutions. Van Eaton et al.16 performed a randomized controlled trial of a locally created rounding template with 161 surgical residents. This template downloads certain information (lab values and recent vital signs) from the hospital system into a sign‐out sheet and allows residents to enter notes about diagnoses, allergies, medications and to‐do items. When implemented, the investigators found the number of patients missed on rounds decreased by 50%. Residents reported an increase of 40% in the amount of time available to pre‐round, due largely to not having to copy data such as vital signs. They reported a decrease in rounding time by 3 hours per week, and this was perceived as helping them meet the ACGME 80 hours work rules. Lastly, the residents reported a higher quality of sign‐outs from their peers and perceived an overall improvement in continuity of care. Petersen and colleagues implemented a computerized sign‐out (auto‐imported medications, name, room number) in an internal medicine residency to improve continuity of care during cross‐coverage and decrease adverse events.17 Prior to the intervention, the frequency of preventable adverse events was 1.7% and it was significantly associated with cross‐coverage. Preventable adverse events were identified using a confidential self‐report system that was also validated by clinician review. After the intervention, the frequency of preventable adverse events dropped to 1.2% (P < 0.1), and cross‐coverage was no longer associated with preventable adverse events. In other studies, technological solutions also improved provider identification and staff communication.18, 19 Together, these technology‐based intervention studies suggest that a computerized sign‐out with auto‐imported fields has the ability to improve physician efficiency and also improve inpatient care (reduction in number of patients missed on rounds, decrease in preventable adverse events).

Studies from nursing demonstrated that supplementing a verbal exchange with written information improved transfer of information, compared to verbal exchange alone.20 One of these studies rated the transfer of information using videotaped simulated handoff cases.21 Last, 1 nursing study that more directly involved patients in the handoff process resulted in improved nursing knowledge and greater patient empowerment (Table 1).22

White papers or consensus statements originated from international and national consortia in patient safety including the Australian Council for Safety and Quality in Healthcare,23 the Junior Doctors Committee of the British Medical Association,24 University Health Consortium,25 the Department of Defense Patient Safety Program,26 and The Joint Commission.27 Several common themes were prevalent in all white papers. First, there exists a need to train new personnel on how to perform an effective handoff. Second, efforts should be undertaken to ensure adequate time for handoffs and reduce interruptions during handoffs. Third, several of the papers supported verbal exchange that facilitates interactive questioning, focuses on ill patients, and delineates actions to be taken. Lastly, content should be updated to ensure transfer of the latest clinical information.

Final Handoff Recommendations

The final handoff recommendations are shown in Figure 2. The recommendations were designed to be consistent with the overall finding of the literature review, which supports the use of a verbal handoff supplemented with written documentation or a technological solution in a structured format. With the exception of 1 recommendation that is specific to service changes, all recommendations are designed to refer to shift changes and service changes. One overarching recommendation refers to the need for a formally recognized handoff plan at a shift change or change of service. The remaining 12 recommendations are divided into 4 that refer to hospitalist groups or programs, 3 that refer to verbal exchange, and 5 that refer to content exchange. The distinction is an important one because program‐level recommendations require organizational support and buy‐in to promote clinician participation and adherence. The 4 program recommendations also form the necessary framework for the remaining recommendations. For example, the second program recommendation describes the need for a standardized template or technology solution for accessing and recording patient information during the handoff. After a program adopts such a mechanism for exchanging patient information, the specific details for use and maintenance are outlined in greater detail in content exchange recommendations.

Figure 2

Because of the limited trials of handoff strategies, none of the recommendations are supported with level of evidence A (multiple numerous randomized controlled trials). In fact, with the exception of using a template or technology solution which was supported with level of evidence B, all handoff recommendations were supported with C level of evidence. The recommendations, however, were rated as Class I (effective) because there were no conflicting expert opinions or studies (Figure 2).