User login
Inconsistent Code Status Documentation
For hospitalized patients, providers should ideally establish advanced directives for cardiopulmonary resuscitation, commonly referred to as a patient's code status. Having an end‐of‐life plan is important and is associated with better quality of life for patients.[1, 2, 3, 4, 5] Advanced directive discussions and documentation are key quality measures to improve end‐of‐life care for vulnerable elders.[6, 7, 8]
Clear and consistent code status documentation is a prerequisite to providing care that respects hospitalized patients' preferences. Code status documentation only occurs in a minority of hospitalized patients, ranging from 25% of patients on a general medical ward to 36% of patients on elderly‐care wards.[9] Even in high‐risk patients, such as patients with metastatic cancer, providers only documented code status 20% of the time.[10] Even when code status documentation occurs, the amount of detail regarding patient goals and values, prognosis, and treatment options is generally poor.[11, 12] There are also concerns about the accuracy of code status documentation.[13, 14, 15, 16, 17, 18, 19] For example, a recent study found that for patients who had discussed their code status during their hospitalization, only 30% had documentation of their preferences in their chart that accurately reflected what was discussed.[20]
Further complicating matters is the fact that providers document key patient information, such as a patient's code status, in multiple places (eg, progress notes, physician orders). As a result, an additional documentation problem of inconsistency can arise for 2 reasons. First, code status documentation can be inconsistent because of incomplete documentation. Incomplete documentation is primarily a problem in patients who do not want to be resuscitated (ie, do not resuscitate [DNR]), because the absence of code status documentation leads front‐line staff to assume that the patient wants to be resuscitated (ie, full code). Second, inconsistent documentation can occur because of conflicting documentation (eg, a patient has a different code status documented in 2 or more places).
Together, these documentation problems have the potential to lead healthcare providers to resuscitate patients who do not wish to be resuscitated, or for patients who wish to be resuscitated to have delays in their resuscitation efforts. This study will extend the knowledge from the previous literature by exploring how the complexity and redundancy of clinical documentation practices affect the quality of code status documentation. To our knowledge, there are no prior studies that focus specifically on the frequency and clinical relevance of inconsistent code status documentation for inpatients across multiple documentation sources.
METHODS
Study Context
This is a point‐prevalence study conducted at 3 academic medical centers (AMCs) affiliated with the University of Toronto. At all 3 AMCs, the majority of general internal medicine (GIM) patients are admitted to 1 of 4 clinical teaching units (CTUs). The physician team on each CTU consists of 1 attending staff, 1 senior resident (second or third year resident), 2 to 3 first‐year residents, and 2 to 3 medical students. CTUs typically care for between 15 and 25 patients. The research ethics boards at each of the AMCs approved this study.
Existing Code Status Documentation Processes
At all 3 AMCs, providers document patient code status in 5 different places: (1) progress notes (admission and daily progress notes in the paper chart), (2) physician orders (computerized orders at 1 site, paper orders at the other 2 sites), (3) electronic sign‐out lists (Web‐based tools used by residents to support patient handover), (4) nursing‐care plan (used by nurses to document care plans for their assigned patients), and (5) DNR sheet (a cover sheet placed at the front of the paper chart in patients who have a DNR order) (see Supporting Information, Appendix, in the online version of this article). None of these documentation sources link automatically to one another. Once a physician establishes a patient's code status, it should be documented in the progress notes. The same physician should also write the code status as a physician order and update the patient's code status in the Web‐based electronic sign‐out list. The nurse responsible for the patient transcribes the code status order in the nursing‐care plan. For DNR patients, nurses or physicians (depending on the AMC) also place the DNR sheet in the front of the chart.
At our 3 AMCs, in the event of a cardiac arrest, resident physicians and nurses are typically the first responders. To quickly determine a patient's code status nurses and resident physicians look for the presence or absence of a DNR sheet. In addition, nurses rely on their nursing‐care plan and resident physicians rely on their electronic sign‐out list.
Eligibility Criteria and Sampling Strategy
Our study included GIM patients admitted to a CTU at 1 of 3 AMCs, and excluded admitted GIM patients who remained in the emergency department (due to differences in code status documentation processes). Data collection took place between September 2010 and September 2011 on days when the principal author (A.S.W.) was available to collect the data.
We collected data for all patients from a single GIM CTU on the same day to minimize the chance that a team updates or changes a patient's code status during data collection. We included each of the 4 CTUs at the 3 study sites once during the study period (ie, 12 full days of data collection).
Study Measures and Data Collection
One study author (A.S.W.) screened the 5 code status documentation sources listed above for each patient and recorded the documented code status as full code, DNR, or blank (if there was nothing entered) in a database. We also collected patient demographic data, admitting diagnosis, length of stay, admission to home ward (ie, the medicine ward affiliated with the CTU team that admitted the patient), free‐text code status documentation, transfer to the intensive care unit during their hospitalization, and whether the patient is receiving comfort measures, up to the time of data collection. Because the study investigators were not members of the team providing care to patients included in the study, we could not directly elicit the patient's actual code status.
The primary study outcome measures were the completeness and consistency of code status documentation across the 5 documentation sources. For completeness, we included data relating to 4 documentation sources only, excluding the DNR sheet because it is only relevant for DNR patients. We defined inconsistent code status documentation a priori as (1) the code status is conflicting in at least 2 documentation sources (eg, full code in 1 source and DNR in another) or (2) the code status is documented in 1 or more documentation source and not documented in at least 1 documentation source (eg, full code in 1 source and blank in another).
We then subdivided code status documentation inconsistencies into nonclinically relevant and clinically relevant subcategories. For example, a nonclinically relevant inconsistency would be if a physician documented full code in the physician orders, but a nurse did not document anything in the nursing‐care plan, because most providers would assume a preference for resuscitation in the absence of code status documentation in the nursing‐care plan.
We defined clinically relevant inconsistencies as those that would reasonably lead healthcare providers referring to different documentation sources to respond differently in the event of a cardiac arrest (eg, the physician orders show DNR whereas the nursing‐care plan is blanka provider who refers to the physician orders would not resuscitate the patient, but another provider who refers to the blank nursing‐care plan would resuscitate the patient).
We determined the proportion of patients with inconsistent code status documentation by listing the 31 different permutations of code status documentation in our data (Figure 1). Using the prespecified definition of inconsistent code status documentation, 3 study authors (I.A.D., B.M.W., R.C.W.) independently determined whether each permutation met the criteria for inconsistent code status documentation, and judged the clinical relevance of each documentation inconsistency. We resolved disagreements by consensus.
Statistical Analysis
We calculated descriptive statistics for all variables, summarizing continuous measures using means and standard deviations, and categorical measures using counts, percentages, and their associated 95% confidence intervals. Logistic regression analyses adjusting for the correlation among observations taken from the same team were carried out. Each of the 4 variables of interest (patient age, length of stay, receiving comfort measures, free text code status documentation) was run in a bivariate model to obtain unadjusted estimates as well as the final multivariable model. All estimates were displayed as odds ratios (ORs) and their associated 95% confidence intervals (CIs). A P value <0.05 was used to denote statistical significance. We also carried out a kappa analysis to assess inter‐rater agreement when judging whether inconsistent documentation is clinically relevant. All analyses were carried out using SAS version 9.3 (SAS Institute, Cary, NC).
RESULTS
There were 194 patients potentially eligible for inclusion. Seven admitted GIM patients who had not been transferred from the emergency department were excluded, leaving 187 patients in the study. The mean patient age was 70 years; 83 (44%) were female. The median length of stay up to the time of data collection was 6 days, with the majority (156 [83%]) of patients admitted to their home ward. Ten (5%) patients were receiving comfort measures.
Completeness of Code Status Documentation
Thirty‐eight (20%; 95% CI, 14%‐26%) patients had complete and consistent code status documentation across all documentation sources, whereas 27 (14%; 95% CI, 9%‐19%) patients had no code status documented in any documentation source. By documentation source, providers documented code status in the progress notes for 89 patients (48%; 95% CI, 40%‐55%), the physician orders for 107 patients (57%; 95% CI, 50%‐64%), the nursing‐care plan for 110 patients (59%; 95% CI, 51%‐66%), and the electronic sign‐out list for 129 patients (69%; 95% CI, 62%‐76%).
Consistency of Code Status Documentation
The remaining 122 patients (65%; 95% CI, 58%‐72%) had at least 1 code status documentation inconsistency. Of these, 38 patients (20%; 95% CI, 14%‐26%) had a clinically relevant code status documentation inconsistency. Code status documentation inconsistency differed by site; the 2 hospitals with paper‐based physician orders had fewer patients with complete and consistent code status documentation compared to the hospital where physician orders are electronic (15% vs 42%, respectively, P<0.001) (Table 1).
| Physician Code Status Order | Sites 1 & 2: Paper‐Based, N=108 | Site 3: Electronic, N=52 | P Value |
|---|---|---|---|
| Consistent code status documentation | 16 (15%) | 22 (42%) | |
| Inconsistent code status documentation, nonclinically relevant | 60 (56%) | 30 (58%) | <0.0001 |
| Inconsistent code status documentation, clinically relevant | 32 (30%) | 6 (12%) |
The permutations of clinically relevant and nonclinically relevant inconsistencies are summarized in Figure 1. We achieved high inter‐rater reliability among the 3 independent reviewers with respect to rating the clinical relevance of documentation inconsistencies (=0.86 [95% CI, 0.76‐0.95]).
To identify correlates of clinically relevant code status documentation inconsistencies, we included 4 variables of interest (patient age, length of stay, receiving comfort measures, and free text code status documentation) in a logistic regression analysis. Bivariate analyses demonstrated that increased age (OR =1.07 [95% CI, 1.05‐1.10] for every 1‐year increase in age, P<0.001) and receiving comfort measures (OR= 10.98 [95% CI, 1.94‐62.12], P=0.007) were associated with a clinically relevant code status documentation inconsistency. Using these 4 variables in a multivariable analysis clustering for physician team, increased age (OR=1.07 [95% CI, 1.04‐1.10] for every 1‐year increase in age, P<0.0001) and receiving comfort measures (OR=9.39 [95% CI, 1.3565.19], P=0.02) remained as independent positive correlates of having a clinically relevant code status documentation inconsistency (Table 2).
| Clinically Relevant Inconsistencies, N=38 | No Inconsistencies and Nonclinically Relevant Inconsistencies, N=149 | P Value | |
|---|---|---|---|
| |||
| Age, y, mean (SD) | 83 (10) | 67 (19) | <0.0001* |
| Length of stay, d, median (IQR) | 6.5 (310) | 6 (219) | 0.39 |
| Receiving comfort measures, n (%) | 7 (18%) | 3 (2%) | <0.0001* |
| Free‐text code status documentation, n (%) | 18 (47%) | 58 (39%) | 0.34 |
DISCUSSION
We found that 2 out of 3 patients had at least 1 inconsistency in code status documentation, and that 1 in 5 patients had at least 1 clinically relevant code status documentation inconsistency. The majority of clinically relevant inconsistencies occurred because there was a DNR order written in some sources of code status documentation, and no orders in other documentation sources. However, there were 4 striking examples where DNR was written in some sources of code status documentation and full code was written in other documentation sources (Figure 1).
Older patients and patients receiving comfort measures were more likely to have a clinically relevant inconsistency in code status documentation. This is particularly concerning, because they are among the most vulnerable patients at highest risk for having an in‐hospital cardiac arrest.
Our study extends the findings of prior studies that identified gaps in completeness and accuracy of code status documentation and describes another important gap in the quality and consistency of code status documentation.[20] This becomes particularly important because efforts aimed at increasing documentation of patients' code status without ensuring consistency across documentation sources may still result in patients being resuscitated or not resuscitated inappropriately.
This issue of poorly integrated health records is relevant for many other aspects of patient care. For example, 1 study found significant discrepancies in patient medication allergy documentation across multiple health records.[21] This fragmentation of documentation of the same patient information in multiple health records requires attention and should be the focus of institutional quality improvement efforts.
There are several potential ways to improve the code status documentation process. First, the use of standard fields or standardized orders can increase the completeness and improve the clarity of code status documentation.[22, 23] For institutions with an electronic medical record, forcing functions may further increase code status documentation. One study found that the implementation of an electronic medical record increased code status documentation from 4% to 63%.[24] We found similarly that the site with electronic physician orders had higher rates of complete and consistent code status documentation.
A second approach is to minimize the number of different sources for code status documentation. Institutions should critically examine each place where providers could document code status and decide whether this information adds value, and create policies to restrict unnecessary duplicate documentation and ensure accurate documentation of code status in several key, centralized locations.[25] A third option would be to automatically synchronize all code status documentation sources.[25] This final approach requires a fully integrated electronic health record.
Our study has several limitations. Although we report a large number of code status documentation inconsistencies, we do not know how many of these lead to incorrect resuscitative measures, so the actual impact on patient care is unknown. Also, because we were focusing on inconsistencies among sources of code status documentation, and not on accurate documentation of a patients' code status, the patients' actual preferences were not elicited and are not known. Finally, we carried out our study in 3 AMCs with residents that rotate from 1 site to another. The transient nature of resident work may increase the likelihood of documentation inconsistencies, because trainees may be less aware of local processes. In addition, the way front‐line staff uses clinical documentation sources to determine a patient's code status may differ at other institutions. Therefore, our estimate of clinical relevance may not be generalizable to other institutions with different front‐line processes or with healthcare teams that are more stable and aware of local documentation processes.
In summary, our study uncovered significant gaps in the quality of code status documentation that span 3 different AMCs. Having multiple, poorly integrated sources for code status documentation leads to a significant number of concerning inconsistencies that create opportunities for healthcare providers to inappropriately deliver or withhold resuscitative measures that conflict with patients' expressed wishes. Institutions need to be aware of this potential documentation hazard and take steps to minimize code status documentation inconsistencies. Even though cardiac arrests occur infrequently, if healthcare teams take inappropriate action because of these code status documentation inconsistencies, the consequences can be devastating.
Disclosure
Nothing to report.
- , , , et al. Advance directives for medical care—a case for greater use. N Engl J Med. 1991;324(13):889–895.
- , , , et al; Canadian Researchers at the End of Life Network (CARENET). What matters most in end‐of‐life care: perceptions of seriously ill patients and their family memebrs. CMAJ. 2006;174(5):627–633.
- , , . Advance directives and outcomes of surrogate decision making before death. N Eng J Med. 2010;362(13):1211–1218.
- , , , et al. Associations between end‐of‐life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment. JAMA. 2008;300(14):1665–1673.
- , , , . The impact of advance care planning on end of life care in elderly patients: randomized controlled trial. BMJ. 2010;340:c1345.
- , . Quality indicators for end‐of‐life care in vulnerable elders. Ann Intern Med. 2001;135(8):667–685.
- , , , . Assessing care of vulnerable elders: methods for developing quality indicators. Ann Intern Med. 2001;135:647–652.
- , , , et al. Perceptions by family members of the dying experience of older and seriously ill patients. Ann Intern Med. 1997;126(2):97–106.
- , . Cardiopulmonary resuscitation: capacity, discussion and documentation. Q J Med. 2006;99(10):683–690.
- , , , et al. Code status documentation in the outpatient electronic medical records or patients with metastatic cancer. J Gen Intern Med. 2009;25(2):150–153.
- , , , et al. Factors associated with discussion of care plans and code status at the time of hospital admission: results from the multicenter hospitalist study. J Hosp Med. 2008;3(6):437–445.
- , , , . Documentation quality of inpatient code status discussions. J Pain Symptom Manage. 2014;48(4):632–638.
- , , , . The use of life‐sustaining treatments in hospitalized persons aged 80 and older. J Am Geriatr Soc. 2002;50(5):930–934.
- The SUPPORT Principal Investigators. A controlled trial to improve care for seriously ill hospitalized patients: the Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments (SUPPORT). JAMA. 1995;274(20):1591–1598.
- , , . Some treatment‐withholding implications of no‐code orders in an academic hospital. Crit Care Med. 1984;12(10):879–881.
- . A prospective study of patients with DNR orders in a teaching hospital. Arch Intern Med. 1998;148(10):2193–2198.
- , . Compliance with do‐not‐resuscitate orders for hospitalized patient transported to radiology department. Ann Int Med. 1998;129(10):801–805.
- , . Identification of inpatient DNR status: a safety hazard begging for standardization. J Hosp Med. 2007;2(6):366–371.
- . Controlling death: the false promise of advance directives. Ann Int Med. 2007;147(1):51–57.
- , , , et al. Failure to engage hospitalized elderly patients and their families in advance care planning. JAMA Intern Med. 2013;173(9):778–787.
- , , , et al. The recording of drug sensitivities for older people living in care homes. Br J Clin Pharmacol. 2010;69(5):553–557.
- , , , , , . A comparison of methods to communicate treatment preferences in nursing facilities: traditional practices versus the physician orders for life‐sustaining treatment program. J Am Geriatr Soc. 2010;58(7):1241–1248.
- , , , et al. Evaluation of a treatment limitation policy with a specific treatment‐limiting order page. Arch Intern Med. 1994;154(4):425–432.
- , , , et al. An electronic medical record intervention increased nursing home advance directive orders and documentation. J Am Geriatr Soc. 2007;55(7):1001–1006.
- , , , et al. Honouring patient's resuscitation wishes: a multiphased effort to improve identification and documentation. BMJ Quality Safety. 2013;22(1):85–92.
For hospitalized patients, providers should ideally establish advanced directives for cardiopulmonary resuscitation, commonly referred to as a patient's code status. Having an end‐of‐life plan is important and is associated with better quality of life for patients.[1, 2, 3, 4, 5] Advanced directive discussions and documentation are key quality measures to improve end‐of‐life care for vulnerable elders.[6, 7, 8]
Clear and consistent code status documentation is a prerequisite to providing care that respects hospitalized patients' preferences. Code status documentation only occurs in a minority of hospitalized patients, ranging from 25% of patients on a general medical ward to 36% of patients on elderly‐care wards.[9] Even in high‐risk patients, such as patients with metastatic cancer, providers only documented code status 20% of the time.[10] Even when code status documentation occurs, the amount of detail regarding patient goals and values, prognosis, and treatment options is generally poor.[11, 12] There are also concerns about the accuracy of code status documentation.[13, 14, 15, 16, 17, 18, 19] For example, a recent study found that for patients who had discussed their code status during their hospitalization, only 30% had documentation of their preferences in their chart that accurately reflected what was discussed.[20]
Further complicating matters is the fact that providers document key patient information, such as a patient's code status, in multiple places (eg, progress notes, physician orders). As a result, an additional documentation problem of inconsistency can arise for 2 reasons. First, code status documentation can be inconsistent because of incomplete documentation. Incomplete documentation is primarily a problem in patients who do not want to be resuscitated (ie, do not resuscitate [DNR]), because the absence of code status documentation leads front‐line staff to assume that the patient wants to be resuscitated (ie, full code). Second, inconsistent documentation can occur because of conflicting documentation (eg, a patient has a different code status documented in 2 or more places).
Together, these documentation problems have the potential to lead healthcare providers to resuscitate patients who do not wish to be resuscitated, or for patients who wish to be resuscitated to have delays in their resuscitation efforts. This study will extend the knowledge from the previous literature by exploring how the complexity and redundancy of clinical documentation practices affect the quality of code status documentation. To our knowledge, there are no prior studies that focus specifically on the frequency and clinical relevance of inconsistent code status documentation for inpatients across multiple documentation sources.
METHODS
Study Context
This is a point‐prevalence study conducted at 3 academic medical centers (AMCs) affiliated with the University of Toronto. At all 3 AMCs, the majority of general internal medicine (GIM) patients are admitted to 1 of 4 clinical teaching units (CTUs). The physician team on each CTU consists of 1 attending staff, 1 senior resident (second or third year resident), 2 to 3 first‐year residents, and 2 to 3 medical students. CTUs typically care for between 15 and 25 patients. The research ethics boards at each of the AMCs approved this study.
Existing Code Status Documentation Processes
At all 3 AMCs, providers document patient code status in 5 different places: (1) progress notes (admission and daily progress notes in the paper chart), (2) physician orders (computerized orders at 1 site, paper orders at the other 2 sites), (3) electronic sign‐out lists (Web‐based tools used by residents to support patient handover), (4) nursing‐care plan (used by nurses to document care plans for their assigned patients), and (5) DNR sheet (a cover sheet placed at the front of the paper chart in patients who have a DNR order) (see Supporting Information, Appendix, in the online version of this article). None of these documentation sources link automatically to one another. Once a physician establishes a patient's code status, it should be documented in the progress notes. The same physician should also write the code status as a physician order and update the patient's code status in the Web‐based electronic sign‐out list. The nurse responsible for the patient transcribes the code status order in the nursing‐care plan. For DNR patients, nurses or physicians (depending on the AMC) also place the DNR sheet in the front of the chart.
At our 3 AMCs, in the event of a cardiac arrest, resident physicians and nurses are typically the first responders. To quickly determine a patient's code status nurses and resident physicians look for the presence or absence of a DNR sheet. In addition, nurses rely on their nursing‐care plan and resident physicians rely on their electronic sign‐out list.
Eligibility Criteria and Sampling Strategy
Our study included GIM patients admitted to a CTU at 1 of 3 AMCs, and excluded admitted GIM patients who remained in the emergency department (due to differences in code status documentation processes). Data collection took place between September 2010 and September 2011 on days when the principal author (A.S.W.) was available to collect the data.
We collected data for all patients from a single GIM CTU on the same day to minimize the chance that a team updates or changes a patient's code status during data collection. We included each of the 4 CTUs at the 3 study sites once during the study period (ie, 12 full days of data collection).
Study Measures and Data Collection
One study author (A.S.W.) screened the 5 code status documentation sources listed above for each patient and recorded the documented code status as full code, DNR, or blank (if there was nothing entered) in a database. We also collected patient demographic data, admitting diagnosis, length of stay, admission to home ward (ie, the medicine ward affiliated with the CTU team that admitted the patient), free‐text code status documentation, transfer to the intensive care unit during their hospitalization, and whether the patient is receiving comfort measures, up to the time of data collection. Because the study investigators were not members of the team providing care to patients included in the study, we could not directly elicit the patient's actual code status.
The primary study outcome measures were the completeness and consistency of code status documentation across the 5 documentation sources. For completeness, we included data relating to 4 documentation sources only, excluding the DNR sheet because it is only relevant for DNR patients. We defined inconsistent code status documentation a priori as (1) the code status is conflicting in at least 2 documentation sources (eg, full code in 1 source and DNR in another) or (2) the code status is documented in 1 or more documentation source and not documented in at least 1 documentation source (eg, full code in 1 source and blank in another).
We then subdivided code status documentation inconsistencies into nonclinically relevant and clinically relevant subcategories. For example, a nonclinically relevant inconsistency would be if a physician documented full code in the physician orders, but a nurse did not document anything in the nursing‐care plan, because most providers would assume a preference for resuscitation in the absence of code status documentation in the nursing‐care plan.
We defined clinically relevant inconsistencies as those that would reasonably lead healthcare providers referring to different documentation sources to respond differently in the event of a cardiac arrest (eg, the physician orders show DNR whereas the nursing‐care plan is blanka provider who refers to the physician orders would not resuscitate the patient, but another provider who refers to the blank nursing‐care plan would resuscitate the patient).
We determined the proportion of patients with inconsistent code status documentation by listing the 31 different permutations of code status documentation in our data (Figure 1). Using the prespecified definition of inconsistent code status documentation, 3 study authors (I.A.D., B.M.W., R.C.W.) independently determined whether each permutation met the criteria for inconsistent code status documentation, and judged the clinical relevance of each documentation inconsistency. We resolved disagreements by consensus.
Statistical Analysis
We calculated descriptive statistics for all variables, summarizing continuous measures using means and standard deviations, and categorical measures using counts, percentages, and their associated 95% confidence intervals. Logistic regression analyses adjusting for the correlation among observations taken from the same team were carried out. Each of the 4 variables of interest (patient age, length of stay, receiving comfort measures, free text code status documentation) was run in a bivariate model to obtain unadjusted estimates as well as the final multivariable model. All estimates were displayed as odds ratios (ORs) and their associated 95% confidence intervals (CIs). A P value <0.05 was used to denote statistical significance. We also carried out a kappa analysis to assess inter‐rater agreement when judging whether inconsistent documentation is clinically relevant. All analyses were carried out using SAS version 9.3 (SAS Institute, Cary, NC).
RESULTS
There were 194 patients potentially eligible for inclusion. Seven admitted GIM patients who had not been transferred from the emergency department were excluded, leaving 187 patients in the study. The mean patient age was 70 years; 83 (44%) were female. The median length of stay up to the time of data collection was 6 days, with the majority (156 [83%]) of patients admitted to their home ward. Ten (5%) patients were receiving comfort measures.
Completeness of Code Status Documentation
Thirty‐eight (20%; 95% CI, 14%‐26%) patients had complete and consistent code status documentation across all documentation sources, whereas 27 (14%; 95% CI, 9%‐19%) patients had no code status documented in any documentation source. By documentation source, providers documented code status in the progress notes for 89 patients (48%; 95% CI, 40%‐55%), the physician orders for 107 patients (57%; 95% CI, 50%‐64%), the nursing‐care plan for 110 patients (59%; 95% CI, 51%‐66%), and the electronic sign‐out list for 129 patients (69%; 95% CI, 62%‐76%).
Consistency of Code Status Documentation
The remaining 122 patients (65%; 95% CI, 58%‐72%) had at least 1 code status documentation inconsistency. Of these, 38 patients (20%; 95% CI, 14%‐26%) had a clinically relevant code status documentation inconsistency. Code status documentation inconsistency differed by site; the 2 hospitals with paper‐based physician orders had fewer patients with complete and consistent code status documentation compared to the hospital where physician orders are electronic (15% vs 42%, respectively, P<0.001) (Table 1).
| Physician Code Status Order | Sites 1 & 2: Paper‐Based, N=108 | Site 3: Electronic, N=52 | P Value |
|---|---|---|---|
| Consistent code status documentation | 16 (15%) | 22 (42%) | |
| Inconsistent code status documentation, nonclinically relevant | 60 (56%) | 30 (58%) | <0.0001 |
| Inconsistent code status documentation, clinically relevant | 32 (30%) | 6 (12%) |
The permutations of clinically relevant and nonclinically relevant inconsistencies are summarized in Figure 1. We achieved high inter‐rater reliability among the 3 independent reviewers with respect to rating the clinical relevance of documentation inconsistencies (=0.86 [95% CI, 0.76‐0.95]).
To identify correlates of clinically relevant code status documentation inconsistencies, we included 4 variables of interest (patient age, length of stay, receiving comfort measures, and free text code status documentation) in a logistic regression analysis. Bivariate analyses demonstrated that increased age (OR =1.07 [95% CI, 1.05‐1.10] for every 1‐year increase in age, P<0.001) and receiving comfort measures (OR= 10.98 [95% CI, 1.94‐62.12], P=0.007) were associated with a clinically relevant code status documentation inconsistency. Using these 4 variables in a multivariable analysis clustering for physician team, increased age (OR=1.07 [95% CI, 1.04‐1.10] for every 1‐year increase in age, P<0.0001) and receiving comfort measures (OR=9.39 [95% CI, 1.3565.19], P=0.02) remained as independent positive correlates of having a clinically relevant code status documentation inconsistency (Table 2).
| Clinically Relevant Inconsistencies, N=38 | No Inconsistencies and Nonclinically Relevant Inconsistencies, N=149 | P Value | |
|---|---|---|---|
| |||
| Age, y, mean (SD) | 83 (10) | 67 (19) | <0.0001* |
| Length of stay, d, median (IQR) | 6.5 (310) | 6 (219) | 0.39 |
| Receiving comfort measures, n (%) | 7 (18%) | 3 (2%) | <0.0001* |
| Free‐text code status documentation, n (%) | 18 (47%) | 58 (39%) | 0.34 |
DISCUSSION
We found that 2 out of 3 patients had at least 1 inconsistency in code status documentation, and that 1 in 5 patients had at least 1 clinically relevant code status documentation inconsistency. The majority of clinically relevant inconsistencies occurred because there was a DNR order written in some sources of code status documentation, and no orders in other documentation sources. However, there were 4 striking examples where DNR was written in some sources of code status documentation and full code was written in other documentation sources (Figure 1).
Older patients and patients receiving comfort measures were more likely to have a clinically relevant inconsistency in code status documentation. This is particularly concerning, because they are among the most vulnerable patients at highest risk for having an in‐hospital cardiac arrest.
Our study extends the findings of prior studies that identified gaps in completeness and accuracy of code status documentation and describes another important gap in the quality and consistency of code status documentation.[20] This becomes particularly important because efforts aimed at increasing documentation of patients' code status without ensuring consistency across documentation sources may still result in patients being resuscitated or not resuscitated inappropriately.
This issue of poorly integrated health records is relevant for many other aspects of patient care. For example, 1 study found significant discrepancies in patient medication allergy documentation across multiple health records.[21] This fragmentation of documentation of the same patient information in multiple health records requires attention and should be the focus of institutional quality improvement efforts.
There are several potential ways to improve the code status documentation process. First, the use of standard fields or standardized orders can increase the completeness and improve the clarity of code status documentation.[22, 23] For institutions with an electronic medical record, forcing functions may further increase code status documentation. One study found that the implementation of an electronic medical record increased code status documentation from 4% to 63%.[24] We found similarly that the site with electronic physician orders had higher rates of complete and consistent code status documentation.
A second approach is to minimize the number of different sources for code status documentation. Institutions should critically examine each place where providers could document code status and decide whether this information adds value, and create policies to restrict unnecessary duplicate documentation and ensure accurate documentation of code status in several key, centralized locations.[25] A third option would be to automatically synchronize all code status documentation sources.[25] This final approach requires a fully integrated electronic health record.
Our study has several limitations. Although we report a large number of code status documentation inconsistencies, we do not know how many of these lead to incorrect resuscitative measures, so the actual impact on patient care is unknown. Also, because we were focusing on inconsistencies among sources of code status documentation, and not on accurate documentation of a patients' code status, the patients' actual preferences were not elicited and are not known. Finally, we carried out our study in 3 AMCs with residents that rotate from 1 site to another. The transient nature of resident work may increase the likelihood of documentation inconsistencies, because trainees may be less aware of local processes. In addition, the way front‐line staff uses clinical documentation sources to determine a patient's code status may differ at other institutions. Therefore, our estimate of clinical relevance may not be generalizable to other institutions with different front‐line processes or with healthcare teams that are more stable and aware of local documentation processes.
In summary, our study uncovered significant gaps in the quality of code status documentation that span 3 different AMCs. Having multiple, poorly integrated sources for code status documentation leads to a significant number of concerning inconsistencies that create opportunities for healthcare providers to inappropriately deliver or withhold resuscitative measures that conflict with patients' expressed wishes. Institutions need to be aware of this potential documentation hazard and take steps to minimize code status documentation inconsistencies. Even though cardiac arrests occur infrequently, if healthcare teams take inappropriate action because of these code status documentation inconsistencies, the consequences can be devastating.
Disclosure
Nothing to report.
For hospitalized patients, providers should ideally establish advanced directives for cardiopulmonary resuscitation, commonly referred to as a patient's code status. Having an end‐of‐life plan is important and is associated with better quality of life for patients.[1, 2, 3, 4, 5] Advanced directive discussions and documentation are key quality measures to improve end‐of‐life care for vulnerable elders.[6, 7, 8]
Clear and consistent code status documentation is a prerequisite to providing care that respects hospitalized patients' preferences. Code status documentation only occurs in a minority of hospitalized patients, ranging from 25% of patients on a general medical ward to 36% of patients on elderly‐care wards.[9] Even in high‐risk patients, such as patients with metastatic cancer, providers only documented code status 20% of the time.[10] Even when code status documentation occurs, the amount of detail regarding patient goals and values, prognosis, and treatment options is generally poor.[11, 12] There are also concerns about the accuracy of code status documentation.[13, 14, 15, 16, 17, 18, 19] For example, a recent study found that for patients who had discussed their code status during their hospitalization, only 30% had documentation of their preferences in their chart that accurately reflected what was discussed.[20]
Further complicating matters is the fact that providers document key patient information, such as a patient's code status, in multiple places (eg, progress notes, physician orders). As a result, an additional documentation problem of inconsistency can arise for 2 reasons. First, code status documentation can be inconsistent because of incomplete documentation. Incomplete documentation is primarily a problem in patients who do not want to be resuscitated (ie, do not resuscitate [DNR]), because the absence of code status documentation leads front‐line staff to assume that the patient wants to be resuscitated (ie, full code). Second, inconsistent documentation can occur because of conflicting documentation (eg, a patient has a different code status documented in 2 or more places).
Together, these documentation problems have the potential to lead healthcare providers to resuscitate patients who do not wish to be resuscitated, or for patients who wish to be resuscitated to have delays in their resuscitation efforts. This study will extend the knowledge from the previous literature by exploring how the complexity and redundancy of clinical documentation practices affect the quality of code status documentation. To our knowledge, there are no prior studies that focus specifically on the frequency and clinical relevance of inconsistent code status documentation for inpatients across multiple documentation sources.
METHODS
Study Context
This is a point‐prevalence study conducted at 3 academic medical centers (AMCs) affiliated with the University of Toronto. At all 3 AMCs, the majority of general internal medicine (GIM) patients are admitted to 1 of 4 clinical teaching units (CTUs). The physician team on each CTU consists of 1 attending staff, 1 senior resident (second or third year resident), 2 to 3 first‐year residents, and 2 to 3 medical students. CTUs typically care for between 15 and 25 patients. The research ethics boards at each of the AMCs approved this study.
Existing Code Status Documentation Processes
At all 3 AMCs, providers document patient code status in 5 different places: (1) progress notes (admission and daily progress notes in the paper chart), (2) physician orders (computerized orders at 1 site, paper orders at the other 2 sites), (3) electronic sign‐out lists (Web‐based tools used by residents to support patient handover), (4) nursing‐care plan (used by nurses to document care plans for their assigned patients), and (5) DNR sheet (a cover sheet placed at the front of the paper chart in patients who have a DNR order) (see Supporting Information, Appendix, in the online version of this article). None of these documentation sources link automatically to one another. Once a physician establishes a patient's code status, it should be documented in the progress notes. The same physician should also write the code status as a physician order and update the patient's code status in the Web‐based electronic sign‐out list. The nurse responsible for the patient transcribes the code status order in the nursing‐care plan. For DNR patients, nurses or physicians (depending on the AMC) also place the DNR sheet in the front of the chart.
At our 3 AMCs, in the event of a cardiac arrest, resident physicians and nurses are typically the first responders. To quickly determine a patient's code status nurses and resident physicians look for the presence or absence of a DNR sheet. In addition, nurses rely on their nursing‐care plan and resident physicians rely on their electronic sign‐out list.
Eligibility Criteria and Sampling Strategy
Our study included GIM patients admitted to a CTU at 1 of 3 AMCs, and excluded admitted GIM patients who remained in the emergency department (due to differences in code status documentation processes). Data collection took place between September 2010 and September 2011 on days when the principal author (A.S.W.) was available to collect the data.
We collected data for all patients from a single GIM CTU on the same day to minimize the chance that a team updates or changes a patient's code status during data collection. We included each of the 4 CTUs at the 3 study sites once during the study period (ie, 12 full days of data collection).
Study Measures and Data Collection
One study author (A.S.W.) screened the 5 code status documentation sources listed above for each patient and recorded the documented code status as full code, DNR, or blank (if there was nothing entered) in a database. We also collected patient demographic data, admitting diagnosis, length of stay, admission to home ward (ie, the medicine ward affiliated with the CTU team that admitted the patient), free‐text code status documentation, transfer to the intensive care unit during their hospitalization, and whether the patient is receiving comfort measures, up to the time of data collection. Because the study investigators were not members of the team providing care to patients included in the study, we could not directly elicit the patient's actual code status.
The primary study outcome measures were the completeness and consistency of code status documentation across the 5 documentation sources. For completeness, we included data relating to 4 documentation sources only, excluding the DNR sheet because it is only relevant for DNR patients. We defined inconsistent code status documentation a priori as (1) the code status is conflicting in at least 2 documentation sources (eg, full code in 1 source and DNR in another) or (2) the code status is documented in 1 or more documentation source and not documented in at least 1 documentation source (eg, full code in 1 source and blank in another).
We then subdivided code status documentation inconsistencies into nonclinically relevant and clinically relevant subcategories. For example, a nonclinically relevant inconsistency would be if a physician documented full code in the physician orders, but a nurse did not document anything in the nursing‐care plan, because most providers would assume a preference for resuscitation in the absence of code status documentation in the nursing‐care plan.
We defined clinically relevant inconsistencies as those that would reasonably lead healthcare providers referring to different documentation sources to respond differently in the event of a cardiac arrest (eg, the physician orders show DNR whereas the nursing‐care plan is blanka provider who refers to the physician orders would not resuscitate the patient, but another provider who refers to the blank nursing‐care plan would resuscitate the patient).
We determined the proportion of patients with inconsistent code status documentation by listing the 31 different permutations of code status documentation in our data (Figure 1). Using the prespecified definition of inconsistent code status documentation, 3 study authors (I.A.D., B.M.W., R.C.W.) independently determined whether each permutation met the criteria for inconsistent code status documentation, and judged the clinical relevance of each documentation inconsistency. We resolved disagreements by consensus.
Statistical Analysis
We calculated descriptive statistics for all variables, summarizing continuous measures using means and standard deviations, and categorical measures using counts, percentages, and their associated 95% confidence intervals. Logistic regression analyses adjusting for the correlation among observations taken from the same team were carried out. Each of the 4 variables of interest (patient age, length of stay, receiving comfort measures, free text code status documentation) was run in a bivariate model to obtain unadjusted estimates as well as the final multivariable model. All estimates were displayed as odds ratios (ORs) and their associated 95% confidence intervals (CIs). A P value <0.05 was used to denote statistical significance. We also carried out a kappa analysis to assess inter‐rater agreement when judging whether inconsistent documentation is clinically relevant. All analyses were carried out using SAS version 9.3 (SAS Institute, Cary, NC).
RESULTS
There were 194 patients potentially eligible for inclusion. Seven admitted GIM patients who had not been transferred from the emergency department were excluded, leaving 187 patients in the study. The mean patient age was 70 years; 83 (44%) were female. The median length of stay up to the time of data collection was 6 days, with the majority (156 [83%]) of patients admitted to their home ward. Ten (5%) patients were receiving comfort measures.
Completeness of Code Status Documentation
Thirty‐eight (20%; 95% CI, 14%‐26%) patients had complete and consistent code status documentation across all documentation sources, whereas 27 (14%; 95% CI, 9%‐19%) patients had no code status documented in any documentation source. By documentation source, providers documented code status in the progress notes for 89 patients (48%; 95% CI, 40%‐55%), the physician orders for 107 patients (57%; 95% CI, 50%‐64%), the nursing‐care plan for 110 patients (59%; 95% CI, 51%‐66%), and the electronic sign‐out list for 129 patients (69%; 95% CI, 62%‐76%).
Consistency of Code Status Documentation
The remaining 122 patients (65%; 95% CI, 58%‐72%) had at least 1 code status documentation inconsistency. Of these, 38 patients (20%; 95% CI, 14%‐26%) had a clinically relevant code status documentation inconsistency. Code status documentation inconsistency differed by site; the 2 hospitals with paper‐based physician orders had fewer patients with complete and consistent code status documentation compared to the hospital where physician orders are electronic (15% vs 42%, respectively, P<0.001) (Table 1).
| Physician Code Status Order | Sites 1 & 2: Paper‐Based, N=108 | Site 3: Electronic, N=52 | P Value |
|---|---|---|---|
| Consistent code status documentation | 16 (15%) | 22 (42%) | |
| Inconsistent code status documentation, nonclinically relevant | 60 (56%) | 30 (58%) | <0.0001 |
| Inconsistent code status documentation, clinically relevant | 32 (30%) | 6 (12%) |
The permutations of clinically relevant and nonclinically relevant inconsistencies are summarized in Figure 1. We achieved high inter‐rater reliability among the 3 independent reviewers with respect to rating the clinical relevance of documentation inconsistencies (=0.86 [95% CI, 0.76‐0.95]).
To identify correlates of clinically relevant code status documentation inconsistencies, we included 4 variables of interest (patient age, length of stay, receiving comfort measures, and free text code status documentation) in a logistic regression analysis. Bivariate analyses demonstrated that increased age (OR =1.07 [95% CI, 1.05‐1.10] for every 1‐year increase in age, P<0.001) and receiving comfort measures (OR= 10.98 [95% CI, 1.94‐62.12], P=0.007) were associated with a clinically relevant code status documentation inconsistency. Using these 4 variables in a multivariable analysis clustering for physician team, increased age (OR=1.07 [95% CI, 1.04‐1.10] for every 1‐year increase in age, P<0.0001) and receiving comfort measures (OR=9.39 [95% CI, 1.3565.19], P=0.02) remained as independent positive correlates of having a clinically relevant code status documentation inconsistency (Table 2).
| Clinically Relevant Inconsistencies, N=38 | No Inconsistencies and Nonclinically Relevant Inconsistencies, N=149 | P Value | |
|---|---|---|---|
| |||
| Age, y, mean (SD) | 83 (10) | 67 (19) | <0.0001* |
| Length of stay, d, median (IQR) | 6.5 (310) | 6 (219) | 0.39 |
| Receiving comfort measures, n (%) | 7 (18%) | 3 (2%) | <0.0001* |
| Free‐text code status documentation, n (%) | 18 (47%) | 58 (39%) | 0.34 |
DISCUSSION
We found that 2 out of 3 patients had at least 1 inconsistency in code status documentation, and that 1 in 5 patients had at least 1 clinically relevant code status documentation inconsistency. The majority of clinically relevant inconsistencies occurred because there was a DNR order written in some sources of code status documentation, and no orders in other documentation sources. However, there were 4 striking examples where DNR was written in some sources of code status documentation and full code was written in other documentation sources (Figure 1).
Older patients and patients receiving comfort measures were more likely to have a clinically relevant inconsistency in code status documentation. This is particularly concerning, because they are among the most vulnerable patients at highest risk for having an in‐hospital cardiac arrest.
Our study extends the findings of prior studies that identified gaps in completeness and accuracy of code status documentation and describes another important gap in the quality and consistency of code status documentation.[20] This becomes particularly important because efforts aimed at increasing documentation of patients' code status without ensuring consistency across documentation sources may still result in patients being resuscitated or not resuscitated inappropriately.
This issue of poorly integrated health records is relevant for many other aspects of patient care. For example, 1 study found significant discrepancies in patient medication allergy documentation across multiple health records.[21] This fragmentation of documentation of the same patient information in multiple health records requires attention and should be the focus of institutional quality improvement efforts.
There are several potential ways to improve the code status documentation process. First, the use of standard fields or standardized orders can increase the completeness and improve the clarity of code status documentation.[22, 23] For institutions with an electronic medical record, forcing functions may further increase code status documentation. One study found that the implementation of an electronic medical record increased code status documentation from 4% to 63%.[24] We found similarly that the site with electronic physician orders had higher rates of complete and consistent code status documentation.
A second approach is to minimize the number of different sources for code status documentation. Institutions should critically examine each place where providers could document code status and decide whether this information adds value, and create policies to restrict unnecessary duplicate documentation and ensure accurate documentation of code status in several key, centralized locations.[25] A third option would be to automatically synchronize all code status documentation sources.[25] This final approach requires a fully integrated electronic health record.
Our study has several limitations. Although we report a large number of code status documentation inconsistencies, we do not know how many of these lead to incorrect resuscitative measures, so the actual impact on patient care is unknown. Also, because we were focusing on inconsistencies among sources of code status documentation, and not on accurate documentation of a patients' code status, the patients' actual preferences were not elicited and are not known. Finally, we carried out our study in 3 AMCs with residents that rotate from 1 site to another. The transient nature of resident work may increase the likelihood of documentation inconsistencies, because trainees may be less aware of local processes. In addition, the way front‐line staff uses clinical documentation sources to determine a patient's code status may differ at other institutions. Therefore, our estimate of clinical relevance may not be generalizable to other institutions with different front‐line processes or with healthcare teams that are more stable and aware of local documentation processes.
In summary, our study uncovered significant gaps in the quality of code status documentation that span 3 different AMCs. Having multiple, poorly integrated sources for code status documentation leads to a significant number of concerning inconsistencies that create opportunities for healthcare providers to inappropriately deliver or withhold resuscitative measures that conflict with patients' expressed wishes. Institutions need to be aware of this potential documentation hazard and take steps to minimize code status documentation inconsistencies. Even though cardiac arrests occur infrequently, if healthcare teams take inappropriate action because of these code status documentation inconsistencies, the consequences can be devastating.
Disclosure
Nothing to report.
- , , , et al. Advance directives for medical care—a case for greater use. N Engl J Med. 1991;324(13):889–895.
- , , , et al; Canadian Researchers at the End of Life Network (CARENET). What matters most in end‐of‐life care: perceptions of seriously ill patients and their family memebrs. CMAJ. 2006;174(5):627–633.
- , , . Advance directives and outcomes of surrogate decision making before death. N Eng J Med. 2010;362(13):1211–1218.
- , , , et al. Associations between end‐of‐life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment. JAMA. 2008;300(14):1665–1673.
- , , , . The impact of advance care planning on end of life care in elderly patients: randomized controlled trial. BMJ. 2010;340:c1345.
- , . Quality indicators for end‐of‐life care in vulnerable elders. Ann Intern Med. 2001;135(8):667–685.
- , , , . Assessing care of vulnerable elders: methods for developing quality indicators. Ann Intern Med. 2001;135:647–652.
- , , , et al. Perceptions by family members of the dying experience of older and seriously ill patients. Ann Intern Med. 1997;126(2):97–106.
- , . Cardiopulmonary resuscitation: capacity, discussion and documentation. Q J Med. 2006;99(10):683–690.
- , , , et al. Code status documentation in the outpatient electronic medical records or patients with metastatic cancer. J Gen Intern Med. 2009;25(2):150–153.
- , , , et al. Factors associated with discussion of care plans and code status at the time of hospital admission: results from the multicenter hospitalist study. J Hosp Med. 2008;3(6):437–445.
- , , , . Documentation quality of inpatient code status discussions. J Pain Symptom Manage. 2014;48(4):632–638.
- , , , . The use of life‐sustaining treatments in hospitalized persons aged 80 and older. J Am Geriatr Soc. 2002;50(5):930–934.
- The SUPPORT Principal Investigators. A controlled trial to improve care for seriously ill hospitalized patients: the Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments (SUPPORT). JAMA. 1995;274(20):1591–1598.
- , , . Some treatment‐withholding implications of no‐code orders in an academic hospital. Crit Care Med. 1984;12(10):879–881.
- . A prospective study of patients with DNR orders in a teaching hospital. Arch Intern Med. 1998;148(10):2193–2198.
- , . Compliance with do‐not‐resuscitate orders for hospitalized patient transported to radiology department. Ann Int Med. 1998;129(10):801–805.
- , . Identification of inpatient DNR status: a safety hazard begging for standardization. J Hosp Med. 2007;2(6):366–371.
- . Controlling death: the false promise of advance directives. Ann Int Med. 2007;147(1):51–57.
- , , , et al. Failure to engage hospitalized elderly patients and their families in advance care planning. JAMA Intern Med. 2013;173(9):778–787.
- , , , et al. The recording of drug sensitivities for older people living in care homes. Br J Clin Pharmacol. 2010;69(5):553–557.
- , , , , , . A comparison of methods to communicate treatment preferences in nursing facilities: traditional practices versus the physician orders for life‐sustaining treatment program. J Am Geriatr Soc. 2010;58(7):1241–1248.
- , , , et al. Evaluation of a treatment limitation policy with a specific treatment‐limiting order page. Arch Intern Med. 1994;154(4):425–432.
- , , , et al. An electronic medical record intervention increased nursing home advance directive orders and documentation. J Am Geriatr Soc. 2007;55(7):1001–1006.
- , , , et al. Honouring patient's resuscitation wishes: a multiphased effort to improve identification and documentation. BMJ Quality Safety. 2013;22(1):85–92.
- , , , et al. Advance directives for medical care—a case for greater use. N Engl J Med. 1991;324(13):889–895.
- , , , et al; Canadian Researchers at the End of Life Network (CARENET). What matters most in end‐of‐life care: perceptions of seriously ill patients and their family memebrs. CMAJ. 2006;174(5):627–633.
- , , . Advance directives and outcomes of surrogate decision making before death. N Eng J Med. 2010;362(13):1211–1218.
- , , , et al. Associations between end‐of‐life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment. JAMA. 2008;300(14):1665–1673.
- , , , . The impact of advance care planning on end of life care in elderly patients: randomized controlled trial. BMJ. 2010;340:c1345.
- , . Quality indicators for end‐of‐life care in vulnerable elders. Ann Intern Med. 2001;135(8):667–685.
- , , , . Assessing care of vulnerable elders: methods for developing quality indicators. Ann Intern Med. 2001;135:647–652.
- , , , et al. Perceptions by family members of the dying experience of older and seriously ill patients. Ann Intern Med. 1997;126(2):97–106.
- , . Cardiopulmonary resuscitation: capacity, discussion and documentation. Q J Med. 2006;99(10):683–690.
- , , , et al. Code status documentation in the outpatient electronic medical records or patients with metastatic cancer. J Gen Intern Med. 2009;25(2):150–153.
- , , , et al. Factors associated with discussion of care plans and code status at the time of hospital admission: results from the multicenter hospitalist study. J Hosp Med. 2008;3(6):437–445.
- , , , . Documentation quality of inpatient code status discussions. J Pain Symptom Manage. 2014;48(4):632–638.
- , , , . The use of life‐sustaining treatments in hospitalized persons aged 80 and older. J Am Geriatr Soc. 2002;50(5):930–934.
- The SUPPORT Principal Investigators. A controlled trial to improve care for seriously ill hospitalized patients: the Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments (SUPPORT). JAMA. 1995;274(20):1591–1598.
- , , . Some treatment‐withholding implications of no‐code orders in an academic hospital. Crit Care Med. 1984;12(10):879–881.
- . A prospective study of patients with DNR orders in a teaching hospital. Arch Intern Med. 1998;148(10):2193–2198.
- , . Compliance with do‐not‐resuscitate orders for hospitalized patient transported to radiology department. Ann Int Med. 1998;129(10):801–805.
- , . Identification of inpatient DNR status: a safety hazard begging for standardization. J Hosp Med. 2007;2(6):366–371.
- . Controlling death: the false promise of advance directives. Ann Int Med. 2007;147(1):51–57.
- , , , et al. Failure to engage hospitalized elderly patients and their families in advance care planning. JAMA Intern Med. 2013;173(9):778–787.
- , , , et al. The recording of drug sensitivities for older people living in care homes. Br J Clin Pharmacol. 2010;69(5):553–557.
- , , , , , . A comparison of methods to communicate treatment preferences in nursing facilities: traditional practices versus the physician orders for life‐sustaining treatment program. J Am Geriatr Soc. 2010;58(7):1241–1248.
- , , , et al. Evaluation of a treatment limitation policy with a specific treatment‐limiting order page. Arch Intern Med. 1994;154(4):425–432.
- , , , et al. An electronic medical record intervention increased nursing home advance directive orders and documentation. J Am Geriatr Soc. 2007;55(7):1001–1006.
- , , , et al. Honouring patient's resuscitation wishes: a multiphased effort to improve identification and documentation. BMJ Quality Safety. 2013;22(1):85–92.
© 2015 Society of Hospital Medicine
Educational Impact of Smartphones
Medical residents are rapidly adopting smartphones. Recent statistics revealed that 85% of medical providers currently own a smartphone, and the majority use it in their clinical work.[1] Smartphone capabilities that include the use of text messaging, e‐mail, and mobile phone functions in the clinical setting may improve efficiency and quality of care by reducing the response time for urgent issues.[2] There is, however, increasing recognition that healthcare information technology can create unintended negative consequences. For example, studies have suggested that healthcare information technologies, such as the computerized physician order entry, may actually increase errors by creating new work, changing clinical workflow, and altering communication patterns.[3, 4, 5]
Smartphone use for clinical communication can have unintended consequences by increasing interruptions, reducing interprofessional relationships, and widening the gap between what nurses and physicians perceive as urgent clinical problems.[6] However, no studies have evaluated the impact of smartphones on the educational experience of medical trainees. Although previous studies have described the use of smartphones by trainees for rapid access to electronic medical resources,[7, 8, 9] we did not identify in our literature review any previous studies on the impact of using the smartphone's primary functionas a communication deviceon the educational experience of residents and medical students. Therefore, our study aimed to examine the impact of using smartphones for clinical communication on medical education.
METHODS
Design
The design of the study was qualitative research methodology using interview data, ethnographic data, and content analysis of text‐based messages.
Setting
From June 2009 to September 2010, we conducted a multisite evaluation study on general internal medicine (GIM) wards at 5 large academic teaching hospitals in the city of Toronto, Canada at St. Michael's Hospital, Sunnybrook Health Sciences Centre, Toronto General Hospital, Toronto Western Hospital, and Mount Sinai Hospital. Each hospital has clinical teaching units consisting typically of 4 medical teams. Each team includes 1 attending physician, 1 senior resident, 2 or more junior residents, and 2 to 4 medical students. Each hospital had 2 to 4 GIM wards in different geographic locations.
Communication Systems
To make it easier for nurses and other health professionals to communicate with the physician teams, all sites centralized communication to 1 team member, who acts as the single point of contact on behalf of their assigned team in the communication of patient‐related issues. We facilitated this communication through a shared device (either a pager or a smartphone). The senior resident typically carried the shared device during the day and the on‐call junior resident at night and on the weekends. Two hospitals provided smartphones to all residents, whereas a third site provided smartphones only to the senior residents. The standard processes of communication required that physicians respond to all calls and text messages. At the 3 sites with institutional smartphones, nurses could send text messages with patient information using a Web‐based system. We encrypted data sent to institutional smartphones to protect patient information.
Data Collection
Using a mixed‐methods ethnographic approach, we collected data using semistructured interviews, ethnographic observations, and content analysis of text messages. The original larger study focused primarily on examining the overall clinical impact of smartphone use.[10] For our current study, we analyzed the data with a focus on evaluating the impact of smartphones on the educational experience of medical trainees on the GIM teaching service. The respective institutions' research ethics boards approved the study.
Interviews
We conducted semistructured interviews with residents, medical students, attending physicians, and other clinicians across all of the sites to examine how clinicians perceived the impact of smartphones on medical education. We used a purposeful sampling strategy where we interviewed different groups of healthcare professionals who we suspected would represent different viewpoints on the use of smartphones for clinical communication. To obtain diverse perspectives, we snowball sampled by asking interviewees to suggest colleagues with differing views to participate in the interviews. The interview guide consisted of open‐ended questions with additional probes to elicit more detailed information from these frontline clinicians who initiate and receive communication. One of the study investigators (V.L.) conducted interviews that varied from 15 to 45 minutes in duration. We recorded, transcribed verbatim, and analyzed the interviews using NVivo software (QSR International, Doncaster, Victoria, Australia). We added additional questions iteratively as themes emerged from the initial interviews. One of the study investigators (V.L.) encouraged participants to speak freely, to raise issues that they perceived to be important, and to support their responses with examples.
Observations
We observed the communication processes in the hospitals by conducting a work‐shadowing approach that followed individual residents in their work environments. These observations included 1‐on‐1 supervision encounters involving attending staff, medical students, and other residents, and informal and formal teaching rounds. The observation periods included the usual working day (from 8 am to 6 pm) as well as the busiest times on call, typically from 6 pm until 11 pm. We sampled different residents for different time periods. We adopted a nonparticipatory observation technique where we observed all interruptions, communication interactions, and patterns from a distance. We defined workflow interruptions as an intrusion of an unplanned and unscheduled task, causing a discontinuation of tasks, a noticeable break, or task switch behaviour.[11] Data collection included timing of events and writing field notes. One of the study investigators (V.L.) performed all the work‐shadowing observations.
E‐mail
To study the volume and content of messages, we collected e‐mail communications between January 2009 and June 2009 from consenting residents at the 2 hospitals that provided smartphones to all GIM residents. E‐mail information included the sender, the receiver, the time of message, and the message content. To look at usage, we calculated the average number of e‐mails sent and received. To assess interruptions on formal teaching sessions, we paid particular attention to e‐mails received and sent during protected educational timeweekdays from 8 am to 9 am (morning report) and 12 pm to 1 pm (noon rounds). We randomly sampled 20% of all e‐mails sent between residents for content analysis and organized content related to medical education into thematic categories.
Analysis
We used a deductive approach to analyze the interview transcripts by applying a conceptual framework that assessed the educational impact of patient safety interventions.[12] This framework identified 5 educational domains (learning, teaching, supervision, assessment, and feedback). Three study investigators mapped interview data, work‐shadowing data, and e‐mail content to themes (V.L., B.W., and R.W.), and grouped data that did not translate into themes into new categories. We then triangulated the data to develop themes of the educational impact of smartphone communication by both perceived use and actual use, and subsequently constructed a framework of how smartphone communication affected education.
RESULTS
We conducted 124 semistructured interviews with residents, medical students, attending physicians, and other clinicians across all the sites to examine how clinicians perceived the impact of smartphones on medical education. We work‐shadowed 40 individual residents for a total of 196 hours (Table 1). We analyzed the 13,714 e‐mails sent from or received to 34 residents. To analyze e‐mail content, we reviewed 1179 e‐mails sent among residents.
| Methods | Sites | |||||
|---|---|---|---|---|---|---|
| St. Michael's Hospital | Sunnybrook Health Sciences Centre | Toronto General Hospital | Toronto Western Hospital | Mount Sinai Hospital | All Hospitals | |
| ||||||
| Work‐shadowing residents | ||||||
| Hours | 60 hours | 35 hours | 57 hours 55 minutes | 27 hours 46 minutes | 15 hours | 196 hours |
| No. of residents | 12 | 7 | 12 | 6 | 3 | 40 |
| Interviews with clinicians | ||||||
| Physicians | 10 | 5 | 13 | 5 | 33 | |
| Medical students | 5 | 4 | 1 | 1 | 11 | |
| Nurses | 9 | 11 | 15 | 14 | 49 | |
| Other health professionsa | 7 | 10 | 8 | 6 | 31 | |
| Total | 31 | 30 | 37 | 26 | 124 | |
We found that 2 key characteristics of smartphone use for clinical communication, namely an increase in connectedness leading to an increase in interruptions, impacted 3 educational domains: teaching, supervision, and professionalism (Figure 1).
Increased Connectedness
As a communication device, smartphones increase the ability to receive and respond to messages through voice, e‐mail, and text messaging. Not surprisingly, with the improved ability and mobility to communicate, medical trainees perceived being more connected with their team members, who included other residents, medical students, and attending staff as well as with other clinical services and professions. These smartphone communication activities appeared to be pervasive, occurring on the wards, at the bedside, while in transit, and in teaching sessions (Box 1: increased connectedness).
Box
Increased connectedness
I've used the Blackberry system and it's nice to be able to quickly text each other little messages especially for meeting times because then you don't have to page them and wait by the phone. So that's been great for in the team. (Interview Resident 3)
It's incredibly useful for when you're paging somebody else. Often times I'll be consulting with another physician on a patient and I'll say This is my BlackBerry. Call me back after you've seen the patient' or Call me back when you have a plan' or, you know, whatever. So that's extremely valuable which we never had with pages and no one would ever page you for that because it was too much of a pain. (Interview Resident 1)
My personal experience has been that if you need to speak to a more senior individual it's much easier to contact them via the BlackBerry. (Interview Medical Student 1)
At 7:25 pm, MD11 returns to the patient's room and continues examining her. While in the patient's room, I could see her talking on the BlackBerrys. I asked her later what calls she had while in the room. It turns out she had 3 phone calls and 2 texts. Two of the calls were from the radiation oncologists and 1 call from the pathologist. She also received 1 text on the Team BlackBerry and 1 text on the Senior's BlackBerry from the pharmacist. (Field Notes, Work-shadowing MD11)
Interruptions
The increased connectedness caused by smartphone use led residents to perceive an increase in the frequency of interruptions. The multitude of communication and contact options made available by smartphones to health providers created an expansive network of connected individuals who were in constant communication with each other. Instead of the difficulties associated with numeric paging and waiting for a response, nurses typically found it easier to call directly or send a text message to residents' smartphones. From the e‐mail analysis, residents received, on a daily basis, on average 25.7 e‐mails, (median, 20; interquartile range [IQR]: 1428) to the team smartphone and sent 7.5 e‐mails (median, 6; IQR: 410). During protected educational time, each resident received an average of 1.0 e‐mail (median, 1; IQR: 01) between 8 am and 9 am and an average of 2.3 e‐mails (median, 2; IQR: 13) during 12 pm to 1 pm (Figure 2). Each of these communication events, whether a phone call, e‐mail, or text‐message, led to an interruption (Box 2). Given that smartphones made it easier for nurses to contact residents, some residents attributed the increase in interruptions to a reduction in the threshold for nurses to communicate.
Box
Increased interruptions
The only negative I can think of is just the incredible number of communications that you get, you know, text messages and e‐mails and everything else. So just the, the number can sometimes be overwhelming. (Interview Resident 1)
Some of [the nurses] rely a little bit more on the BlackBerry so that they will tend to call you a bit more frequently for things that maybe sometimes they should try to find answer for themselves (Interview Resident 2)
And now with the option of being able to, if you really needed to, call them and talk to them directly, I think that kind of improves communication. They're easier to find. (Interview Nurse 4)
Supervision
Smartphone communication appeared to positively impact trainee supervision. Increased connectedness between team members allowed junior trainees to have access and rapidly communicate with a more experienced clinician, which provided them with greater support. Residents found smartphones particularly useful in situations where they felt uncomfortable or where they did not feel competent. Some of these instances related to procedural competence, with residents feeling more comfortable knowing they have rapid access to support (Box 3: increased support).
Box
Supervision
Increased support
It makes me feel more comfortable in the sense that I can instantly make a call or a text and have a question answered if I need an answer. Or if it were an emergency having the ability to talk on the phone and be talked through an emergency situation, or a procedure for example like if you were in a remote area or the physician was in a remote area and you were in hospital and you would need some of that guidance or counselling, there's no substitution. (Interview Medical Student 1)
I'm ready can u dblchk [sic] that I landmarked correctly. (Email from Junior to Senior)
MD3 returns to the patient's room to do a paracentesis with [junior resident]. He calls on his BB to [senior resident] to inform her that they are starting and then hangs up. [Senior resident] arrives at the patient's room. (Field Notes, Workshadowing MD3)
Decreased autonomy
The difference with the Blackberry is they're more likely to say By the way, this happened. Should I do this?' And I write back Yes', No.' If they didn't have that contact like I said they probably would have done something and then because they're making a decision on their own they could very easily have spent the time to research whatever to figure whether that was the right thing to do before doing it. Now they have an outlet where they can pass an idea off of me and then have me make, it's easier for me to make a decision for them. So that can negatively impact education. (Interview Attending 1)
What do I do for a high phosphate?(Email from Junior to Senior)
Hey Pt X's k is 5.5. Was going to shift her. What do u think? (Email from Junior to Senior)
You probably saw the hb 92. Let's give prbc asap while he's on HD.(Email from Staff to Residents on the team)
hb‐ hemoglobin, prbc packed red blood cells, HD ‐ hemodialysis
Hi. Just checking the bloodwork. What is happening to ms X? [sic] Creatinine rising still. Is a foley in? Urology reconsulted? (Email from Attending Staff to Junior Resident)
On the other hand, supervisors perceived that the easy rapid access afforded by smartphone use lowered the threshold for trainees to contact them. In some instances, these attending physicians felt that their trainees would text them for advice when they could have looked up the information themselves. As a result, the increased reliance on the attending physician's input prior to committing to a management plan decreased the trainee's autonomy and independent decision making (Box 3: decreased autonomy). In addition to trainee requests for increased staff involvement, smartphone use made it easier for attending physicians to initiate text messages to their residents as well. In some instances, staff physicians adopted a more hands‐on approach by directing their residents on how to manage their patients. It is unclear if trainees perceived this taking over of care as negatively influencing their education.
Teaching
Medical teams also frequently used smartphones to communicate the location and timing of educational rounds. We observed instances where residents communicated updated information relating to scheduled rounds, as well as for informing team members about spontaneous teaching sessions (Box 4: communicating rounds). Despite this initial benefit, staff physicians worried that interruptions resulting from smartphone use during educational sessions lowered the effectiveness of these sessions for all learners by creating a fragmented learning experience (Box 4: fragmented learning). Our data indicated that residents carrying the team smartphones received and sent a high number of e‐mails throughout the day, which continued at a similar rate during the protected educational time (Figure 2). Additionally, some of the teaching experiences that traditionally would occur in a face‐to‐face manner appeared to have migrated to text‐based interactions. It is unclear whether trainees perceive these text‐based interactions as more or less effective teaching encounters (Box 4: text‐based teaching).
Box
Teaching
Communicating rounds
One is that they can more efficiently communicate about the timing and location of education rounds in case they forget or sort of as an organizer for them (Interview Attending 3)
Physical Exam rounds is at 1:00 outside the morning report room. K. has kindly volunteered! If you miss us then the exam will be on the 3rd floor in room X. Pt X. See you there (Email from chief medical resident to trainees)
Fragmented learning
Because Blackberry is there, it's something that is potentially time occupying and can take the attention away from things and this is true of any Blackberries. People who have Blackberries they always look at their Blackberries so, you know, there are times when I'm sitting face to face with people and residents are looking at their Blackberries. So it's another way that they can be distracted. (Interview Attending 1).
I've seen that be an issue. I've certainly seen them losing concentration during a teaching session because they're being Blackberried, getting Blackberry messages. (Interview Attending 3)
2:06 Team meeting with Attending in a conference room.
2:29 Team BlackBerry (BB) beeps. Senior glances at BB. She dials a number on the Team BB. Speaks on the Team BB and turns to [Junior resident] to inform her that the family is here. She returns to the caller. Senior then hangs up and resumes to her teaching.
2:35 Attending's BB rings. She takes a look and ends the BB call.
2:39 Senior's BB rings. Senior picks up and talks about a patient's case and condition. Senior turns to [junior resident] and asks a question. Team members resume talking among themselves.
2:46 Senior hangs up on the phone call.
2:49 Team discusses another patient's condition/case.
2:57 Junior resident uses her BB to text.
3:02 Team BB beeps. It is a message about a patient's case.
3:05 Meeting ends. (Field Note excerpts, Work‐shadowing MD6)
Text‐based teaching
The resident would get very frustrated with how many questions we have once we've started. Like if three different medical students or four different medical students or four different places all texting him with, oh by the way, what does this stand for?, and he's responding to each of them individually then he has to answer it four different times as opposed to just in person when he can get us all together in a group and it's actually a learning experience. If questions are answered in an email, it's not really helpful for the rest of us. (Interview, medical student SB1)
That would be a great unifying diagnosis, but there may be some underlying element of ROH/NASH also I would hold off on A/C as we do not know if he has varices. Will need to review noncontrast CT ?HCC. Thx (Email from Consulting Staff to Junior)
A/C anticoagulation, CT computed tomography, HCC‐ hepatocellular carcinoma, NASH non‐alcoholic steatohepatitis, ROH alcohol
Professionalism
Our data revealed that smartphone interruptions occurred during teaching rounds and interactions with patients and with other clinical staff. Often these interruptions involved messages or phone calls pertaining to clinical concerns or tasks that nurses communicated to the residents via their smartphone (Box 5). Yet, by responding to these interruptions and initiating communications on their smartphones during patient care encounters and formal teaching sessions, trainees were perceived by other clinicians who were in attendance with them as being rude or disrespectful. Attending staff also tended to role model similar smartphone behaviors. Although we did not specifically work‐shadow attending staff, we did observe frequent usage of their personal smartphones during their interactions with residents.
Box
Professionalism
I don't like it when I see them checking messages when you're trying to talk to them. I think you're losing some of that communication sort of polite behaviour that maybe we knew a little bit more before all this texting and Blackberry. (Interview Allied Health 5)
I think that the etiquette of the Blackberry can be offensive, could be offensive especially with some of our older patients (Interview Allied Health 6)
Senior walks out of the patient's room while typing on the BlackBerry. She finishes typing and returns to the room at 5:36. Senior looks at her BlackBerry and starts typing inside the room in front of the patient. She paused to look at the patient and the resident doing the procedure [paracentesis]. She resumes texting again and walks out of the room at 5:38. Another resident walks out and Senior speaks with the resident. Senior returns to the room and speaks with the patient. She asks the patient if he has ever gotten a successful tap before. Senior looks at her BlackBerry and starts typing. (Field NotesWork‐shadowing MD2)
I think it is almost completely negative in terms of its medical education [Any positive] factors are grossly outweighed by the significant disruptions to their ability to concentrate and participate in the educational session. And I think almost to some extent it's an implicit permission that gets granted to the house staff to disrupt their own teaching experience and disrupt others around them because everybody is doing it because everybody is being Blackberried. So it almost becomes the new social norm and while that may be a new social norm I'm not sure that that's a good thing How big is the negative impact? That's much harder to say. It's probably not a big impact on top of the endless other disruptions in the day to teaching, but it is measurable because it's a new factor so it's observable by me on top of all the other factors which have been there for years. (Interview Attending 3)
2:10‐Attending goes to the whiteboard to teach research methods to the team. Spotted Medical student#1 looking at his IPhone and typing.
2:15‐Med student#1 using the calculator function on his IPhone.
2:20‐Attending glances at his BB quickly.
2:28‐Attending resumes discussion of the patients' cases (Field notes, Work‐shadowing MD7).
DISCUSSION
The educational impacts of smartphone use for communication appear to center on increased connectedness of medical trainees and increased interruptions, which have positive and negative impacts in the areas of teaching, supervision, and professionalism. Smartphone communication provided potential educational benefits through (1) safer supervision with rapid access to help and (2) easier coordination of teaching sessions. Threats to the educational experience included (1) a high level of interruptions to both teachers and learners, which may reduce the effectiveness of formal and informal teaching; (2) replacement of face‐to‐face teaching with texting; (3) a potential erosion of autonomy and independence due to easy access to supervisors and easy ability for supervisors to take over; and (4) professionalism issues with difficulties balancing between clinical service demands and communication during patient and interprofessional encounters.
This study is the first to describe the intersection of clinical communication with smartphones and medical education. A recent study found that residents reported high use of smartphones during rounds for patient care as well as personal issues.[13] We have previously described the perceived impacts of smartphones on clinical communication, which included improved efficiency but concerns for increased interruptions and threats to professionalism.[6] We also observed that sites that used smartphones had increased interruptions compared to those with just pagers.[10] We have also described the content of e‐mail messages between clinicians and found that all e‐mails from nurses to physicians involved clinical care, but e‐mail exchanges between physicians were split between clinical care (60.4%), coordination within the team (53.5%), medical education (9.4%), and social communication (3.9%).[14] This study adds to the literature by focusing on the impacts of smartphone use to medical education and describing the perceived and observed impacts. This study provides a further example of how healthcare information technology can cause unintended consequences on medical education and appear to relate to the linked impacts of increased connectivity and the increased interruptions.[3] In essence, the trainee becomes more global, less local. Being more global translates to increased connections with people separated in physical space. Yet, this increased global connectedness resulted in the trainee being less local, with attention diverted elsewhere, taking away from the quality of patient interactions and interactions with other interprofessional team members. It also reduces the effectiveness of educational sessions for all participants. Although the level of supervision and autonomy are independently related, being too connected to supervisors may lower trainee autonomy by reducing independent thinking around patient issues.[15] It may also move teaching and learning from face‐to‐face conversations to text‐based messages. Although there have been existing tensions between service delivery and medical education, increased connectedness may tilt the balance toward the demands of service delivery and efficiency optimization at the expense of the educational experience. Finally, smartphone use appeared to create an internal tension among trainees, who have to juggle balancing professional behaviors and expectations in their dual role as learner and care provider; it would be educationally unprofessional to interrupt a teaching session and respond to a text message. However, failing to respond to a nurse who has sent a message and is expecting a response would be clinically unprofessional.
To address these threats, we advocate improving systems and processes to reduce interruptions and provide education on the tensions created by increased connectedness. Smarter communication systems could limit interrupting messages to urgent messages and queue nonurgent messages.[16] They could also inform senders about protected educational time. Even more sophisticated systems could inform the sender on the status of the receiver. For example, systems could indicate if they are available or if they are busy in an educational session or an important meeting with a patient and their family. Processes to reduce interruptions include interprofessional consensus on what constitutes an urgent issue and giving explicit permission to learners to ignore their smartphones during educational sessions except for critical communications purposes. Finally, education around smartphone communication for both learners and teachers may help minimize threats to learner autonomy, to face‐to‐face teaching, and to professionalism.[17]
Our study has several limitations. We derived this information from a general study of the impact of smartphones on clinical communication. Our study can be seen as hypothesis generating, and further research is warranted to validate these findings. There may be limits to generalizability as all sites adopted similar communication processes that included centralizing communications to make it easier for senders to reach a responsible physician.
In conclusion, we have provided a summary of the impact of rapidly emerging information technology on the educational experience of medical trainees and identified both positive and negative impacts. Of note, the negative impacts appear to be related to being more global and less local and high interruptions. Further research is required to confirm these unintended consequences as well as to develop solutions to address them. Educators should be aware of these findings and the need to develop curriculum to address and manage the negative impacts of smartphone use in the clinical training environment.
Acknowledgments
Disclosure: Nothing to report.
- , . Smartphone app use among medical providers in ACGME training programs. J Med Syst. 2012;36:3135–3139.
- , , , et al. The use of smartphones for clinical communication on internal medicine wards. J Hosp Med. 2010;5:553–559.
- , , , , , . Anticipating and addressing the unintended consequences of health IT and policy: a report from the AMIA 2009 Health Policy Meeting. J Am Med Inform Assoc. 2011;18(1):82–90.
- , , , , . Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc. 2006;13(5):547–556.
- , , , . “e‐Iatrogenesis”: the most critical unintended consequence of CPOE and other HIT. J Am Med Inform Assoc. 2007;14(3):387–388.
- , , , et al. An evaluation of the use of smartphones to communicate between clinicians: a mixed‐methods study. J Med Internet Res. 2011;13(3):e59.
- . Smartphones in clinical practice, medical education, and research. Arch Intern Med. 2011;171(14):1294–1296.
- , , , . Use of handheld computers in medical education. A systematic review. J Gen Intern Med. 2006;21(5):531–537.
- , , , . A review and a framework of handheld computer adoption in healthcare. Int J Med Inform. 2005;74(5):409–422.
- , , , et al. The intended and unintended consequences of communication systems on General Internal Medicine inpatient care delivery: a prospective observational case study of five teaching hospitals [published online ahead of print January 25, 2013]. J Am Med Inform Assoc. doi:10.1136/amiajnl‐2012‐001160.
- , , , , . Hospital doctors' workflow interruptions and activities: an observation study. BMJ Qual Saf. 2011;20(6):491–497.
- , , , et al. Computerised provider order entry and residency education in an academic medical centre. Med Educ. 2012;46:795–806.
- , , , . Smartphone use during inpatient attending rounds: Prevalence, patterns and potential for distraction. J Hosp Med. 2012;7(8):595–599.
- , , , et al. Understanding interprofessional communication: a content analysis of email communications between doctors and nurses. Applied Clinical Informatics. 2012;3:38–51.
- , , , . Preserving professional credibility: grounded theory study of medical trainees' requests for clinical support. BMJ. 2009;338:b128.
- , , , , . Beyond paging: building a web‐based communication tool for nurses and physicians. J Gen Intern Med. 2009;24(1):105–110.
- , . Distracted doctoring: smartphones before patients? CMAJ. 2012;184:1440.
Medical residents are rapidly adopting smartphones. Recent statistics revealed that 85% of medical providers currently own a smartphone, and the majority use it in their clinical work.[1] Smartphone capabilities that include the use of text messaging, e‐mail, and mobile phone functions in the clinical setting may improve efficiency and quality of care by reducing the response time for urgent issues.[2] There is, however, increasing recognition that healthcare information technology can create unintended negative consequences. For example, studies have suggested that healthcare information technologies, such as the computerized physician order entry, may actually increase errors by creating new work, changing clinical workflow, and altering communication patterns.[3, 4, 5]
Smartphone use for clinical communication can have unintended consequences by increasing interruptions, reducing interprofessional relationships, and widening the gap between what nurses and physicians perceive as urgent clinical problems.[6] However, no studies have evaluated the impact of smartphones on the educational experience of medical trainees. Although previous studies have described the use of smartphones by trainees for rapid access to electronic medical resources,[7, 8, 9] we did not identify in our literature review any previous studies on the impact of using the smartphone's primary functionas a communication deviceon the educational experience of residents and medical students. Therefore, our study aimed to examine the impact of using smartphones for clinical communication on medical education.
METHODS
Design
The design of the study was qualitative research methodology using interview data, ethnographic data, and content analysis of text‐based messages.
Setting
From June 2009 to September 2010, we conducted a multisite evaluation study on general internal medicine (GIM) wards at 5 large academic teaching hospitals in the city of Toronto, Canada at St. Michael's Hospital, Sunnybrook Health Sciences Centre, Toronto General Hospital, Toronto Western Hospital, and Mount Sinai Hospital. Each hospital has clinical teaching units consisting typically of 4 medical teams. Each team includes 1 attending physician, 1 senior resident, 2 or more junior residents, and 2 to 4 medical students. Each hospital had 2 to 4 GIM wards in different geographic locations.
Communication Systems
To make it easier for nurses and other health professionals to communicate with the physician teams, all sites centralized communication to 1 team member, who acts as the single point of contact on behalf of their assigned team in the communication of patient‐related issues. We facilitated this communication through a shared device (either a pager or a smartphone). The senior resident typically carried the shared device during the day and the on‐call junior resident at night and on the weekends. Two hospitals provided smartphones to all residents, whereas a third site provided smartphones only to the senior residents. The standard processes of communication required that physicians respond to all calls and text messages. At the 3 sites with institutional smartphones, nurses could send text messages with patient information using a Web‐based system. We encrypted data sent to institutional smartphones to protect patient information.
Data Collection
Using a mixed‐methods ethnographic approach, we collected data using semistructured interviews, ethnographic observations, and content analysis of text messages. The original larger study focused primarily on examining the overall clinical impact of smartphone use.[10] For our current study, we analyzed the data with a focus on evaluating the impact of smartphones on the educational experience of medical trainees on the GIM teaching service. The respective institutions' research ethics boards approved the study.
Interviews
We conducted semistructured interviews with residents, medical students, attending physicians, and other clinicians across all of the sites to examine how clinicians perceived the impact of smartphones on medical education. We used a purposeful sampling strategy where we interviewed different groups of healthcare professionals who we suspected would represent different viewpoints on the use of smartphones for clinical communication. To obtain diverse perspectives, we snowball sampled by asking interviewees to suggest colleagues with differing views to participate in the interviews. The interview guide consisted of open‐ended questions with additional probes to elicit more detailed information from these frontline clinicians who initiate and receive communication. One of the study investigators (V.L.) conducted interviews that varied from 15 to 45 minutes in duration. We recorded, transcribed verbatim, and analyzed the interviews using NVivo software (QSR International, Doncaster, Victoria, Australia). We added additional questions iteratively as themes emerged from the initial interviews. One of the study investigators (V.L.) encouraged participants to speak freely, to raise issues that they perceived to be important, and to support their responses with examples.
Observations
We observed the communication processes in the hospitals by conducting a work‐shadowing approach that followed individual residents in their work environments. These observations included 1‐on‐1 supervision encounters involving attending staff, medical students, and other residents, and informal and formal teaching rounds. The observation periods included the usual working day (from 8 am to 6 pm) as well as the busiest times on call, typically from 6 pm until 11 pm. We sampled different residents for different time periods. We adopted a nonparticipatory observation technique where we observed all interruptions, communication interactions, and patterns from a distance. We defined workflow interruptions as an intrusion of an unplanned and unscheduled task, causing a discontinuation of tasks, a noticeable break, or task switch behaviour.[11] Data collection included timing of events and writing field notes. One of the study investigators (V.L.) performed all the work‐shadowing observations.
E‐mail
To study the volume and content of messages, we collected e‐mail communications between January 2009 and June 2009 from consenting residents at the 2 hospitals that provided smartphones to all GIM residents. E‐mail information included the sender, the receiver, the time of message, and the message content. To look at usage, we calculated the average number of e‐mails sent and received. To assess interruptions on formal teaching sessions, we paid particular attention to e‐mails received and sent during protected educational timeweekdays from 8 am to 9 am (morning report) and 12 pm to 1 pm (noon rounds). We randomly sampled 20% of all e‐mails sent between residents for content analysis and organized content related to medical education into thematic categories.
Analysis
We used a deductive approach to analyze the interview transcripts by applying a conceptual framework that assessed the educational impact of patient safety interventions.[12] This framework identified 5 educational domains (learning, teaching, supervision, assessment, and feedback). Three study investigators mapped interview data, work‐shadowing data, and e‐mail content to themes (V.L., B.W., and R.W.), and grouped data that did not translate into themes into new categories. We then triangulated the data to develop themes of the educational impact of smartphone communication by both perceived use and actual use, and subsequently constructed a framework of how smartphone communication affected education.
RESULTS
We conducted 124 semistructured interviews with residents, medical students, attending physicians, and other clinicians across all the sites to examine how clinicians perceived the impact of smartphones on medical education. We work‐shadowed 40 individual residents for a total of 196 hours (Table 1). We analyzed the 13,714 e‐mails sent from or received to 34 residents. To analyze e‐mail content, we reviewed 1179 e‐mails sent among residents.
| Methods | Sites | |||||
|---|---|---|---|---|---|---|
| St. Michael's Hospital | Sunnybrook Health Sciences Centre | Toronto General Hospital | Toronto Western Hospital | Mount Sinai Hospital | All Hospitals | |
| ||||||
| Work‐shadowing residents | ||||||
| Hours | 60 hours | 35 hours | 57 hours 55 minutes | 27 hours 46 minutes | 15 hours | 196 hours |
| No. of residents | 12 | 7 | 12 | 6 | 3 | 40 |
| Interviews with clinicians | ||||||
| Physicians | 10 | 5 | 13 | 5 | 33 | |
| Medical students | 5 | 4 | 1 | 1 | 11 | |
| Nurses | 9 | 11 | 15 | 14 | 49 | |
| Other health professionsa | 7 | 10 | 8 | 6 | 31 | |
| Total | 31 | 30 | 37 | 26 | 124 | |
We found that 2 key characteristics of smartphone use for clinical communication, namely an increase in connectedness leading to an increase in interruptions, impacted 3 educational domains: teaching, supervision, and professionalism (Figure 1).
Increased Connectedness
As a communication device, smartphones increase the ability to receive and respond to messages through voice, e‐mail, and text messaging. Not surprisingly, with the improved ability and mobility to communicate, medical trainees perceived being more connected with their team members, who included other residents, medical students, and attending staff as well as with other clinical services and professions. These smartphone communication activities appeared to be pervasive, occurring on the wards, at the bedside, while in transit, and in teaching sessions (Box 1: increased connectedness).
Box
Increased connectedness
I've used the Blackberry system and it's nice to be able to quickly text each other little messages especially for meeting times because then you don't have to page them and wait by the phone. So that's been great for in the team. (Interview Resident 3)
It's incredibly useful for when you're paging somebody else. Often times I'll be consulting with another physician on a patient and I'll say This is my BlackBerry. Call me back after you've seen the patient' or Call me back when you have a plan' or, you know, whatever. So that's extremely valuable which we never had with pages and no one would ever page you for that because it was too much of a pain. (Interview Resident 1)
My personal experience has been that if you need to speak to a more senior individual it's much easier to contact them via the BlackBerry. (Interview Medical Student 1)
At 7:25 pm, MD11 returns to the patient's room and continues examining her. While in the patient's room, I could see her talking on the BlackBerrys. I asked her later what calls she had while in the room. It turns out she had 3 phone calls and 2 texts. Two of the calls were from the radiation oncologists and 1 call from the pathologist. She also received 1 text on the Team BlackBerry and 1 text on the Senior's BlackBerry from the pharmacist. (Field Notes, Work-shadowing MD11)
Interruptions
The increased connectedness caused by smartphone use led residents to perceive an increase in the frequency of interruptions. The multitude of communication and contact options made available by smartphones to health providers created an expansive network of connected individuals who were in constant communication with each other. Instead of the difficulties associated with numeric paging and waiting for a response, nurses typically found it easier to call directly or send a text message to residents' smartphones. From the e‐mail analysis, residents received, on a daily basis, on average 25.7 e‐mails, (median, 20; interquartile range [IQR]: 1428) to the team smartphone and sent 7.5 e‐mails (median, 6; IQR: 410). During protected educational time, each resident received an average of 1.0 e‐mail (median, 1; IQR: 01) between 8 am and 9 am and an average of 2.3 e‐mails (median, 2; IQR: 13) during 12 pm to 1 pm (Figure 2). Each of these communication events, whether a phone call, e‐mail, or text‐message, led to an interruption (Box 2). Given that smartphones made it easier for nurses to contact residents, some residents attributed the increase in interruptions to a reduction in the threshold for nurses to communicate.
Box
Increased interruptions
The only negative I can think of is just the incredible number of communications that you get, you know, text messages and e‐mails and everything else. So just the, the number can sometimes be overwhelming. (Interview Resident 1)
Some of [the nurses] rely a little bit more on the BlackBerry so that they will tend to call you a bit more frequently for things that maybe sometimes they should try to find answer for themselves (Interview Resident 2)
And now with the option of being able to, if you really needed to, call them and talk to them directly, I think that kind of improves communication. They're easier to find. (Interview Nurse 4)
Supervision
Smartphone communication appeared to positively impact trainee supervision. Increased connectedness between team members allowed junior trainees to have access and rapidly communicate with a more experienced clinician, which provided them with greater support. Residents found smartphones particularly useful in situations where they felt uncomfortable or where they did not feel competent. Some of these instances related to procedural competence, with residents feeling more comfortable knowing they have rapid access to support (Box 3: increased support).
Box
Supervision
Increased support
It makes me feel more comfortable in the sense that I can instantly make a call or a text and have a question answered if I need an answer. Or if it were an emergency having the ability to talk on the phone and be talked through an emergency situation, or a procedure for example like if you were in a remote area or the physician was in a remote area and you were in hospital and you would need some of that guidance or counselling, there's no substitution. (Interview Medical Student 1)
I'm ready can u dblchk [sic] that I landmarked correctly. (Email from Junior to Senior)
MD3 returns to the patient's room to do a paracentesis with [junior resident]. He calls on his BB to [senior resident] to inform her that they are starting and then hangs up. [Senior resident] arrives at the patient's room. (Field Notes, Workshadowing MD3)
Decreased autonomy
The difference with the Blackberry is they're more likely to say By the way, this happened. Should I do this?' And I write back Yes', No.' If they didn't have that contact like I said they probably would have done something and then because they're making a decision on their own they could very easily have spent the time to research whatever to figure whether that was the right thing to do before doing it. Now they have an outlet where they can pass an idea off of me and then have me make, it's easier for me to make a decision for them. So that can negatively impact education. (Interview Attending 1)
What do I do for a high phosphate?(Email from Junior to Senior)
Hey Pt X's k is 5.5. Was going to shift her. What do u think? (Email from Junior to Senior)
You probably saw the hb 92. Let's give prbc asap while he's on HD.(Email from Staff to Residents on the team)
hb‐ hemoglobin, prbc packed red blood cells, HD ‐ hemodialysis
Hi. Just checking the bloodwork. What is happening to ms X? [sic] Creatinine rising still. Is a foley in? Urology reconsulted? (Email from Attending Staff to Junior Resident)
On the other hand, supervisors perceived that the easy rapid access afforded by smartphone use lowered the threshold for trainees to contact them. In some instances, these attending physicians felt that their trainees would text them for advice when they could have looked up the information themselves. As a result, the increased reliance on the attending physician's input prior to committing to a management plan decreased the trainee's autonomy and independent decision making (Box 3: decreased autonomy). In addition to trainee requests for increased staff involvement, smartphone use made it easier for attending physicians to initiate text messages to their residents as well. In some instances, staff physicians adopted a more hands‐on approach by directing their residents on how to manage their patients. It is unclear if trainees perceived this taking over of care as negatively influencing their education.
Teaching
Medical teams also frequently used smartphones to communicate the location and timing of educational rounds. We observed instances where residents communicated updated information relating to scheduled rounds, as well as for informing team members about spontaneous teaching sessions (Box 4: communicating rounds). Despite this initial benefit, staff physicians worried that interruptions resulting from smartphone use during educational sessions lowered the effectiveness of these sessions for all learners by creating a fragmented learning experience (Box 4: fragmented learning). Our data indicated that residents carrying the team smartphones received and sent a high number of e‐mails throughout the day, which continued at a similar rate during the protected educational time (Figure 2). Additionally, some of the teaching experiences that traditionally would occur in a face‐to‐face manner appeared to have migrated to text‐based interactions. It is unclear whether trainees perceive these text‐based interactions as more or less effective teaching encounters (Box 4: text‐based teaching).
Box
Teaching
Communicating rounds
One is that they can more efficiently communicate about the timing and location of education rounds in case they forget or sort of as an organizer for them (Interview Attending 3)
Physical Exam rounds is at 1:00 outside the morning report room. K. has kindly volunteered! If you miss us then the exam will be on the 3rd floor in room X. Pt X. See you there (Email from chief medical resident to trainees)
Fragmented learning
Because Blackberry is there, it's something that is potentially time occupying and can take the attention away from things and this is true of any Blackberries. People who have Blackberries they always look at their Blackberries so, you know, there are times when I'm sitting face to face with people and residents are looking at their Blackberries. So it's another way that they can be distracted. (Interview Attending 1).
I've seen that be an issue. I've certainly seen them losing concentration during a teaching session because they're being Blackberried, getting Blackberry messages. (Interview Attending 3)
2:06 Team meeting with Attending in a conference room.
2:29 Team BlackBerry (BB) beeps. Senior glances at BB. She dials a number on the Team BB. Speaks on the Team BB and turns to [Junior resident] to inform her that the family is here. She returns to the caller. Senior then hangs up and resumes to her teaching.
2:35 Attending's BB rings. She takes a look and ends the BB call.
2:39 Senior's BB rings. Senior picks up and talks about a patient's case and condition. Senior turns to [junior resident] and asks a question. Team members resume talking among themselves.
2:46 Senior hangs up on the phone call.
2:49 Team discusses another patient's condition/case.
2:57 Junior resident uses her BB to text.
3:02 Team BB beeps. It is a message about a patient's case.
3:05 Meeting ends. (Field Note excerpts, Work‐shadowing MD6)
Text‐based teaching
The resident would get very frustrated with how many questions we have once we've started. Like if three different medical students or four different medical students or four different places all texting him with, oh by the way, what does this stand for?, and he's responding to each of them individually then he has to answer it four different times as opposed to just in person when he can get us all together in a group and it's actually a learning experience. If questions are answered in an email, it's not really helpful for the rest of us. (Interview, medical student SB1)
That would be a great unifying diagnosis, but there may be some underlying element of ROH/NASH also I would hold off on A/C as we do not know if he has varices. Will need to review noncontrast CT ?HCC. Thx (Email from Consulting Staff to Junior)
A/C anticoagulation, CT computed tomography, HCC‐ hepatocellular carcinoma, NASH non‐alcoholic steatohepatitis, ROH alcohol
Professionalism
Our data revealed that smartphone interruptions occurred during teaching rounds and interactions with patients and with other clinical staff. Often these interruptions involved messages or phone calls pertaining to clinical concerns or tasks that nurses communicated to the residents via their smartphone (Box 5). Yet, by responding to these interruptions and initiating communications on their smartphones during patient care encounters and formal teaching sessions, trainees were perceived by other clinicians who were in attendance with them as being rude or disrespectful. Attending staff also tended to role model similar smartphone behaviors. Although we did not specifically work‐shadow attending staff, we did observe frequent usage of their personal smartphones during their interactions with residents.
Box
Professionalism
I don't like it when I see them checking messages when you're trying to talk to them. I think you're losing some of that communication sort of polite behaviour that maybe we knew a little bit more before all this texting and Blackberry. (Interview Allied Health 5)
I think that the etiquette of the Blackberry can be offensive, could be offensive especially with some of our older patients (Interview Allied Health 6)
Senior walks out of the patient's room while typing on the BlackBerry. She finishes typing and returns to the room at 5:36. Senior looks at her BlackBerry and starts typing inside the room in front of the patient. She paused to look at the patient and the resident doing the procedure [paracentesis]. She resumes texting again and walks out of the room at 5:38. Another resident walks out and Senior speaks with the resident. Senior returns to the room and speaks with the patient. She asks the patient if he has ever gotten a successful tap before. Senior looks at her BlackBerry and starts typing. (Field NotesWork‐shadowing MD2)
I think it is almost completely negative in terms of its medical education [Any positive] factors are grossly outweighed by the significant disruptions to their ability to concentrate and participate in the educational session. And I think almost to some extent it's an implicit permission that gets granted to the house staff to disrupt their own teaching experience and disrupt others around them because everybody is doing it because everybody is being Blackberried. So it almost becomes the new social norm and while that may be a new social norm I'm not sure that that's a good thing How big is the negative impact? That's much harder to say. It's probably not a big impact on top of the endless other disruptions in the day to teaching, but it is measurable because it's a new factor so it's observable by me on top of all the other factors which have been there for years. (Interview Attending 3)
2:10‐Attending goes to the whiteboard to teach research methods to the team. Spotted Medical student#1 looking at his IPhone and typing.
2:15‐Med student#1 using the calculator function on his IPhone.
2:20‐Attending glances at his BB quickly.
2:28‐Attending resumes discussion of the patients' cases (Field notes, Work‐shadowing MD7).
DISCUSSION
The educational impacts of smartphone use for communication appear to center on increased connectedness of medical trainees and increased interruptions, which have positive and negative impacts in the areas of teaching, supervision, and professionalism. Smartphone communication provided potential educational benefits through (1) safer supervision with rapid access to help and (2) easier coordination of teaching sessions. Threats to the educational experience included (1) a high level of interruptions to both teachers and learners, which may reduce the effectiveness of formal and informal teaching; (2) replacement of face‐to‐face teaching with texting; (3) a potential erosion of autonomy and independence due to easy access to supervisors and easy ability for supervisors to take over; and (4) professionalism issues with difficulties balancing between clinical service demands and communication during patient and interprofessional encounters.
This study is the first to describe the intersection of clinical communication with smartphones and medical education. A recent study found that residents reported high use of smartphones during rounds for patient care as well as personal issues.[13] We have previously described the perceived impacts of smartphones on clinical communication, which included improved efficiency but concerns for increased interruptions and threats to professionalism.[6] We also observed that sites that used smartphones had increased interruptions compared to those with just pagers.[10] We have also described the content of e‐mail messages between clinicians and found that all e‐mails from nurses to physicians involved clinical care, but e‐mail exchanges between physicians were split between clinical care (60.4%), coordination within the team (53.5%), medical education (9.4%), and social communication (3.9%).[14] This study adds to the literature by focusing on the impacts of smartphone use to medical education and describing the perceived and observed impacts. This study provides a further example of how healthcare information technology can cause unintended consequences on medical education and appear to relate to the linked impacts of increased connectivity and the increased interruptions.[3] In essence, the trainee becomes more global, less local. Being more global translates to increased connections with people separated in physical space. Yet, this increased global connectedness resulted in the trainee being less local, with attention diverted elsewhere, taking away from the quality of patient interactions and interactions with other interprofessional team members. It also reduces the effectiveness of educational sessions for all participants. Although the level of supervision and autonomy are independently related, being too connected to supervisors may lower trainee autonomy by reducing independent thinking around patient issues.[15] It may also move teaching and learning from face‐to‐face conversations to text‐based messages. Although there have been existing tensions between service delivery and medical education, increased connectedness may tilt the balance toward the demands of service delivery and efficiency optimization at the expense of the educational experience. Finally, smartphone use appeared to create an internal tension among trainees, who have to juggle balancing professional behaviors and expectations in their dual role as learner and care provider; it would be educationally unprofessional to interrupt a teaching session and respond to a text message. However, failing to respond to a nurse who has sent a message and is expecting a response would be clinically unprofessional.
To address these threats, we advocate improving systems and processes to reduce interruptions and provide education on the tensions created by increased connectedness. Smarter communication systems could limit interrupting messages to urgent messages and queue nonurgent messages.[16] They could also inform senders about protected educational time. Even more sophisticated systems could inform the sender on the status of the receiver. For example, systems could indicate if they are available or if they are busy in an educational session or an important meeting with a patient and their family. Processes to reduce interruptions include interprofessional consensus on what constitutes an urgent issue and giving explicit permission to learners to ignore their smartphones during educational sessions except for critical communications purposes. Finally, education around smartphone communication for both learners and teachers may help minimize threats to learner autonomy, to face‐to‐face teaching, and to professionalism.[17]
Our study has several limitations. We derived this information from a general study of the impact of smartphones on clinical communication. Our study can be seen as hypothesis generating, and further research is warranted to validate these findings. There may be limits to generalizability as all sites adopted similar communication processes that included centralizing communications to make it easier for senders to reach a responsible physician.
In conclusion, we have provided a summary of the impact of rapidly emerging information technology on the educational experience of medical trainees and identified both positive and negative impacts. Of note, the negative impacts appear to be related to being more global and less local and high interruptions. Further research is required to confirm these unintended consequences as well as to develop solutions to address them. Educators should be aware of these findings and the need to develop curriculum to address and manage the negative impacts of smartphone use in the clinical training environment.
Acknowledgments
Disclosure: Nothing to report.
Medical residents are rapidly adopting smartphones. Recent statistics revealed that 85% of medical providers currently own a smartphone, and the majority use it in their clinical work.[1] Smartphone capabilities that include the use of text messaging, e‐mail, and mobile phone functions in the clinical setting may improve efficiency and quality of care by reducing the response time for urgent issues.[2] There is, however, increasing recognition that healthcare information technology can create unintended negative consequences. For example, studies have suggested that healthcare information technologies, such as the computerized physician order entry, may actually increase errors by creating new work, changing clinical workflow, and altering communication patterns.[3, 4, 5]
Smartphone use for clinical communication can have unintended consequences by increasing interruptions, reducing interprofessional relationships, and widening the gap between what nurses and physicians perceive as urgent clinical problems.[6] However, no studies have evaluated the impact of smartphones on the educational experience of medical trainees. Although previous studies have described the use of smartphones by trainees for rapid access to electronic medical resources,[7, 8, 9] we did not identify in our literature review any previous studies on the impact of using the smartphone's primary functionas a communication deviceon the educational experience of residents and medical students. Therefore, our study aimed to examine the impact of using smartphones for clinical communication on medical education.
METHODS
Design
The design of the study was qualitative research methodology using interview data, ethnographic data, and content analysis of text‐based messages.
Setting
From June 2009 to September 2010, we conducted a multisite evaluation study on general internal medicine (GIM) wards at 5 large academic teaching hospitals in the city of Toronto, Canada at St. Michael's Hospital, Sunnybrook Health Sciences Centre, Toronto General Hospital, Toronto Western Hospital, and Mount Sinai Hospital. Each hospital has clinical teaching units consisting typically of 4 medical teams. Each team includes 1 attending physician, 1 senior resident, 2 or more junior residents, and 2 to 4 medical students. Each hospital had 2 to 4 GIM wards in different geographic locations.
Communication Systems
To make it easier for nurses and other health professionals to communicate with the physician teams, all sites centralized communication to 1 team member, who acts as the single point of contact on behalf of their assigned team in the communication of patient‐related issues. We facilitated this communication through a shared device (either a pager or a smartphone). The senior resident typically carried the shared device during the day and the on‐call junior resident at night and on the weekends. Two hospitals provided smartphones to all residents, whereas a third site provided smartphones only to the senior residents. The standard processes of communication required that physicians respond to all calls and text messages. At the 3 sites with institutional smartphones, nurses could send text messages with patient information using a Web‐based system. We encrypted data sent to institutional smartphones to protect patient information.
Data Collection
Using a mixed‐methods ethnographic approach, we collected data using semistructured interviews, ethnographic observations, and content analysis of text messages. The original larger study focused primarily on examining the overall clinical impact of smartphone use.[10] For our current study, we analyzed the data with a focus on evaluating the impact of smartphones on the educational experience of medical trainees on the GIM teaching service. The respective institutions' research ethics boards approved the study.
Interviews
We conducted semistructured interviews with residents, medical students, attending physicians, and other clinicians across all of the sites to examine how clinicians perceived the impact of smartphones on medical education. We used a purposeful sampling strategy where we interviewed different groups of healthcare professionals who we suspected would represent different viewpoints on the use of smartphones for clinical communication. To obtain diverse perspectives, we snowball sampled by asking interviewees to suggest colleagues with differing views to participate in the interviews. The interview guide consisted of open‐ended questions with additional probes to elicit more detailed information from these frontline clinicians who initiate and receive communication. One of the study investigators (V.L.) conducted interviews that varied from 15 to 45 minutes in duration. We recorded, transcribed verbatim, and analyzed the interviews using NVivo software (QSR International, Doncaster, Victoria, Australia). We added additional questions iteratively as themes emerged from the initial interviews. One of the study investigators (V.L.) encouraged participants to speak freely, to raise issues that they perceived to be important, and to support their responses with examples.
Observations
We observed the communication processes in the hospitals by conducting a work‐shadowing approach that followed individual residents in their work environments. These observations included 1‐on‐1 supervision encounters involving attending staff, medical students, and other residents, and informal and formal teaching rounds. The observation periods included the usual working day (from 8 am to 6 pm) as well as the busiest times on call, typically from 6 pm until 11 pm. We sampled different residents for different time periods. We adopted a nonparticipatory observation technique where we observed all interruptions, communication interactions, and patterns from a distance. We defined workflow interruptions as an intrusion of an unplanned and unscheduled task, causing a discontinuation of tasks, a noticeable break, or task switch behaviour.[11] Data collection included timing of events and writing field notes. One of the study investigators (V.L.) performed all the work‐shadowing observations.
E‐mail
To study the volume and content of messages, we collected e‐mail communications between January 2009 and June 2009 from consenting residents at the 2 hospitals that provided smartphones to all GIM residents. E‐mail information included the sender, the receiver, the time of message, and the message content. To look at usage, we calculated the average number of e‐mails sent and received. To assess interruptions on formal teaching sessions, we paid particular attention to e‐mails received and sent during protected educational timeweekdays from 8 am to 9 am (morning report) and 12 pm to 1 pm (noon rounds). We randomly sampled 20% of all e‐mails sent between residents for content analysis and organized content related to medical education into thematic categories.
Analysis
We used a deductive approach to analyze the interview transcripts by applying a conceptual framework that assessed the educational impact of patient safety interventions.[12] This framework identified 5 educational domains (learning, teaching, supervision, assessment, and feedback). Three study investigators mapped interview data, work‐shadowing data, and e‐mail content to themes (V.L., B.W., and R.W.), and grouped data that did not translate into themes into new categories. We then triangulated the data to develop themes of the educational impact of smartphone communication by both perceived use and actual use, and subsequently constructed a framework of how smartphone communication affected education.
RESULTS
We conducted 124 semistructured interviews with residents, medical students, attending physicians, and other clinicians across all the sites to examine how clinicians perceived the impact of smartphones on medical education. We work‐shadowed 40 individual residents for a total of 196 hours (Table 1). We analyzed the 13,714 e‐mails sent from or received to 34 residents. To analyze e‐mail content, we reviewed 1179 e‐mails sent among residents.
| Methods | Sites | |||||
|---|---|---|---|---|---|---|
| St. Michael's Hospital | Sunnybrook Health Sciences Centre | Toronto General Hospital | Toronto Western Hospital | Mount Sinai Hospital | All Hospitals | |
| ||||||
| Work‐shadowing residents | ||||||
| Hours | 60 hours | 35 hours | 57 hours 55 minutes | 27 hours 46 minutes | 15 hours | 196 hours |
| No. of residents | 12 | 7 | 12 | 6 | 3 | 40 |
| Interviews with clinicians | ||||||
| Physicians | 10 | 5 | 13 | 5 | 33 | |
| Medical students | 5 | 4 | 1 | 1 | 11 | |
| Nurses | 9 | 11 | 15 | 14 | 49 | |
| Other health professionsa | 7 | 10 | 8 | 6 | 31 | |
| Total | 31 | 30 | 37 | 26 | 124 | |
We found that 2 key characteristics of smartphone use for clinical communication, namely an increase in connectedness leading to an increase in interruptions, impacted 3 educational domains: teaching, supervision, and professionalism (Figure 1).
Increased Connectedness
As a communication device, smartphones increase the ability to receive and respond to messages through voice, e‐mail, and text messaging. Not surprisingly, with the improved ability and mobility to communicate, medical trainees perceived being more connected with their team members, who included other residents, medical students, and attending staff as well as with other clinical services and professions. These smartphone communication activities appeared to be pervasive, occurring on the wards, at the bedside, while in transit, and in teaching sessions (Box 1: increased connectedness).
Box
Increased connectedness
I've used the Blackberry system and it's nice to be able to quickly text each other little messages especially for meeting times because then you don't have to page them and wait by the phone. So that's been great for in the team. (Interview Resident 3)
It's incredibly useful for when you're paging somebody else. Often times I'll be consulting with another physician on a patient and I'll say This is my BlackBerry. Call me back after you've seen the patient' or Call me back when you have a plan' or, you know, whatever. So that's extremely valuable which we never had with pages and no one would ever page you for that because it was too much of a pain. (Interview Resident 1)
My personal experience has been that if you need to speak to a more senior individual it's much easier to contact them via the BlackBerry. (Interview Medical Student 1)
At 7:25 pm, MD11 returns to the patient's room and continues examining her. While in the patient's room, I could see her talking on the BlackBerrys. I asked her later what calls she had while in the room. It turns out she had 3 phone calls and 2 texts. Two of the calls were from the radiation oncologists and 1 call from the pathologist. She also received 1 text on the Team BlackBerry and 1 text on the Senior's BlackBerry from the pharmacist. (Field Notes, Work-shadowing MD11)
Interruptions
The increased connectedness caused by smartphone use led residents to perceive an increase in the frequency of interruptions. The multitude of communication and contact options made available by smartphones to health providers created an expansive network of connected individuals who were in constant communication with each other. Instead of the difficulties associated with numeric paging and waiting for a response, nurses typically found it easier to call directly or send a text message to residents' smartphones. From the e‐mail analysis, residents received, on a daily basis, on average 25.7 e‐mails, (median, 20; interquartile range [IQR]: 1428) to the team smartphone and sent 7.5 e‐mails (median, 6; IQR: 410). During protected educational time, each resident received an average of 1.0 e‐mail (median, 1; IQR: 01) between 8 am and 9 am and an average of 2.3 e‐mails (median, 2; IQR: 13) during 12 pm to 1 pm (Figure 2). Each of these communication events, whether a phone call, e‐mail, or text‐message, led to an interruption (Box 2). Given that smartphones made it easier for nurses to contact residents, some residents attributed the increase in interruptions to a reduction in the threshold for nurses to communicate.
Box
Increased interruptions
The only negative I can think of is just the incredible number of communications that you get, you know, text messages and e‐mails and everything else. So just the, the number can sometimes be overwhelming. (Interview Resident 1)
Some of [the nurses] rely a little bit more on the BlackBerry so that they will tend to call you a bit more frequently for things that maybe sometimes they should try to find answer for themselves (Interview Resident 2)
And now with the option of being able to, if you really needed to, call them and talk to them directly, I think that kind of improves communication. They're easier to find. (Interview Nurse 4)
Supervision
Smartphone communication appeared to positively impact trainee supervision. Increased connectedness between team members allowed junior trainees to have access and rapidly communicate with a more experienced clinician, which provided them with greater support. Residents found smartphones particularly useful in situations where they felt uncomfortable or where they did not feel competent. Some of these instances related to procedural competence, with residents feeling more comfortable knowing they have rapid access to support (Box 3: increased support).
Box
Supervision
Increased support
It makes me feel more comfortable in the sense that I can instantly make a call or a text and have a question answered if I need an answer. Or if it were an emergency having the ability to talk on the phone and be talked through an emergency situation, or a procedure for example like if you were in a remote area or the physician was in a remote area and you were in hospital and you would need some of that guidance or counselling, there's no substitution. (Interview Medical Student 1)
I'm ready can u dblchk [sic] that I landmarked correctly. (Email from Junior to Senior)
MD3 returns to the patient's room to do a paracentesis with [junior resident]. He calls on his BB to [senior resident] to inform her that they are starting and then hangs up. [Senior resident] arrives at the patient's room. (Field Notes, Workshadowing MD3)
Decreased autonomy
The difference with the Blackberry is they're more likely to say By the way, this happened. Should I do this?' And I write back Yes', No.' If they didn't have that contact like I said they probably would have done something and then because they're making a decision on their own they could very easily have spent the time to research whatever to figure whether that was the right thing to do before doing it. Now they have an outlet where they can pass an idea off of me and then have me make, it's easier for me to make a decision for them. So that can negatively impact education. (Interview Attending 1)
What do I do for a high phosphate?(Email from Junior to Senior)
Hey Pt X's k is 5.5. Was going to shift her. What do u think? (Email from Junior to Senior)
You probably saw the hb 92. Let's give prbc asap while he's on HD.(Email from Staff to Residents on the team)
hb‐ hemoglobin, prbc packed red blood cells, HD ‐ hemodialysis
Hi. Just checking the bloodwork. What is happening to ms X? [sic] Creatinine rising still. Is a foley in? Urology reconsulted? (Email from Attending Staff to Junior Resident)
On the other hand, supervisors perceived that the easy rapid access afforded by smartphone use lowered the threshold for trainees to contact them. In some instances, these attending physicians felt that their trainees would text them for advice when they could have looked up the information themselves. As a result, the increased reliance on the attending physician's input prior to committing to a management plan decreased the trainee's autonomy and independent decision making (Box 3: decreased autonomy). In addition to trainee requests for increased staff involvement, smartphone use made it easier for attending physicians to initiate text messages to their residents as well. In some instances, staff physicians adopted a more hands‐on approach by directing their residents on how to manage their patients. It is unclear if trainees perceived this taking over of care as negatively influencing their education.
Teaching
Medical teams also frequently used smartphones to communicate the location and timing of educational rounds. We observed instances where residents communicated updated information relating to scheduled rounds, as well as for informing team members about spontaneous teaching sessions (Box 4: communicating rounds). Despite this initial benefit, staff physicians worried that interruptions resulting from smartphone use during educational sessions lowered the effectiveness of these sessions for all learners by creating a fragmented learning experience (Box 4: fragmented learning). Our data indicated that residents carrying the team smartphones received and sent a high number of e‐mails throughout the day, which continued at a similar rate during the protected educational time (Figure 2). Additionally, some of the teaching experiences that traditionally would occur in a face‐to‐face manner appeared to have migrated to text‐based interactions. It is unclear whether trainees perceive these text‐based interactions as more or less effective teaching encounters (Box 4: text‐based teaching).
Box
Teaching
Communicating rounds
One is that they can more efficiently communicate about the timing and location of education rounds in case they forget or sort of as an organizer for them (Interview Attending 3)
Physical Exam rounds is at 1:00 outside the morning report room. K. has kindly volunteered! If you miss us then the exam will be on the 3rd floor in room X. Pt X. See you there (Email from chief medical resident to trainees)
Fragmented learning
Because Blackberry is there, it's something that is potentially time occupying and can take the attention away from things and this is true of any Blackberries. People who have Blackberries they always look at their Blackberries so, you know, there are times when I'm sitting face to face with people and residents are looking at their Blackberries. So it's another way that they can be distracted. (Interview Attending 1).
I've seen that be an issue. I've certainly seen them losing concentration during a teaching session because they're being Blackberried, getting Blackberry messages. (Interview Attending 3)
2:06 Team meeting with Attending in a conference room.
2:29 Team BlackBerry (BB) beeps. Senior glances at BB. She dials a number on the Team BB. Speaks on the Team BB and turns to [Junior resident] to inform her that the family is here. She returns to the caller. Senior then hangs up and resumes to her teaching.
2:35 Attending's BB rings. She takes a look and ends the BB call.
2:39 Senior's BB rings. Senior picks up and talks about a patient's case and condition. Senior turns to [junior resident] and asks a question. Team members resume talking among themselves.
2:46 Senior hangs up on the phone call.
2:49 Team discusses another patient's condition/case.
2:57 Junior resident uses her BB to text.
3:02 Team BB beeps. It is a message about a patient's case.
3:05 Meeting ends. (Field Note excerpts, Work‐shadowing MD6)
Text‐based teaching
The resident would get very frustrated with how many questions we have once we've started. Like if three different medical students or four different medical students or four different places all texting him with, oh by the way, what does this stand for?, and he's responding to each of them individually then he has to answer it four different times as opposed to just in person when he can get us all together in a group and it's actually a learning experience. If questions are answered in an email, it's not really helpful for the rest of us. (Interview, medical student SB1)
That would be a great unifying diagnosis, but there may be some underlying element of ROH/NASH also I would hold off on A/C as we do not know if he has varices. Will need to review noncontrast CT ?HCC. Thx (Email from Consulting Staff to Junior)
A/C anticoagulation, CT computed tomography, HCC‐ hepatocellular carcinoma, NASH non‐alcoholic steatohepatitis, ROH alcohol
Professionalism
Our data revealed that smartphone interruptions occurred during teaching rounds and interactions with patients and with other clinical staff. Often these interruptions involved messages or phone calls pertaining to clinical concerns or tasks that nurses communicated to the residents via their smartphone (Box 5). Yet, by responding to these interruptions and initiating communications on their smartphones during patient care encounters and formal teaching sessions, trainees were perceived by other clinicians who were in attendance with them as being rude or disrespectful. Attending staff also tended to role model similar smartphone behaviors. Although we did not specifically work‐shadow attending staff, we did observe frequent usage of their personal smartphones during their interactions with residents.
Box
Professionalism
I don't like it when I see them checking messages when you're trying to talk to them. I think you're losing some of that communication sort of polite behaviour that maybe we knew a little bit more before all this texting and Blackberry. (Interview Allied Health 5)
I think that the etiquette of the Blackberry can be offensive, could be offensive especially with some of our older patients (Interview Allied Health 6)
Senior walks out of the patient's room while typing on the BlackBerry. She finishes typing and returns to the room at 5:36. Senior looks at her BlackBerry and starts typing inside the room in front of the patient. She paused to look at the patient and the resident doing the procedure [paracentesis]. She resumes texting again and walks out of the room at 5:38. Another resident walks out and Senior speaks with the resident. Senior returns to the room and speaks with the patient. She asks the patient if he has ever gotten a successful tap before. Senior looks at her BlackBerry and starts typing. (Field NotesWork‐shadowing MD2)
I think it is almost completely negative in terms of its medical education [Any positive] factors are grossly outweighed by the significant disruptions to their ability to concentrate and participate in the educational session. And I think almost to some extent it's an implicit permission that gets granted to the house staff to disrupt their own teaching experience and disrupt others around them because everybody is doing it because everybody is being Blackberried. So it almost becomes the new social norm and while that may be a new social norm I'm not sure that that's a good thing How big is the negative impact? That's much harder to say. It's probably not a big impact on top of the endless other disruptions in the day to teaching, but it is measurable because it's a new factor so it's observable by me on top of all the other factors which have been there for years. (Interview Attending 3)
2:10‐Attending goes to the whiteboard to teach research methods to the team. Spotted Medical student#1 looking at his IPhone and typing.
2:15‐Med student#1 using the calculator function on his IPhone.
2:20‐Attending glances at his BB quickly.
2:28‐Attending resumes discussion of the patients' cases (Field notes, Work‐shadowing MD7).
DISCUSSION
The educational impacts of smartphone use for communication appear to center on increased connectedness of medical trainees and increased interruptions, which have positive and negative impacts in the areas of teaching, supervision, and professionalism. Smartphone communication provided potential educational benefits through (1) safer supervision with rapid access to help and (2) easier coordination of teaching sessions. Threats to the educational experience included (1) a high level of interruptions to both teachers and learners, which may reduce the effectiveness of formal and informal teaching; (2) replacement of face‐to‐face teaching with texting; (3) a potential erosion of autonomy and independence due to easy access to supervisors and easy ability for supervisors to take over; and (4) professionalism issues with difficulties balancing between clinical service demands and communication during patient and interprofessional encounters.
This study is the first to describe the intersection of clinical communication with smartphones and medical education. A recent study found that residents reported high use of smartphones during rounds for patient care as well as personal issues.[13] We have previously described the perceived impacts of smartphones on clinical communication, which included improved efficiency but concerns for increased interruptions and threats to professionalism.[6] We also observed that sites that used smartphones had increased interruptions compared to those with just pagers.[10] We have also described the content of e‐mail messages between clinicians and found that all e‐mails from nurses to physicians involved clinical care, but e‐mail exchanges between physicians were split between clinical care (60.4%), coordination within the team (53.5%), medical education (9.4%), and social communication (3.9%).[14] This study adds to the literature by focusing on the impacts of smartphone use to medical education and describing the perceived and observed impacts. This study provides a further example of how healthcare information technology can cause unintended consequences on medical education and appear to relate to the linked impacts of increased connectivity and the increased interruptions.[3] In essence, the trainee becomes more global, less local. Being more global translates to increased connections with people separated in physical space. Yet, this increased global connectedness resulted in the trainee being less local, with attention diverted elsewhere, taking away from the quality of patient interactions and interactions with other interprofessional team members. It also reduces the effectiveness of educational sessions for all participants. Although the level of supervision and autonomy are independently related, being too connected to supervisors may lower trainee autonomy by reducing independent thinking around patient issues.[15] It may also move teaching and learning from face‐to‐face conversations to text‐based messages. Although there have been existing tensions between service delivery and medical education, increased connectedness may tilt the balance toward the demands of service delivery and efficiency optimization at the expense of the educational experience. Finally, smartphone use appeared to create an internal tension among trainees, who have to juggle balancing professional behaviors and expectations in their dual role as learner and care provider; it would be educationally unprofessional to interrupt a teaching session and respond to a text message. However, failing to respond to a nurse who has sent a message and is expecting a response would be clinically unprofessional.
To address these threats, we advocate improving systems and processes to reduce interruptions and provide education on the tensions created by increased connectedness. Smarter communication systems could limit interrupting messages to urgent messages and queue nonurgent messages.[16] They could also inform senders about protected educational time. Even more sophisticated systems could inform the sender on the status of the receiver. For example, systems could indicate if they are available or if they are busy in an educational session or an important meeting with a patient and their family. Processes to reduce interruptions include interprofessional consensus on what constitutes an urgent issue and giving explicit permission to learners to ignore their smartphones during educational sessions except for critical communications purposes. Finally, education around smartphone communication for both learners and teachers may help minimize threats to learner autonomy, to face‐to‐face teaching, and to professionalism.[17]
Our study has several limitations. We derived this information from a general study of the impact of smartphones on clinical communication. Our study can be seen as hypothesis generating, and further research is warranted to validate these findings. There may be limits to generalizability as all sites adopted similar communication processes that included centralizing communications to make it easier for senders to reach a responsible physician.
In conclusion, we have provided a summary of the impact of rapidly emerging information technology on the educational experience of medical trainees and identified both positive and negative impacts. Of note, the negative impacts appear to be related to being more global and less local and high interruptions. Further research is required to confirm these unintended consequences as well as to develop solutions to address them. Educators should be aware of these findings and the need to develop curriculum to address and manage the negative impacts of smartphone use in the clinical training environment.
Acknowledgments
Disclosure: Nothing to report.
- , . Smartphone app use among medical providers in ACGME training programs. J Med Syst. 2012;36:3135–3139.
- , , , et al. The use of smartphones for clinical communication on internal medicine wards. J Hosp Med. 2010;5:553–559.
- , , , , , . Anticipating and addressing the unintended consequences of health IT and policy: a report from the AMIA 2009 Health Policy Meeting. J Am Med Inform Assoc. 2011;18(1):82–90.
- , , , , . Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc. 2006;13(5):547–556.
- , , , . “e‐Iatrogenesis”: the most critical unintended consequence of CPOE and other HIT. J Am Med Inform Assoc. 2007;14(3):387–388.
- , , , et al. An evaluation of the use of smartphones to communicate between clinicians: a mixed‐methods study. J Med Internet Res. 2011;13(3):e59.
- . Smartphones in clinical practice, medical education, and research. Arch Intern Med. 2011;171(14):1294–1296.
- , , , . Use of handheld computers in medical education. A systematic review. J Gen Intern Med. 2006;21(5):531–537.
- , , , . A review and a framework of handheld computer adoption in healthcare. Int J Med Inform. 2005;74(5):409–422.
- , , , et al. The intended and unintended consequences of communication systems on General Internal Medicine inpatient care delivery: a prospective observational case study of five teaching hospitals [published online ahead of print January 25, 2013]. J Am Med Inform Assoc. doi:10.1136/amiajnl‐2012‐001160.
- , , , , . Hospital doctors' workflow interruptions and activities: an observation study. BMJ Qual Saf. 2011;20(6):491–497.
- , , , et al. Computerised provider order entry and residency education in an academic medical centre. Med Educ. 2012;46:795–806.
- , , , . Smartphone use during inpatient attending rounds: Prevalence, patterns and potential for distraction. J Hosp Med. 2012;7(8):595–599.
- , , , et al. Understanding interprofessional communication: a content analysis of email communications between doctors and nurses. Applied Clinical Informatics. 2012;3:38–51.
- , , , . Preserving professional credibility: grounded theory study of medical trainees' requests for clinical support. BMJ. 2009;338:b128.
- , , , , . Beyond paging: building a web‐based communication tool for nurses and physicians. J Gen Intern Med. 2009;24(1):105–110.
- , . Distracted doctoring: smartphones before patients? CMAJ. 2012;184:1440.
- , . Smartphone app use among medical providers in ACGME training programs. J Med Syst. 2012;36:3135–3139.
- , , , et al. The use of smartphones for clinical communication on internal medicine wards. J Hosp Med. 2010;5:553–559.
- , , , , , . Anticipating and addressing the unintended consequences of health IT and policy: a report from the AMIA 2009 Health Policy Meeting. J Am Med Inform Assoc. 2011;18(1):82–90.
- , , , , . Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc. 2006;13(5):547–556.
- , , , . “e‐Iatrogenesis”: the most critical unintended consequence of CPOE and other HIT. J Am Med Inform Assoc. 2007;14(3):387–388.
- , , , et al. An evaluation of the use of smartphones to communicate between clinicians: a mixed‐methods study. J Med Internet Res. 2011;13(3):e59.
- . Smartphones in clinical practice, medical education, and research. Arch Intern Med. 2011;171(14):1294–1296.
- , , , . Use of handheld computers in medical education. A systematic review. J Gen Intern Med. 2006;21(5):531–537.
- , , , . A review and a framework of handheld computer adoption in healthcare. Int J Med Inform. 2005;74(5):409–422.
- , , , et al. The intended and unintended consequences of communication systems on General Internal Medicine inpatient care delivery: a prospective observational case study of five teaching hospitals [published online ahead of print January 25, 2013]. J Am Med Inform Assoc. doi:10.1136/amiajnl‐2012‐001160.
- , , , , . Hospital doctors' workflow interruptions and activities: an observation study. BMJ Qual Saf. 2011;20(6):491–497.
- , , , et al. Computerised provider order entry and residency education in an academic medical centre. Med Educ. 2012;46:795–806.
- , , , . Smartphone use during inpatient attending rounds: Prevalence, patterns and potential for distraction. J Hosp Med. 2012;7(8):595–599.
- , , , et al. Understanding interprofessional communication: a content analysis of email communications between doctors and nurses. Applied Clinical Informatics. 2012;3:38–51.
- , , , . Preserving professional credibility: grounded theory study of medical trainees' requests for clinical support. BMJ. 2009;338:b128.
- , , , , . Beyond paging: building a web‐based communication tool for nurses and physicians. J Gen Intern Med. 2009;24(1):105–110.
- , . Distracted doctoring: smartphones before patients? CMAJ. 2012;184:1440.
© 2013 Society of Hospital Medicine
