Original Research

The gold standard for diagnosing dermatophytosis is the use of direct microscopic examination together with fungal culture.¹ However, in the last 2 decades, molecular techniques that currently are available worldwide have improved the diagnosis procedure.^2,3 In the practice of dermatology, potassium hydroxide (KOH) testing is a commonly used method for the diagnosis of superficial fungal infections.⁴ The sensitivity and specificity of KOH testing in patients with tinea pedis have been reported as 73.3% and 42.5%, respectively.⁵ Repetition of this test after an initial negative test result is recommended if the clinical picture strongly suggests a fungal infection.^6,7 Alternatively, several repetitions of direct microscopic examinations also have been proposed for detecting other microorganisms. For example, 3 negative sputum smears traditionally are recommended to exclude a diagnosis of pulmonary tuberculosis.⁸ However, after numerous investigations in various regions of the world, the World Health Organization reduced the recommended number of these specimens from 3 to 2 in 2007.⁹

The literature suggests that successive mycological tests, both with direct microscopy and fungal cultures, improve the diagnosis of onychomycosis.^1,10,11 Therefore, if such investigations are increased in number, recommendations for successive mycological tests may be more reliable. In the current study, we aimed to investigate the value of successive KOH testing in the management of patients with clinically suspected tinea pedis.

Methods

Patients and Clinical Evaluation
One hundred thirty-five consecutive patients (63 male; 72 female) with clinical symptoms suggestive of intertriginous, vesiculobullous, and/or moccasin-type tinea pedis were enrolled in this prospective study. The mean age (SD) of patients was 45.9 (14.7) years (range, 11–77 years). Almost exclusively, the clinical symptoms suggestive of tinea pedis were desquamation or maceration in the toe webs, blistering lesions on the soles, and diffuse or patchy scaling or keratosis on the soles. A single dermatologist (B.F.K.) clinically evaluated the patients and found only 1 region showing different patterns suggestive of tinea pedis in 72 patients, 2 regions in 61 patients, and 3 regions in 2 patients. Therefore, 200 lesions from the 135 patients were chosen for the KOH test. The dermatologist recorded her level of suspicion for a fungal infection as low or high for each lesion, depending on the absence or presence of signs (eg, unilateral involvement, a well-defined border). None of the patients had used topical or systemic antifungal therapy for at least 1 month prior to the study.¹²

Clinical Sampling and Direct Microscopic Examination
The dermatologist took 3 samples of skin scrapings from each of the 200 lesions. All 3 samples from a given lesion were obtained from sites with the same clinical symptoms in a single session. Special attention was paid to samples from the active advancing borders of the lesions and the roofs of blisters if they were present.¹³ Upon completion of every 15 samples from every 5 lesions, the dermatologist randomized the order of the samples (https://www.random.org/). She then gave the samples, without the identities of the patients or any clinical information, to an experienced laboratory technician for direct microscopic examination. The technician prepared and examined the samples as described elsewhere^5,7,14 and recorded the results as positive if hyphal elements were present or negative if they were not. The study was reviewed and approved by the Çukurova University Faculty of Medicine Ethics Committee (Adana, Turkey). Informed consent was obtained from each patient or from his/her guardian(s) prior to initiating the study.

Statistical Analysis
Statistical analysis was conducted using the χ² test in the SPSS software version 20.0. McNemar test was used for analysis of the paired data.

 

Results

Among the 135 patients, lesions were suggestive of the intertriginous type of tinea pedis in 24 patients, moccasin type in 50 patients, and both intertriginous and moccasin type in 58 patients. Among the remaining 3 patients, 1 had lesions suggestive of the vesiculobullous type, and another patient had both the vesiculobullous and intertriginous types; the last patient demonstrated lesions that were inconsistent with any of these 3 subtypes of tinea pedis, and a well-defined eczematous plaque was observed on the dorsal surface of the patient’s left foot.

Among the 200 lesions from which skin scrapings were taken for KOH testing, 83 were in the toe webs, 110 were on the soles, and 7 were on the dorsal surfaces of the feet. Of these 7 dorsal lesions, 6 were extensions from lesions on the toe webs or soles and 1 was inconsistent with the 3 subtypes of tinea pedis. Among the 200 lesions, the main clinical symptom was maceration in 38 lesions, desquamation or scaling in 132 lesions, keratosis in 28 lesions, and blistering in 2 lesions. The dermatologist recorded the level of suspicion for tinea pedis as low in 68 lesions and high in 132.

According to the order in which the dermatologist took the 3 samples from each lesion, the KOH test was positive in 95 of the first set of 200 samples, 94 of the second set, and 86 of the third set; however, from the second set, the incremental yield (ie, the number of lesions in which the first KOH test was negative and the second was positive) was 10. The number of lesions in which the first and the second tests were negative and the third was positive was only 4. Therefore, the number of lesions with a positive KOH test was significantly increased from 95 to 105 by performing the second KOH test (P=.002). This number again increased from 105 to 109 when a third test was performed; however, this increase was not statistically significant (P=.125)(Table 1).

According to an evaluation that was not stratified by the dermatologist’s order of sampling, 72 lesions (36.0%) showed KOH test positivity in all 3 samples, 22 (11.0%) were positive in 2 samples, 15 (7.5%) were positive in only 1 sample, and 91 (45.5%) were positive in none of the samples (Table 2). When the data were subdivided based on the sites of the lesions, the toe web lesions (n=83) showed rates of 41.0%, 9.6%, and 4.8% for 3, 2, and 1 positive KOH tests, respectively. For the sole lesions (n=110), the rates were somewhat different at 31.8%, 11.8%, and 10.0%, respectively, but the difference was not statistically significant (P=.395).

For the subgroups based on the main clinical symptoms, the percentage of lesions having at least 1 positive KOH test from the 3 samples was 35.7% for the keratotic lesions (n=28). This rate was lower than macerated lesions (n=38) and desquamating or scaling lesions (n=132), which were 52.6% and 59.1%, respectively (Table 2). On the other hand, the percentage of lesions that produced only 1 or 2 positive KOH tests from the 3 samples was 25.0% for the keratotic lesions, which was higher than the rates for the macerated lesions and the desquamating or scaling lesions (13.1% and 18.9%, respectively). In particular, the difference between the keratotic lesions and the desquamating or scaling lesions in the distribution of the rates of 0, 1, 2, and 3 positive KOH tests was statistically significant (P=.019). The macerated, desquamating or scaling, keratotic, and blistering lesions are presented in the Figure.

If the dermatologist indicated a high suspicion of fungal infection, it was more likely that at least 1 of 3 KOH test results was positive. The rate of at least 1 positive test was 64.4% for the highly suspicious lesions (n=132) and 35.3% for the lesions with low suspicion of a fungal infection (n=68)(Table 2). The difference was statistically significant (P<.001). Conversely, if the suspicion was low, it was more likely that only 1 or 2 KOH tests were positive. The percentages of lesions having 3, 2, or 1 positive KOH tests were 14.7%, 8.8%, and 11.8%, respectively, for the low-suspicion lesions and 47.0%, 12.1%, and 5.3%, respectively, for the high-suspicion lesions. The difference was statistically significant (P<.001).

Comment

In the current study, we aimed to investigate if successive KOH tests provide an incremental diagnostic yield in the management of patients with clinically suspected tinea pedis and if these results differ among the subgroups of patients. Both in the evaluation taking into account the order of sampling and in the evaluation disregarding this order, we found that the second sample was necessary for all subgroups, and even the third sample was necessary for patients with keratotic lesions. The main limitation of the study was that we lacked a gold-standard technique (eg, a molecular-based technique); therefore, we are unable to comment on the false-negative and false-positive results of the successive KOH testing.

Summerbell et al¹¹ found in their study that in initial specimens of toenails with apparent lesions taken from 473 patients, the KOH test was 73.8% sensitive for dermatophytes, and this rate was only somewhat higher for cultures (74.6%). Arabatzis et al² investigated 92 skin, nail, and hair specimens from 67 patients with suspected dermatophytosis and found that the KOH test was superior to culture for the detection of dermatophytes (43% vs 33%). Moreover and more importantly, they noted that a real-time polymerase chain reaction (PCR) assay yielded a higher detection rate (51%).² In another study, Wisselink et al³ examined 1437 clinical samples and demonstrated a great increase in the detection of dermatophytes using a real-time PCR assay (48.5%) compared to culture (26.9%). However, PCR may not reflect active disease and could lead to false-positive results.^2,3 Therefore, the aforementioned weakness of our study will be overcome in further studies investigating the benefit of successive KOH testing compared to a molecular-based assay, such as the real-time PCR assay.

In this study, repeating the KOH test provided better results for achieving the diagnosis of tinea pedis in a large number of samples from clinically suspected lesions. Additionally, the distribution of 3, 2, or 1 positive results on the 3 KOH tests was different among the subgroups of lesions. Overall, positivity was less frequent in the keratotic lesions compared to the macerated or desquamating or scaling lesions. Moreover, positivity on all 3 tests also was less frequent in the keratotic lesions. Inversely, the frequency of samples with only 1 or 2 positive results was higher in this subgroup. The necessity for the second, even the third, tests was greater in this subgroup.

Our findings were consistent with the results of the studies performed with successive mycological tests on the nail specimens. Meireles et al¹ repeated 156 mycological nail tests 3 times and found the rate of positivity in the first test to be 19.9%. When the results of the first and second tests were combined, this rate increased to 28.2%, and when the results of all 3 tests were combined, it increased to 37.8%.¹ Gupta¹⁰ demonstrated that even a fourth culture provided an incremental diagnostic yield in the diagnosis of onychomycosis, yet 4 cultures may not be clinically practical. Furthermore, periodic acid–Schiff staining is a more effective measure of positivity in onychomycosis.¹⁵

Although the overall rate of positivity on the 3 tests in our study was unsurprisingly higher in lesions rated highly suspicious for a fungal infection, the rate of only 1 or 2 positive tests was surprisingly somewhat higher in low-suspicion lesions, which suggested that repeating the KOH test would be beneficial, even if the clinical suspicion for tinea pedis was low. The novel contribution of this study includes the finding that mycological information was markedly improved in highly suspicious tinea pedis lesions regardless of the infection site (Table 1) by using 3 successive KOH tests; the percentage of lesions with 1, 2, or 3 positive KOH tests was 5.3%, 12.1%, and 47.0%, respectively (Table 2). A single physician from a single geographical location introduces a limitation to the study for a variety of reasons, including bias in the cases chosen and possible overrepresentation of the causative organism due to region-specific incidence. It is unknown how different causative organisms affect KOH results. The lack of fungal culture results limits the value of this information.

 

Conclusion

In this study, we investigated the benefit of successive KOH testing in the laboratory diagnosis of tinea pedis and found that the use of second samples in particular provided a substantial increase in diagnostic yield. In other words, the utilization of successive KOH testing remarkably improved the diagnosis of tinea pedis. Therefore, we suggest that at least 2 samples of skin scrapings should be taken for the diagnosis of tinea pedis and that the number of samples should be at least 3 for keratotic lesions. However, further study by using a gold-standard method such as a molecular-based assay as well as taking the samples in daily or weekly intervals is recommended to achieve a more reliable result.

Acknowledgment

The authors would like to thank Gökçen Şahin (Adana, Turkey) for providing technical support in direct microscopic examination.

The gold standard for diagnosing dermatophytosis is the use of direct microscopic examination together with fungal culture.¹ However, in the last 2 decades, molecular techniques that currently are available worldwide have improved the diagnosis procedure.^2,3 In the practice of dermatology, potassium hydroxide (KOH) testing is a commonly used method for the diagnosis of superficial fungal infections.⁴ The sensitivity and specificity of KOH testing in patients with tinea pedis have been reported as 73.3% and 42.5%, respectively.⁵ Repetition of this test after an initial negative test result is recommended if the clinical picture strongly suggests a fungal infection.^6,7 Alternatively, several repetitions of direct microscopic examinations also have been proposed for detecting other microorganisms. For example, 3 negative sputum smears traditionally are recommended to exclude a diagnosis of pulmonary tuberculosis.⁸ However, after numerous investigations in various regions of the world, the World Health Organization reduced the recommended number of these specimens from 3 to 2 in 2007.⁹

The literature suggests that successive mycological tests, both with direct microscopy and fungal cultures, improve the diagnosis of onychomycosis.^1,10,11 Therefore, if such investigations are increased in number, recommendations for successive mycological tests may be more reliable. In the current study, we aimed to investigate the value of successive KOH testing in the management of patients with clinically suspected tinea pedis.

Methods

Patients and Clinical Evaluation
One hundred thirty-five consecutive patients (63 male; 72 female) with clinical symptoms suggestive of intertriginous, vesiculobullous, and/or moccasin-type tinea pedis were enrolled in this prospective study. The mean age (SD) of patients was 45.9 (14.7) years (range, 11–77 years). Almost exclusively, the clinical symptoms suggestive of tinea pedis were desquamation or maceration in the toe webs, blistering lesions on the soles, and diffuse or patchy scaling or keratosis on the soles. A single dermatologist (B.F.K.) clinically evaluated the patients and found only 1 region showing different patterns suggestive of tinea pedis in 72 patients, 2 regions in 61 patients, and 3 regions in 2 patients. Therefore, 200 lesions from the 135 patients were chosen for the KOH test. The dermatologist recorded her level of suspicion for a fungal infection as low or high for each lesion, depending on the absence or presence of signs (eg, unilateral involvement, a well-defined border). None of the patients had used topical or systemic antifungal therapy for at least 1 month prior to the study.¹²

Clinical Sampling and Direct Microscopic Examination
The dermatologist took 3 samples of skin scrapings from each of the 200 lesions. All 3 samples from a given lesion were obtained from sites with the same clinical symptoms in a single session. Special attention was paid to samples from the active advancing borders of the lesions and the roofs of blisters if they were present.¹³ Upon completion of every 15 samples from every 5 lesions, the dermatologist randomized the order of the samples (https://www.random.org/). She then gave the samples, without the identities of the patients or any clinical information, to an experienced laboratory technician for direct microscopic examination. The technician prepared and examined the samples as described elsewhere^5,7,14 and recorded the results as positive if hyphal elements were present or negative if they were not. The study was reviewed and approved by the Çukurova University Faculty of Medicine Ethics Committee (Adana, Turkey). Informed consent was obtained from each patient or from his/her guardian(s) prior to initiating the study.

Statistical Analysis
Statistical analysis was conducted using the χ² test in the SPSS software version 20.0. McNemar test was used for analysis of the paired data.

 

Results

Among the 135 patients, lesions were suggestive of the intertriginous type of tinea pedis in 24 patients, moccasin type in 50 patients, and both intertriginous and moccasin type in 58 patients. Among the remaining 3 patients, 1 had lesions suggestive of the vesiculobullous type, and another patient had both the vesiculobullous and intertriginous types; the last patient demonstrated lesions that were inconsistent with any of these 3 subtypes of tinea pedis, and a well-defined eczematous plaque was observed on the dorsal surface of the patient’s left foot.

Among the 200 lesions from which skin scrapings were taken for KOH testing, 83 were in the toe webs, 110 were on the soles, and 7 were on the dorsal surfaces of the feet. Of these 7 dorsal lesions, 6 were extensions from lesions on the toe webs or soles and 1 was inconsistent with the 3 subtypes of tinea pedis. Among the 200 lesions, the main clinical symptom was maceration in 38 lesions, desquamation or scaling in 132 lesions, keratosis in 28 lesions, and blistering in 2 lesions. The dermatologist recorded the level of suspicion for tinea pedis as low in 68 lesions and high in 132.

According to the order in which the dermatologist took the 3 samples from each lesion, the KOH test was positive in 95 of the first set of 200 samples, 94 of the second set, and 86 of the third set; however, from the second set, the incremental yield (ie, the number of lesions in which the first KOH test was negative and the second was positive) was 10. The number of lesions in which the first and the second tests were negative and the third was positive was only 4. Therefore, the number of lesions with a positive KOH test was significantly increased from 95 to 105 by performing the second KOH test (P=.002). This number again increased from 105 to 109 when a third test was performed; however, this increase was not statistically significant (P=.125)(Table 1).

According to an evaluation that was not stratified by the dermatologist’s order of sampling, 72 lesions (36.0%) showed KOH test positivity in all 3 samples, 22 (11.0%) were positive in 2 samples, 15 (7.5%) were positive in only 1 sample, and 91 (45.5%) were positive in none of the samples (Table 2). When the data were subdivided based on the sites of the lesions, the toe web lesions (n=83) showed rates of 41.0%, 9.6%, and 4.8% for 3, 2, and 1 positive KOH tests, respectively. For the sole lesions (n=110), the rates were somewhat different at 31.8%, 11.8%, and 10.0%, respectively, but the difference was not statistically significant (P=.395).

For the subgroups based on the main clinical symptoms, the percentage of lesions having at least 1 positive KOH test from the 3 samples was 35.7% for the keratotic lesions (n=28). This rate was lower than macerated lesions (n=38) and desquamating or scaling lesions (n=132), which were 52.6% and 59.1%, respectively (Table 2). On the other hand, the percentage of lesions that produced only 1 or 2 positive KOH tests from the 3 samples was 25.0% for the keratotic lesions, which was higher than the rates for the macerated lesions and the desquamating or scaling lesions (13.1% and 18.9%, respectively). In particular, the difference between the keratotic lesions and the desquamating or scaling lesions in the distribution of the rates of 0, 1, 2, and 3 positive KOH tests was statistically significant (P=.019). The macerated, desquamating or scaling, keratotic, and blistering lesions are presented in the Figure.

If the dermatologist indicated a high suspicion of fungal infection, it was more likely that at least 1 of 3 KOH test results was positive. The rate of at least 1 positive test was 64.4% for the highly suspicious lesions (n=132) and 35.3% for the lesions with low suspicion of a fungal infection (n=68)(Table 2). The difference was statistically significant (P<.001). Conversely, if the suspicion was low, it was more likely that only 1 or 2 KOH tests were positive. The percentages of lesions having 3, 2, or 1 positive KOH tests were 14.7%, 8.8%, and 11.8%, respectively, for the low-suspicion lesions and 47.0%, 12.1%, and 5.3%, respectively, for the high-suspicion lesions. The difference was statistically significant (P<.001).

Comment

In the current study, we aimed to investigate if successive KOH tests provide an incremental diagnostic yield in the management of patients with clinically suspected tinea pedis and if these results differ among the subgroups of patients. Both in the evaluation taking into account the order of sampling and in the evaluation disregarding this order, we found that the second sample was necessary for all subgroups, and even the third sample was necessary for patients with keratotic lesions. The main limitation of the study was that we lacked a gold-standard technique (eg, a molecular-based technique); therefore, we are unable to comment on the false-negative and false-positive results of the successive KOH testing.

Summerbell et al¹¹ found in their study that in initial specimens of toenails with apparent lesions taken from 473 patients, the KOH test was 73.8% sensitive for dermatophytes, and this rate was only somewhat higher for cultures (74.6%). Arabatzis et al² investigated 92 skin, nail, and hair specimens from 67 patients with suspected dermatophytosis and found that the KOH test was superior to culture for the detection of dermatophytes (43% vs 33%). Moreover and more importantly, they noted that a real-time polymerase chain reaction (PCR) assay yielded a higher detection rate (51%).² In another study, Wisselink et al³ examined 1437 clinical samples and demonstrated a great increase in the detection of dermatophytes using a real-time PCR assay (48.5%) compared to culture (26.9%). However, PCR may not reflect active disease and could lead to false-positive results.^2,3 Therefore, the aforementioned weakness of our study will be overcome in further studies investigating the benefit of successive KOH testing compared to a molecular-based assay, such as the real-time PCR assay.

In this study, repeating the KOH test provided better results for achieving the diagnosis of tinea pedis in a large number of samples from clinically suspected lesions. Additionally, the distribution of 3, 2, or 1 positive results on the 3 KOH tests was different among the subgroups of lesions. Overall, positivity was less frequent in the keratotic lesions compared to the macerated or desquamating or scaling lesions. Moreover, positivity on all 3 tests also was less frequent in the keratotic lesions. Inversely, the frequency of samples with only 1 or 2 positive results was higher in this subgroup. The necessity for the second, even the third, tests was greater in this subgroup.

Our findings were consistent with the results of the studies performed with successive mycological tests on the nail specimens. Meireles et al¹ repeated 156 mycological nail tests 3 times and found the rate of positivity in the first test to be 19.9%. When the results of the first and second tests were combined, this rate increased to 28.2%, and when the results of all 3 tests were combined, it increased to 37.8%.¹ Gupta¹⁰ demonstrated that even a fourth culture provided an incremental diagnostic yield in the diagnosis of onychomycosis, yet 4 cultures may not be clinically practical. Furthermore, periodic acid–Schiff staining is a more effective measure of positivity in onychomycosis.¹⁵

Although the overall rate of positivity on the 3 tests in our study was unsurprisingly higher in lesions rated highly suspicious for a fungal infection, the rate of only 1 or 2 positive tests was surprisingly somewhat higher in low-suspicion lesions, which suggested that repeating the KOH test would be beneficial, even if the clinical suspicion for tinea pedis was low. The novel contribution of this study includes the finding that mycological information was markedly improved in highly suspicious tinea pedis lesions regardless of the infection site (Table 1) by using 3 successive KOH tests; the percentage of lesions with 1, 2, or 3 positive KOH tests was 5.3%, 12.1%, and 47.0%, respectively (Table 2). A single physician from a single geographical location introduces a limitation to the study for a variety of reasons, including bias in the cases chosen and possible overrepresentation of the causative organism due to region-specific incidence. It is unknown how different causative organisms affect KOH results. The lack of fungal culture results limits the value of this information.

 

Conclusion

In this study, we investigated the benefit of successive KOH testing in the laboratory diagnosis of tinea pedis and found that the use of second samples in particular provided a substantial increase in diagnostic yield. In other words, the utilization of successive KOH testing remarkably improved the diagnosis of tinea pedis. Therefore, we suggest that at least 2 samples of skin scrapings should be taken for the diagnosis of tinea pedis and that the number of samples should be at least 3 for keratotic lesions. However, further study by using a gold-standard method such as a molecular-based assay as well as taking the samples in daily or weekly intervals is recommended to achieve a more reliable result.

Acknowledgment

The authors would like to thank Gökçen Şahin (Adana, Turkey) for providing technical support in direct microscopic examination.

The gold standard for diagnosing dermatophytosis is the use of direct microscopic examination together with fungal culture.¹ However, in the last 2 decades, molecular techniques that currently are available worldwide have improved the diagnosis procedure.^2,3 In the practice of dermatology, potassium hydroxide (KOH) testing is a commonly used method for the diagnosis of superficial fungal infections.⁴ The sensitivity and specificity of KOH testing in patients with tinea pedis have been reported as 73.3% and 42.5%, respectively.⁵ Repetition of this test after an initial negative test result is recommended if the clinical picture strongly suggests a fungal infection.^6,7 Alternatively, several repetitions of direct microscopic examinations also have been proposed for detecting other microorganisms. For example, 3 negative sputum smears traditionally are recommended to exclude a diagnosis of pulmonary tuberculosis.⁸ However, after numerous investigations in various regions of the world, the World Health Organization reduced the recommended number of these specimens from 3 to 2 in 2007.⁹

The literature suggests that successive mycological tests, both with direct microscopy and fungal cultures, improve the diagnosis of onychomycosis.^1,10,11 Therefore, if such investigations are increased in number, recommendations for successive mycological tests may be more reliable. In the current study, we aimed to investigate the value of successive KOH testing in the management of patients with clinically suspected tinea pedis.

Methods

Patients and Clinical Evaluation
One hundred thirty-five consecutive patients (63 male; 72 female) with clinical symptoms suggestive of intertriginous, vesiculobullous, and/or moccasin-type tinea pedis were enrolled in this prospective study. The mean age (SD) of patients was 45.9 (14.7) years (range, 11–77 years). Almost exclusively, the clinical symptoms suggestive of tinea pedis were desquamation or maceration in the toe webs, blistering lesions on the soles, and diffuse or patchy scaling or keratosis on the soles. A single dermatologist (B.F.K.) clinically evaluated the patients and found only 1 region showing different patterns suggestive of tinea pedis in 72 patients, 2 regions in 61 patients, and 3 regions in 2 patients. Therefore, 200 lesions from the 135 patients were chosen for the KOH test. The dermatologist recorded her level of suspicion for a fungal infection as low or high for each lesion, depending on the absence or presence of signs (eg, unilateral involvement, a well-defined border). None of the patients had used topical or systemic antifungal therapy for at least 1 month prior to the study.¹²

Clinical Sampling and Direct Microscopic Examination
The dermatologist took 3 samples of skin scrapings from each of the 200 lesions. All 3 samples from a given lesion were obtained from sites with the same clinical symptoms in a single session. Special attention was paid to samples from the active advancing borders of the lesions and the roofs of blisters if they were present.¹³ Upon completion of every 15 samples from every 5 lesions, the dermatologist randomized the order of the samples (https://www.random.org/). She then gave the samples, without the identities of the patients or any clinical information, to an experienced laboratory technician for direct microscopic examination. The technician prepared and examined the samples as described elsewhere^5,7,14 and recorded the results as positive if hyphal elements were present or negative if they were not. The study was reviewed and approved by the Çukurova University Faculty of Medicine Ethics Committee (Adana, Turkey). Informed consent was obtained from each patient or from his/her guardian(s) prior to initiating the study.

Statistical Analysis
Statistical analysis was conducted using the χ² test in the SPSS software version 20.0. McNemar test was used for analysis of the paired data.

 

Results

Among the 135 patients, lesions were suggestive of the intertriginous type of tinea pedis in 24 patients, moccasin type in 50 patients, and both intertriginous and moccasin type in 58 patients. Among the remaining 3 patients, 1 had lesions suggestive of the vesiculobullous type, and another patient had both the vesiculobullous and intertriginous types; the last patient demonstrated lesions that were inconsistent with any of these 3 subtypes of tinea pedis, and a well-defined eczematous plaque was observed on the dorsal surface of the patient’s left foot.

Among the 200 lesions from which skin scrapings were taken for KOH testing, 83 were in the toe webs, 110 were on the soles, and 7 were on the dorsal surfaces of the feet. Of these 7 dorsal lesions, 6 were extensions from lesions on the toe webs or soles and 1 was inconsistent with the 3 subtypes of tinea pedis. Among the 200 lesions, the main clinical symptom was maceration in 38 lesions, desquamation or scaling in 132 lesions, keratosis in 28 lesions, and blistering in 2 lesions. The dermatologist recorded the level of suspicion for tinea pedis as low in 68 lesions and high in 132.

According to the order in which the dermatologist took the 3 samples from each lesion, the KOH test was positive in 95 of the first set of 200 samples, 94 of the second set, and 86 of the third set; however, from the second set, the incremental yield (ie, the number of lesions in which the first KOH test was negative and the second was positive) was 10. The number of lesions in which the first and the second tests were negative and the third was positive was only 4. Therefore, the number of lesions with a positive KOH test was significantly increased from 95 to 105 by performing the second KOH test (P=.002). This number again increased from 105 to 109 when a third test was performed; however, this increase was not statistically significant (P=.125)(Table 1).

According to an evaluation that was not stratified by the dermatologist’s order of sampling, 72 lesions (36.0%) showed KOH test positivity in all 3 samples, 22 (11.0%) were positive in 2 samples, 15 (7.5%) were positive in only 1 sample, and 91 (45.5%) were positive in none of the samples (Table 2). When the data were subdivided based on the sites of the lesions, the toe web lesions (n=83) showed rates of 41.0%, 9.6%, and 4.8% for 3, 2, and 1 positive KOH tests, respectively. For the sole lesions (n=110), the rates were somewhat different at 31.8%, 11.8%, and 10.0%, respectively, but the difference was not statistically significant (P=.395).

For the subgroups based on the main clinical symptoms, the percentage of lesions having at least 1 positive KOH test from the 3 samples was 35.7% for the keratotic lesions (n=28). This rate was lower than macerated lesions (n=38) and desquamating or scaling lesions (n=132), which were 52.6% and 59.1%, respectively (Table 2). On the other hand, the percentage of lesions that produced only 1 or 2 positive KOH tests from the 3 samples was 25.0% for the keratotic lesions, which was higher than the rates for the macerated lesions and the desquamating or scaling lesions (13.1% and 18.9%, respectively). In particular, the difference between the keratotic lesions and the desquamating or scaling lesions in the distribution of the rates of 0, 1, 2, and 3 positive KOH tests was statistically significant (P=.019). The macerated, desquamating or scaling, keratotic, and blistering lesions are presented in the Figure.

If the dermatologist indicated a high suspicion of fungal infection, it was more likely that at least 1 of 3 KOH test results was positive. The rate of at least 1 positive test was 64.4% for the highly suspicious lesions (n=132) and 35.3% for the lesions with low suspicion of a fungal infection (n=68)(Table 2). The difference was statistically significant (P<.001). Conversely, if the suspicion was low, it was more likely that only 1 or 2 KOH tests were positive. The percentages of lesions having 3, 2, or 1 positive KOH tests were 14.7%, 8.8%, and 11.8%, respectively, for the low-suspicion lesions and 47.0%, 12.1%, and 5.3%, respectively, for the high-suspicion lesions. The difference was statistically significant (P<.001).

Comment

In the current study, we aimed to investigate if successive KOH tests provide an incremental diagnostic yield in the management of patients with clinically suspected tinea pedis and if these results differ among the subgroups of patients. Both in the evaluation taking into account the order of sampling and in the evaluation disregarding this order, we found that the second sample was necessary for all subgroups, and even the third sample was necessary for patients with keratotic lesions. The main limitation of the study was that we lacked a gold-standard technique (eg, a molecular-based technique); therefore, we are unable to comment on the false-negative and false-positive results of the successive KOH testing.

Summerbell et al¹¹ found in their study that in initial specimens of toenails with apparent lesions taken from 473 patients, the KOH test was 73.8% sensitive for dermatophytes, and this rate was only somewhat higher for cultures (74.6%). Arabatzis et al² investigated 92 skin, nail, and hair specimens from 67 patients with suspected dermatophytosis and found that the KOH test was superior to culture for the detection of dermatophytes (43% vs 33%). Moreover and more importantly, they noted that a real-time polymerase chain reaction (PCR) assay yielded a higher detection rate (51%).² In another study, Wisselink et al³ examined 1437 clinical samples and demonstrated a great increase in the detection of dermatophytes using a real-time PCR assay (48.5%) compared to culture (26.9%). However, PCR may not reflect active disease and could lead to false-positive results.^2,3 Therefore, the aforementioned weakness of our study will be overcome in further studies investigating the benefit of successive KOH testing compared to a molecular-based assay, such as the real-time PCR assay.

In this study, repeating the KOH test provided better results for achieving the diagnosis of tinea pedis in a large number of samples from clinically suspected lesions. Additionally, the distribution of 3, 2, or 1 positive results on the 3 KOH tests was different among the subgroups of lesions. Overall, positivity was less frequent in the keratotic lesions compared to the macerated or desquamating or scaling lesions. Moreover, positivity on all 3 tests also was less frequent in the keratotic lesions. Inversely, the frequency of samples with only 1 or 2 positive results was higher in this subgroup. The necessity for the second, even the third, tests was greater in this subgroup.

Our findings were consistent with the results of the studies performed with successive mycological tests on the nail specimens. Meireles et al¹ repeated 156 mycological nail tests 3 times and found the rate of positivity in the first test to be 19.9%. When the results of the first and second tests were combined, this rate increased to 28.2%, and when the results of all 3 tests were combined, it increased to 37.8%.¹ Gupta¹⁰ demonstrated that even a fourth culture provided an incremental diagnostic yield in the diagnosis of onychomycosis, yet 4 cultures may not be clinically practical. Furthermore, periodic acid–Schiff staining is a more effective measure of positivity in onychomycosis.¹⁵

Although the overall rate of positivity on the 3 tests in our study was unsurprisingly higher in lesions rated highly suspicious for a fungal infection, the rate of only 1 or 2 positive tests was surprisingly somewhat higher in low-suspicion lesions, which suggested that repeating the KOH test would be beneficial, even if the clinical suspicion for tinea pedis was low. The novel contribution of this study includes the finding that mycological information was markedly improved in highly suspicious tinea pedis lesions regardless of the infection site (Table 1) by using 3 successive KOH tests; the percentage of lesions with 1, 2, or 3 positive KOH tests was 5.3%, 12.1%, and 47.0%, respectively (Table 2). A single physician from a single geographical location introduces a limitation to the study for a variety of reasons, including bias in the cases chosen and possible overrepresentation of the causative organism due to region-specific incidence. It is unknown how different causative organisms affect KOH results. The lack of fungal culture results limits the value of this information.

 

Conclusion

In this study, we investigated the benefit of successive KOH testing in the laboratory diagnosis of tinea pedis and found that the use of second samples in particular provided a substantial increase in diagnostic yield. In other words, the utilization of successive KOH testing remarkably improved the diagnosis of tinea pedis. Therefore, we suggest that at least 2 samples of skin scrapings should be taken for the diagnosis of tinea pedis and that the number of samples should be at least 3 for keratotic lesions. However, further study by using a gold-standard method such as a molecular-based assay as well as taking the samples in daily or weekly intervals is recommended to achieve a more reliable result.

Acknowledgment

The authors would like to thank Gökçen Şahin (Adana, Turkey) for providing technical support in direct microscopic examination.

Take-Home Points

• Studies of AEs after orthopedic surgery commonly use composite AE outcomes.
• These types of outcomes treat AEs with different clinical significance similarly.
• This study created a single severity-weighted outcome that can be used to characterize the overall severity of a given patient’s postoperative course.
• Future studies may benefit from using this new severity-weighted outcome score.

Recently there has been an increase in the use of national databases for orthopedic surgery research.^1-4 Studies commonly compare rates of postoperative adverse events (AEs) across different demographic, comorbidity, and procedural characteristics.^5-23 Their conclusions often highlight different modifiable and/or nonmodifiable risk factors associated with the occurrence of postoperative events.

The several dozen AEs that have been investigated range from very severe (eg, death, myocardial infarction, coma) to less severe (eg, urinary tract infection [UTI], anemia requiring blood transfusion). A common approach for these studies is to consider many AEs together in the same analysis, asking a question such as, “What are risk factors for the occurrence of ‘adverse events’ after spine surgery?” Such studies test for associations with the occurrence of “any adverse event,” the occurrence of any “serious adverse event,” or similar composite outcomes. How common this type of study has become is indicated by the fact that in 2013 and 2014, at least 12 such studies were published in Clinical Orthopaedics and Related Research and the Journal of Bone and Joint Surgery,^5-14,21-23 and many more in other orthopedic journals.^15-20 However, there is a problem in using this type of composite outcome to perform such analyses: AEs with highly varying degrees of severity have identical impacts on the outcome variable, changing it from negative (“no adverse event”) to positive (“at least one adverse event”). As a result, the system may treat a very severe AE such as death and a very minor AE such as UTI similarly. Even in studies that use the slightly more specific composite outcome of “serious adverse events,” death and a nonlethal thromboembolic event would be treated similarly. Failure to differentiate these AEs in terms of their clinical significance detracts from the clinical applicability of conclusions drawn from studies using these types of composite AE outcomes.

In one of many examples that can be considered, a retrospective cohort study compared general and spinal anesthesia used in total knee arthroplasty.¹⁰ The rate of any AEs was higher with general anesthesia than with spinal anesthesia (12.34% vs 10.72%; P = .003). However, the only 2 specific AEs that had statistically significant differences were anemia requiring blood transfusion (6.07% vs 5.02%; P = .009) and superficial surgical-site infection (SSI; 0.92% vs 0.68%; P < .001). These 2 AEs are of relatively low severity; nevertheless, because these AEs are common, their differences constituted the majority of the difference in the rate of any AEs. In contrast, differences in the more severe AEs, such as death (0.11% vs 0.22%; P > .05), septic shock (0.14% vs 0.12%; P > .05), and myocardial infarction (0.20% vs 0.20%; P > .05), were small and not statistically significant. Had more weight been given to these more severe events, the outcome of the study likely would have been “no difference.”

To address this shortcoming in orthopedic research methodology, we created a severity-weighted outcome score that can be used to determine the overall “severity” of any given patient’s postoperative course. We also tested this novel outcome score for correlation with procedure type and patient characteristics using orthopedic patients from the American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP). Our intention is for database investigators to be able to use this outcome score in place of the composite outcomes that are dominating this type of research.

Methods

Generation of Severity Weights

Our method is described generally as utility weighting, assigning value weights reflective of overall impact to differing outcome states.²⁴ Parallel methods have been used to generate the disability weights used to determine disability-adjusted life years for the Global Burden of Disease project²⁵ and many other areas of health, economic, and policy research.

All orthopedic faculty members at 2 geographically disparate, large US academic institutions were invited to participate in a severity-weighting exercise. Each surgeon who agreed to participate performed the exercise independently.

Each participant was given a stack of 23 index cards, each listing the name and description of an AE monitored by ACS-NSQIP (Table 1).²⁶ In addition, in the upper right corner of each card was a box in which the participant could write a number. Each stack of cards was provided in a distinct randomized order. Written instructions for participants were exactly as follows:

• STEP 1: Please reorder the AE cards by your perception of “severity” for a patient experiencing that event after an orthopedic procedure.
• STEP 2: Once your cards are in order, please determine how many postoperative occurrences of each event you would “trade” for 1 patient experiencing postoperative death. Place this number of occurrences in the box in the upper right corner of each card.
• NOTES: As you consider each AE:
• Please consider an “average” occurrence of that AE, but note that in no case does the AE result in perioperative death.
• Please consider only the “severity” for the patient. (Do not consider the extent to which the event may be related to surgical error.)
• Please consider that the numbers you assign are relative to each other. Hence, if you would trade 20 of “event A” for 1 death, and if you would trade 40 of “event B” for 1 death, the implication is that you would trade 20 of “event A” for 40 of “event B.”
• You may readjust the order of your cards at any point.

Participants’ responses were recorded. For each number provided by each participant, the inverse (reciprocal) was taken and multiplied by 100%. This new number was taken to be the percentage severity of death that the given participant considered the given AE to embody. For example, as a hypothetical on one end of the spectrum, if a participant reported 1 (he/she would trade 1 AE X for 1 death), then the severity would be 1/1 × 100% = 100% of death, a very severe AE. Conversely, if a participant reported a very large number like 100,000 (he/she would trade 100,000 AEs X for 1 death), then the severity would be 1/100,000 × 100% = 0.001% of death, a very minor AE. More commonly, a participant will report a number like 25, which would translate to 4% of death (1/25 × 100% = 4%). For each AE, weights were then averaged across participants to derive a mean severity weight to be used to generate a novel composite outcome score.

Definition of Novel Composite Outcome Score

The novel composite outcome score would be expressed as a percentage to be interpreted as percentage severity of death, which we termed severity-weighted outcome relative to death (SWORD). For each patient, SWORD was defined as no AE (0%) or postoperative death (100%), with other AEs assigned mean severity weights based on faculty members’ survey responses. A patient with multiple AEs would be assigned the weight for the more severe AE. This method was chosen over summing the AE weights because in many cases the AEs were thought to overlap; hence, summing would be inappropriate. For example, generally a deep SSI would result in a return to the operating room, and one would not want to double-count this AE. Similarly, it would not make sense for a patient who died of a complication to have a SWORD of >100%, which would be the summing result.

Application to ACS-NSQIP Patients

ACS-NSQIP is a surgical registry that prospectively identifies patients undergoing major surgery at any of >500 institutions nationwide.^26,27 Patients are characterized at baseline and are followed for AEs over the first 30 postoperative days.

Patients undergoing any of 8 common orthopedic procedures were identified in the 2012 ACS-NSQIP database using International Classification of Diseases, Ninth Revision (ICD-9) codes and Current Procedural Terminology (CPT) codes (Table 2). Any patient with missing data was excluded from this population before analysis.

First, mean SWORD was calculated and reported for patients undergoing each of the 8 procedures. Analysis of variance (ANOVA) was used to test for associations of mean SWORD with type of procedure both before and after multivariate adjustment for demographics (sex; age in years, <40, 40-49, 50-59, 60-69, 70-79, 80-89, ≥90) and comorbidities (diabetes, hypertension, chronic obstructive pulmonary disease, exertional dyspnea, end-stage renal disease, congestive heart failure).

Second, patients undergoing the procedure with the highest mean SWORD (hip fracture surgery) were examined in depth. Among only these patients, multivariate ANOVA was used to test for associations of mean SWORD with the same demographics and comorbidities.

All statistical tests were 2-tailed. Significance was set at α = 0.05 (P < .05).

All 23 institution A faculty members (100%) and 24 (89%) of the 27 institution B faculty members completed the exercise.

Total number of participants was 47, and the overall response rate was 94%. Participant characteristics are listed in Table 3.

In the ACS-NSQIP database, 85,109 patients were identified on the basis of the initial inclusion criteria.

After patients with missing data were excluded, 85,031 remained for analysis. Patient characteristics are listed in Table 4.

 

Results

Figure 1 shows mean severity weights and standard errors generated from faculty responses. Mean (standard error) severity weight for UTI was 0.23% (0.08%); blood transfusion, 0.28% (0.09%); pneumonia, 0.55% (0.15%); hospital readmission, 0.59% (0.23%); wound dehiscence, 0.64% (0.17%); deep vein thrombosis, 0.64% (0.19%); superficial SSI, 0.68% (0.23%); return to operating room, 0.91% (0.29%); progressive renal insufficiency, 0.93% (0.27%); graft/prosthesis/flap failure, 1.20% (0.34%); unplanned intubation, 1.38% (0.53%); deep SSI, 1.45% (0.38%); failure to wean from ventilator, 1.45% (0.48%); organ/space SSI, 1.76% (0.46%); sepsis without shock, 1.77% (0.42%); peripheral nerve injury, 1.83% (0.47%); pulmonary embolism, 2.99% (0.76%); acute renal failure, 3.95% (0.85%); myocardial infarction, 4.16% (0.98%); septic shock, 7.17% (1.36%); stroke, 8.73% (1.74%); cardiac arrest requiring cardiopulmonary resuscitation, 9.97% (2.46%); and coma, 15.14% (3.04%).

Among ACS-NSQIP patients, mean SWORD ranged from 0.2% (elective anterior cervical decompression and fusion) to 6.0% (hip fracture surgery) (Figure 2).

Mean SWORD was associated with procedure type both before (P < .001) and after (P < .001) controlling for demographic and comorbidity differences between populations. Among ACS-NSQIP patients having hip fracture surgery, mean SWORD was independently associated with older age, male sex, and 4 of 6 tested comorbidities (Ps < .05) (Figure 3).

Discussion

The use of national databases in studies has become increasingly common in orthopedic surgery.^1-4

However, many of these studies use composite outcomes such as “any adverse events” and “serious adverse events” to generate primary results.^5-23 Such methods implicitly consider the severity of markedly different AEs (death, UTI) to be the same. Our study provides orthopedics researchers with a tool that can be used to overcome this methodologic deficit.

The academic orthopedic surgeons who participated in our severity-weighting exercise thought the various AEs have markedly different severities. The least severe AE (UTI) was considered 0.23% as severe as postoperative death, with other events spanning the range up to 15.14% as severe as death. This wide range of severities demonstrates the problem with composite outcomes that implicitly consider all AEs similarly severe. Use of these markedly disparate weights in the development of SWORD enables this outcome to be more clinically applicable than outcomes such as “any adverse events.”

SWORD was highly associated with procedure type both before and after adjustment for demographics and comorbidities. Among patients undergoing the highest SWORD procedure (hip fracture surgery), SWORD was also associated with age, sex, and 4 of 6 tested comorbidities. Together, our findings show how SWORD is intended to be used in studies: to identify demographic, comorbidity, and procedural risk factors for an adverse postoperative course. We propose that researchers use our weighted outcome as their primary outcome—it is more meaningful than the simpler composite outcomes commonly used.

Outside orthopedic surgery, a small series of studies has addressed severity weighting of postoperative AEs.^25,28-30 However, their approach was very different, as they were not designed to generate weights that could be transferred to future studies; rather, they simply compared severities of postoperative courses for patients within each individual study. In each study, a review of each original patient record was required, as the severity of each patient’s postoperative course was characterized according to the degree of any postoperative intervention—from no intervention to minor interventions such as placement of an intravenous catheter and major interventions such as endoscopic, radiologic, and surgical procedures. Only after the degree of intervention was defined could an outcome score be assigned to a given patient. However, databases do not depict the degree of intervention with nearly enough detail for this type of approach; they typically identify only occurrence or nonoccurrence of each event. Our work, which arose independently from this body of literature, enables an entirely different type of analysis. SWORD, which is not based on degree of intervention but on perceived severity of an “average” event, enables direct application of severity weights to large databases that store simple information on occurrence and nonoccurrence of specific AEs.

This study had several limitations. Most significantly, the generated severity weights were based on the surgeons’ subjective perceptions of severity, not on definitive assessments of the impacts of specific AEs on actual patients. We did not query the specialists who treat the complications or who present data on the costs and disabilities that may arise from these AEs. In addition, to develop our severity weighting scale, we queried faculty at only 2 institutions. A survey of surgeons throughout the United States would be more representative and would minimize selection bias. This is a potential research area. Another limitation is that scoring was subjective, based on surgeons’ perceptions of patients—in contrast to the Global Burden of Disease project, in which severity was based more objectively on epidemiologic data from >150 countries.

Orthopedic database research itself has often-noted limitations, including inability to sufficiently control for confounders, potential inaccuracies in data coding, limited follow-up, and lack of orthopedic-specific outcomes.^1-4,31-33 However, this research also has much to offer, has increased tremendously over the past several years, and is expected to continue to expand. Many of the limitations of database studies cannot be entirely reversed. In providing a system for weighting postoperative AEs, our study fills a methodologic void. Future studies in orthopedics may benefit from using the severity-weighted outcome score presented here. Other fields with growth in database research may consider using similar methods to create severity-weighting systems of their own.

Am J Orthop. 2017;46(4):E235-E243. Copyright Frontline Medical Communications Inc. 2017. All rights reserved.

Take-Home Points

• Studies of AEs after orthopedic surgery commonly use composite AE outcomes.
• These types of outcomes treat AEs with different clinical significance similarly.
• This study created a single severity-weighted outcome that can be used to characterize the overall severity of a given patient’s postoperative course.
• Future studies may benefit from using this new severity-weighted outcome score.

Recently there has been an increase in the use of national databases for orthopedic surgery research.^1-4 Studies commonly compare rates of postoperative adverse events (AEs) across different demographic, comorbidity, and procedural characteristics.^5-23 Their conclusions often highlight different modifiable and/or nonmodifiable risk factors associated with the occurrence of postoperative events.

The several dozen AEs that have been investigated range from very severe (eg, death, myocardial infarction, coma) to less severe (eg, urinary tract infection [UTI], anemia requiring blood transfusion). A common approach for these studies is to consider many AEs together in the same analysis, asking a question such as, “What are risk factors for the occurrence of ‘adverse events’ after spine surgery?” Such studies test for associations with the occurrence of “any adverse event,” the occurrence of any “serious adverse event,” or similar composite outcomes. How common this type of study has become is indicated by the fact that in 2013 and 2014, at least 12 such studies were published in Clinical Orthopaedics and Related Research and the Journal of Bone and Joint Surgery,^5-14,21-23 and many more in other orthopedic journals.^15-20 However, there is a problem in using this type of composite outcome to perform such analyses: AEs with highly varying degrees of severity have identical impacts on the outcome variable, changing it from negative (“no adverse event”) to positive (“at least one adverse event”). As a result, the system may treat a very severe AE such as death and a very minor AE such as UTI similarly. Even in studies that use the slightly more specific composite outcome of “serious adverse events,” death and a nonlethal thromboembolic event would be treated similarly. Failure to differentiate these AEs in terms of their clinical significance detracts from the clinical applicability of conclusions drawn from studies using these types of composite AE outcomes.

In one of many examples that can be considered, a retrospective cohort study compared general and spinal anesthesia used in total knee arthroplasty.¹⁰ The rate of any AEs was higher with general anesthesia than with spinal anesthesia (12.34% vs 10.72%; P = .003). However, the only 2 specific AEs that had statistically significant differences were anemia requiring blood transfusion (6.07% vs 5.02%; P = .009) and superficial surgical-site infection (SSI; 0.92% vs 0.68%; P < .001). These 2 AEs are of relatively low severity; nevertheless, because these AEs are common, their differences constituted the majority of the difference in the rate of any AEs. In contrast, differences in the more severe AEs, such as death (0.11% vs 0.22%; P > .05), septic shock (0.14% vs 0.12%; P > .05), and myocardial infarction (0.20% vs 0.20%; P > .05), were small and not statistically significant. Had more weight been given to these more severe events, the outcome of the study likely would have been “no difference.”

To address this shortcoming in orthopedic research methodology, we created a severity-weighted outcome score that can be used to determine the overall “severity” of any given patient’s postoperative course. We also tested this novel outcome score for correlation with procedure type and patient characteristics using orthopedic patients from the American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP). Our intention is for database investigators to be able to use this outcome score in place of the composite outcomes that are dominating this type of research.

Methods

Generation of Severity Weights

Our method is described generally as utility weighting, assigning value weights reflective of overall impact to differing outcome states.²⁴ Parallel methods have been used to generate the disability weights used to determine disability-adjusted life years for the Global Burden of Disease project²⁵ and many other areas of health, economic, and policy research.

All orthopedic faculty members at 2 geographically disparate, large US academic institutions were invited to participate in a severity-weighting exercise. Each surgeon who agreed to participate performed the exercise independently.

Each participant was given a stack of 23 index cards, each listing the name and description of an AE monitored by ACS-NSQIP (Table 1).²⁶ In addition, in the upper right corner of each card was a box in which the participant could write a number. Each stack of cards was provided in a distinct randomized order. Written instructions for participants were exactly as follows:

• STEP 1: Please reorder the AE cards by your perception of “severity” for a patient experiencing that event after an orthopedic procedure.
• STEP 2: Once your cards are in order, please determine how many postoperative occurrences of each event you would “trade” for 1 patient experiencing postoperative death. Place this number of occurrences in the box in the upper right corner of each card.
• NOTES: As you consider each AE:
• Please consider an “average” occurrence of that AE, but note that in no case does the AE result in perioperative death.
• Please consider only the “severity” for the patient. (Do not consider the extent to which the event may be related to surgical error.)
• Please consider that the numbers you assign are relative to each other. Hence, if you would trade 20 of “event A” for 1 death, and if you would trade 40 of “event B” for 1 death, the implication is that you would trade 20 of “event A” for 40 of “event B.”
• You may readjust the order of your cards at any point.

Participants’ responses were recorded. For each number provided by each participant, the inverse (reciprocal) was taken and multiplied by 100%. This new number was taken to be the percentage severity of death that the given participant considered the given AE to embody. For example, as a hypothetical on one end of the spectrum, if a participant reported 1 (he/she would trade 1 AE X for 1 death), then the severity would be 1/1 × 100% = 100% of death, a very severe AE. Conversely, if a participant reported a very large number like 100,000 (he/she would trade 100,000 AEs X for 1 death), then the severity would be 1/100,000 × 100% = 0.001% of death, a very minor AE. More commonly, a participant will report a number like 25, which would translate to 4% of death (1/25 × 100% = 4%). For each AE, weights were then averaged across participants to derive a mean severity weight to be used to generate a novel composite outcome score.

Definition of Novel Composite Outcome Score

The novel composite outcome score would be expressed as a percentage to be interpreted as percentage severity of death, which we termed severity-weighted outcome relative to death (SWORD). For each patient, SWORD was defined as no AE (0%) or postoperative death (100%), with other AEs assigned mean severity weights based on faculty members’ survey responses. A patient with multiple AEs would be assigned the weight for the more severe AE. This method was chosen over summing the AE weights because in many cases the AEs were thought to overlap; hence, summing would be inappropriate. For example, generally a deep SSI would result in a return to the operating room, and one would not want to double-count this AE. Similarly, it would not make sense for a patient who died of a complication to have a SWORD of >100%, which would be the summing result.

Application to ACS-NSQIP Patients

ACS-NSQIP is a surgical registry that prospectively identifies patients undergoing major surgery at any of >500 institutions nationwide.^26,27 Patients are characterized at baseline and are followed for AEs over the first 30 postoperative days.

Patients undergoing any of 8 common orthopedic procedures were identified in the 2012 ACS-NSQIP database using International Classification of Diseases, Ninth Revision (ICD-9) codes and Current Procedural Terminology (CPT) codes (Table 2). Any patient with missing data was excluded from this population before analysis.

First, mean SWORD was calculated and reported for patients undergoing each of the 8 procedures. Analysis of variance (ANOVA) was used to test for associations of mean SWORD with type of procedure both before and after multivariate adjustment for demographics (sex; age in years, <40, 40-49, 50-59, 60-69, 70-79, 80-89, ≥90) and comorbidities (diabetes, hypertension, chronic obstructive pulmonary disease, exertional dyspnea, end-stage renal disease, congestive heart failure).

Second, patients undergoing the procedure with the highest mean SWORD (hip fracture surgery) were examined in depth. Among only these patients, multivariate ANOVA was used to test for associations of mean SWORD with the same demographics and comorbidities.

All statistical tests were 2-tailed. Significance was set at α = 0.05 (P < .05).

All 23 institution A faculty members (100%) and 24 (89%) of the 27 institution B faculty members completed the exercise.

Total number of participants was 47, and the overall response rate was 94%. Participant characteristics are listed in Table 3.

In the ACS-NSQIP database, 85,109 patients were identified on the basis of the initial inclusion criteria.

After patients with missing data were excluded, 85,031 remained for analysis. Patient characteristics are listed in Table 4.

 

Results

Figure 1 shows mean severity weights and standard errors generated from faculty responses. Mean (standard error) severity weight for UTI was 0.23% (0.08%); blood transfusion, 0.28% (0.09%); pneumonia, 0.55% (0.15%); hospital readmission, 0.59% (0.23%); wound dehiscence, 0.64% (0.17%); deep vein thrombosis, 0.64% (0.19%); superficial SSI, 0.68% (0.23%); return to operating room, 0.91% (0.29%); progressive renal insufficiency, 0.93% (0.27%); graft/prosthesis/flap failure, 1.20% (0.34%); unplanned intubation, 1.38% (0.53%); deep SSI, 1.45% (0.38%); failure to wean from ventilator, 1.45% (0.48%); organ/space SSI, 1.76% (0.46%); sepsis without shock, 1.77% (0.42%); peripheral nerve injury, 1.83% (0.47%); pulmonary embolism, 2.99% (0.76%); acute renal failure, 3.95% (0.85%); myocardial infarction, 4.16% (0.98%); septic shock, 7.17% (1.36%); stroke, 8.73% (1.74%); cardiac arrest requiring cardiopulmonary resuscitation, 9.97% (2.46%); and coma, 15.14% (3.04%).

Among ACS-NSQIP patients, mean SWORD ranged from 0.2% (elective anterior cervical decompression and fusion) to 6.0% (hip fracture surgery) (Figure 2).

Mean SWORD was associated with procedure type both before (P < .001) and after (P < .001) controlling for demographic and comorbidity differences between populations. Among ACS-NSQIP patients having hip fracture surgery, mean SWORD was independently associated with older age, male sex, and 4 of 6 tested comorbidities (Ps < .05) (Figure 3).

Discussion

The use of national databases in studies has become increasingly common in orthopedic surgery.^1-4

However, many of these studies use composite outcomes such as “any adverse events” and “serious adverse events” to generate primary results.^5-23 Such methods implicitly consider the severity of markedly different AEs (death, UTI) to be the same. Our study provides orthopedics researchers with a tool that can be used to overcome this methodologic deficit.

The academic orthopedic surgeons who participated in our severity-weighting exercise thought the various AEs have markedly different severities. The least severe AE (UTI) was considered 0.23% as severe as postoperative death, with other events spanning the range up to 15.14% as severe as death. This wide range of severities demonstrates the problem with composite outcomes that implicitly consider all AEs similarly severe. Use of these markedly disparate weights in the development of SWORD enables this outcome to be more clinically applicable than outcomes such as “any adverse events.”

SWORD was highly associated with procedure type both before and after adjustment for demographics and comorbidities. Among patients undergoing the highest SWORD procedure (hip fracture surgery), SWORD was also associated with age, sex, and 4 of 6 tested comorbidities. Together, our findings show how SWORD is intended to be used in studies: to identify demographic, comorbidity, and procedural risk factors for an adverse postoperative course. We propose that researchers use our weighted outcome as their primary outcome—it is more meaningful than the simpler composite outcomes commonly used.

Outside orthopedic surgery, a small series of studies has addressed severity weighting of postoperative AEs.^25,28-30 However, their approach was very different, as they were not designed to generate weights that could be transferred to future studies; rather, they simply compared severities of postoperative courses for patients within each individual study. In each study, a review of each original patient record was required, as the severity of each patient’s postoperative course was characterized according to the degree of any postoperative intervention—from no intervention to minor interventions such as placement of an intravenous catheter and major interventions such as endoscopic, radiologic, and surgical procedures. Only after the degree of intervention was defined could an outcome score be assigned to a given patient. However, databases do not depict the degree of intervention with nearly enough detail for this type of approach; they typically identify only occurrence or nonoccurrence of each event. Our work, which arose independently from this body of literature, enables an entirely different type of analysis. SWORD, which is not based on degree of intervention but on perceived severity of an “average” event, enables direct application of severity weights to large databases that store simple information on occurrence and nonoccurrence of specific AEs.

This study had several limitations. Most significantly, the generated severity weights were based on the surgeons’ subjective perceptions of severity, not on definitive assessments of the impacts of specific AEs on actual patients. We did not query the specialists who treat the complications or who present data on the costs and disabilities that may arise from these AEs. In addition, to develop our severity weighting scale, we queried faculty at only 2 institutions. A survey of surgeons throughout the United States would be more representative and would minimize selection bias. This is a potential research area. Another limitation is that scoring was subjective, based on surgeons’ perceptions of patients—in contrast to the Global Burden of Disease project, in which severity was based more objectively on epidemiologic data from >150 countries.

Orthopedic database research itself has often-noted limitations, including inability to sufficiently control for confounders, potential inaccuracies in data coding, limited follow-up, and lack of orthopedic-specific outcomes.^1-4,31-33 However, this research also has much to offer, has increased tremendously over the past several years, and is expected to continue to expand. Many of the limitations of database studies cannot be entirely reversed. In providing a system for weighting postoperative AEs, our study fills a methodologic void. Future studies in orthopedics may benefit from using the severity-weighted outcome score presented here. Other fields with growth in database research may consider using similar methods to create severity-weighting systems of their own.

Am J Orthop. 2017;46(4):E235-E243. Copyright Frontline Medical Communications Inc. 2017. All rights reserved.

Take-Home Points

• Studies of AEs after orthopedic surgery commonly use composite AE outcomes.
• These types of outcomes treat AEs with different clinical significance similarly.
• This study created a single severity-weighted outcome that can be used to characterize the overall severity of a given patient’s postoperative course.
• Future studies may benefit from using this new severity-weighted outcome score.

Recently there has been an increase in the use of national databases for orthopedic surgery research.^1-4 Studies commonly compare rates of postoperative adverse events (AEs) across different demographic, comorbidity, and procedural characteristics.^5-23 Their conclusions often highlight different modifiable and/or nonmodifiable risk factors associated with the occurrence of postoperative events.

The several dozen AEs that have been investigated range from very severe (eg, death, myocardial infarction, coma) to less severe (eg, urinary tract infection [UTI], anemia requiring blood transfusion). A common approach for these studies is to consider many AEs together in the same analysis, asking a question such as, “What are risk factors for the occurrence of ‘adverse events’ after spine surgery?” Such studies test for associations with the occurrence of “any adverse event,” the occurrence of any “serious adverse event,” or similar composite outcomes. How common this type of study has become is indicated by the fact that in 2013 and 2014, at least 12 such studies were published in Clinical Orthopaedics and Related Research and the Journal of Bone and Joint Surgery,^5-14,21-23 and many more in other orthopedic journals.^15-20 However, there is a problem in using this type of composite outcome to perform such analyses: AEs with highly varying degrees of severity have identical impacts on the outcome variable, changing it from negative (“no adverse event”) to positive (“at least one adverse event”). As a result, the system may treat a very severe AE such as death and a very minor AE such as UTI similarly. Even in studies that use the slightly more specific composite outcome of “serious adverse events,” death and a nonlethal thromboembolic event would be treated similarly. Failure to differentiate these AEs in terms of their clinical significance detracts from the clinical applicability of conclusions drawn from studies using these types of composite AE outcomes.

In one of many examples that can be considered, a retrospective cohort study compared general and spinal anesthesia used in total knee arthroplasty.¹⁰ The rate of any AEs was higher with general anesthesia than with spinal anesthesia (12.34% vs 10.72%; P = .003). However, the only 2 specific AEs that had statistically significant differences were anemia requiring blood transfusion (6.07% vs 5.02%; P = .009) and superficial surgical-site infection (SSI; 0.92% vs 0.68%; P < .001). These 2 AEs are of relatively low severity; nevertheless, because these AEs are common, their differences constituted the majority of the difference in the rate of any AEs. In contrast, differences in the more severe AEs, such as death (0.11% vs 0.22%; P > .05), septic shock (0.14% vs 0.12%; P > .05), and myocardial infarction (0.20% vs 0.20%; P > .05), were small and not statistically significant. Had more weight been given to these more severe events, the outcome of the study likely would have been “no difference.”

To address this shortcoming in orthopedic research methodology, we created a severity-weighted outcome score that can be used to determine the overall “severity” of any given patient’s postoperative course. We also tested this novel outcome score for correlation with procedure type and patient characteristics using orthopedic patients from the American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP). Our intention is for database investigators to be able to use this outcome score in place of the composite outcomes that are dominating this type of research.

Methods

Generation of Severity Weights

Our method is described generally as utility weighting, assigning value weights reflective of overall impact to differing outcome states.²⁴ Parallel methods have been used to generate the disability weights used to determine disability-adjusted life years for the Global Burden of Disease project²⁵ and many other areas of health, economic, and policy research.

All orthopedic faculty members at 2 geographically disparate, large US academic institutions were invited to participate in a severity-weighting exercise. Each surgeon who agreed to participate performed the exercise independently.

Each participant was given a stack of 23 index cards, each listing the name and description of an AE monitored by ACS-NSQIP (Table 1).²⁶ In addition, in the upper right corner of each card was a box in which the participant could write a number. Each stack of cards was provided in a distinct randomized order. Written instructions for participants were exactly as follows:

• STEP 1: Please reorder the AE cards by your perception of “severity” for a patient experiencing that event after an orthopedic procedure.
• STEP 2: Once your cards are in order, please determine how many postoperative occurrences of each event you would “trade” for 1 patient experiencing postoperative death. Place this number of occurrences in the box in the upper right corner of each card.
• NOTES: As you consider each AE:
• Please consider an “average” occurrence of that AE, but note that in no case does the AE result in perioperative death.
• Please consider only the “severity” for the patient. (Do not consider the extent to which the event may be related to surgical error.)
• Please consider that the numbers you assign are relative to each other. Hence, if you would trade 20 of “event A” for 1 death, and if you would trade 40 of “event B” for 1 death, the implication is that you would trade 20 of “event A” for 40 of “event B.”
• You may readjust the order of your cards at any point.

Participants’ responses were recorded. For each number provided by each participant, the inverse (reciprocal) was taken and multiplied by 100%. This new number was taken to be the percentage severity of death that the given participant considered the given AE to embody. For example, as a hypothetical on one end of the spectrum, if a participant reported 1 (he/she would trade 1 AE X for 1 death), then the severity would be 1/1 × 100% = 100% of death, a very severe AE. Conversely, if a participant reported a very large number like 100,000 (he/she would trade 100,000 AEs X for 1 death), then the severity would be 1/100,000 × 100% = 0.001% of death, a very minor AE. More commonly, a participant will report a number like 25, which would translate to 4% of death (1/25 × 100% = 4%). For each AE, weights were then averaged across participants to derive a mean severity weight to be used to generate a novel composite outcome score.

Definition of Novel Composite Outcome Score

The novel composite outcome score would be expressed as a percentage to be interpreted as percentage severity of death, which we termed severity-weighted outcome relative to death (SWORD). For each patient, SWORD was defined as no AE (0%) or postoperative death (100%), with other AEs assigned mean severity weights based on faculty members’ survey responses. A patient with multiple AEs would be assigned the weight for the more severe AE. This method was chosen over summing the AE weights because in many cases the AEs were thought to overlap; hence, summing would be inappropriate. For example, generally a deep SSI would result in a return to the operating room, and one would not want to double-count this AE. Similarly, it would not make sense for a patient who died of a complication to have a SWORD of >100%, which would be the summing result.

Application to ACS-NSQIP Patients

ACS-NSQIP is a surgical registry that prospectively identifies patients undergoing major surgery at any of >500 institutions nationwide.^26,27 Patients are characterized at baseline and are followed for AEs over the first 30 postoperative days.

Patients undergoing any of 8 common orthopedic procedures were identified in the 2012 ACS-NSQIP database using International Classification of Diseases, Ninth Revision (ICD-9) codes and Current Procedural Terminology (CPT) codes (Table 2). Any patient with missing data was excluded from this population before analysis.

First, mean SWORD was calculated and reported for patients undergoing each of the 8 procedures. Analysis of variance (ANOVA) was used to test for associations of mean SWORD with type of procedure both before and after multivariate adjustment for demographics (sex; age in years, <40, 40-49, 50-59, 60-69, 70-79, 80-89, ≥90) and comorbidities (diabetes, hypertension, chronic obstructive pulmonary disease, exertional dyspnea, end-stage renal disease, congestive heart failure).

Second, patients undergoing the procedure with the highest mean SWORD (hip fracture surgery) were examined in depth. Among only these patients, multivariate ANOVA was used to test for associations of mean SWORD with the same demographics and comorbidities.

All statistical tests were 2-tailed. Significance was set at α = 0.05 (P < .05).

All 23 institution A faculty members (100%) and 24 (89%) of the 27 institution B faculty members completed the exercise.

Total number of participants was 47, and the overall response rate was 94%. Participant characteristics are listed in Table 3.

In the ACS-NSQIP database, 85,109 patients were identified on the basis of the initial inclusion criteria.

After patients with missing data were excluded, 85,031 remained for analysis. Patient characteristics are listed in Table 4.

 

Results

Figure 1 shows mean severity weights and standard errors generated from faculty responses. Mean (standard error) severity weight for UTI was 0.23% (0.08%); blood transfusion, 0.28% (0.09%); pneumonia, 0.55% (0.15%); hospital readmission, 0.59% (0.23%); wound dehiscence, 0.64% (0.17%); deep vein thrombosis, 0.64% (0.19%); superficial SSI, 0.68% (0.23%); return to operating room, 0.91% (0.29%); progressive renal insufficiency, 0.93% (0.27%); graft/prosthesis/flap failure, 1.20% (0.34%); unplanned intubation, 1.38% (0.53%); deep SSI, 1.45% (0.38%); failure to wean from ventilator, 1.45% (0.48%); organ/space SSI, 1.76% (0.46%); sepsis without shock, 1.77% (0.42%); peripheral nerve injury, 1.83% (0.47%); pulmonary embolism, 2.99% (0.76%); acute renal failure, 3.95% (0.85%); myocardial infarction, 4.16% (0.98%); septic shock, 7.17% (1.36%); stroke, 8.73% (1.74%); cardiac arrest requiring cardiopulmonary resuscitation, 9.97% (2.46%); and coma, 15.14% (3.04%).

Among ACS-NSQIP patients, mean SWORD ranged from 0.2% (elective anterior cervical decompression and fusion) to 6.0% (hip fracture surgery) (Figure 2).

Mean SWORD was associated with procedure type both before (P < .001) and after (P < .001) controlling for demographic and comorbidity differences between populations. Among ACS-NSQIP patients having hip fracture surgery, mean SWORD was independently associated with older age, male sex, and 4 of 6 tested comorbidities (Ps < .05) (Figure 3).

Discussion

The use of national databases in studies has become increasingly common in orthopedic surgery.^1-4

However, many of these studies use composite outcomes such as “any adverse events” and “serious adverse events” to generate primary results.^5-23 Such methods implicitly consider the severity of markedly different AEs (death, UTI) to be the same. Our study provides orthopedics researchers with a tool that can be used to overcome this methodologic deficit.

The academic orthopedic surgeons who participated in our severity-weighting exercise thought the various AEs have markedly different severities. The least severe AE (UTI) was considered 0.23% as severe as postoperative death, with other events spanning the range up to 15.14% as severe as death. This wide range of severities demonstrates the problem with composite outcomes that implicitly consider all AEs similarly severe. Use of these markedly disparate weights in the development of SWORD enables this outcome to be more clinically applicable than outcomes such as “any adverse events.”

SWORD was highly associated with procedure type both before and after adjustment for demographics and comorbidities. Among patients undergoing the highest SWORD procedure (hip fracture surgery), SWORD was also associated with age, sex, and 4 of 6 tested comorbidities. Together, our findings show how SWORD is intended to be used in studies: to identify demographic, comorbidity, and procedural risk factors for an adverse postoperative course. We propose that researchers use our weighted outcome as their primary outcome—it is more meaningful than the simpler composite outcomes commonly used.

Outside orthopedic surgery, a small series of studies has addressed severity weighting of postoperative AEs.^25,28-30 However, their approach was very different, as they were not designed to generate weights that could be transferred to future studies; rather, they simply compared severities of postoperative courses for patients within each individual study. In each study, a review of each original patient record was required, as the severity of each patient’s postoperative course was characterized according to the degree of any postoperative intervention—from no intervention to minor interventions such as placement of an intravenous catheter and major interventions such as endoscopic, radiologic, and surgical procedures. Only after the degree of intervention was defined could an outcome score be assigned to a given patient. However, databases do not depict the degree of intervention with nearly enough detail for this type of approach; they typically identify only occurrence or nonoccurrence of each event. Our work, which arose independently from this body of literature, enables an entirely different type of analysis. SWORD, which is not based on degree of intervention but on perceived severity of an “average” event, enables direct application of severity weights to large databases that store simple information on occurrence and nonoccurrence of specific AEs.

This study had several limitations. Most significantly, the generated severity weights were based on the surgeons’ subjective perceptions of severity, not on definitive assessments of the impacts of specific AEs on actual patients. We did not query the specialists who treat the complications or who present data on the costs and disabilities that may arise from these AEs. In addition, to develop our severity weighting scale, we queried faculty at only 2 institutions. A survey of surgeons throughout the United States would be more representative and would minimize selection bias. This is a potential research area. Another limitation is that scoring was subjective, based on surgeons’ perceptions of patients—in contrast to the Global Burden of Disease project, in which severity was based more objectively on epidemiologic data from >150 countries.

Orthopedic database research itself has often-noted limitations, including inability to sufficiently control for confounders, potential inaccuracies in data coding, limited follow-up, and lack of orthopedic-specific outcomes.^1-4,31-33 However, this research also has much to offer, has increased tremendously over the past several years, and is expected to continue to expand. Many of the limitations of database studies cannot be entirely reversed. In providing a system for weighting postoperative AEs, our study fills a methodologic void. Future studies in orthopedics may benefit from using the severity-weighted outcome score presented here. Other fields with growth in database research may consider using similar methods to create severity-weighting systems of their own.

Am J Orthop. 2017;46(4):E235-E243. Copyright Frontline Medical Communications Inc. 2017. All rights reserved.

From the Anne Arundel Health System Research Institute, Annapolis, MD.

 

Abstract

• Objective: To measure clinical outcomes associated with heparin-induced thrombocytopenia (HIT) and acquisition costs of heparin after implementing a new order set promoting unfractionated heparin (UFH) use instead of low-molecular-weight heparin (LMWH) for venous thromboembolism (VTE) prophylaxis.
• Methods: This was single-center, retrospective, pre-post intervention analysis utilizing pharmacy, laboratory, and clinical data sources. Subjects were patients receiving VTE thromboprophyalxis with heparin at an acute care hospital. Usage rates for UFH and LMWH, acquisition costs for heparins, number of HIT assays, best practice advisories for HIT, and confirmed cases of HIT and HIT with thrombosis were assessed.
• Results: After order set intervention, UFH use increased from 43% of all prophylaxis orders to 86%. Net annual savings in acquisition costs for VTE prophylaxis was $131,000. After the intervention, HIT best practice advisories and number of monthly HIT assays fell 35% and 15%, respectively. In the 9-month pre-intervention period, HIT and HITT occurred in zero of 6717 patients receiving VTE prophylaxis. In the 25 months of post-intervention follow-up, HIT occurred in 3 of 44,240 patients (P = 0.86) receiving VTE prophylaxis, 2 of whom had HITT, all after receiving UFH. The median duration of UFH and LMWH use was 3.0 and 3.5 days, respectively.
• Conclusion: UFH use in hospitals can be safely maintained or increased among patient subpopulations that are not at high risk for HIT. A more nuanced approach to prophylaxis, taking into account individual patient risk and expected duration of therapy, may provide desired cost savings without provoking HIT.

Key words: heparin; heparin-induced thrombocytopenia; venous thromboembolism prophylaxis; cost-effectiveness.

 

Heparin-induced thrombocytopenia (HIT) and its more severe clinical complication, HIT with thrombosis (HITT), complicate the use of heparin products for venous thromboembolic (VTE) prophylaxis. The clinical characteristics and time course of thrombocytopenia in relation to heparin are well characterized (typically 30%–50% drop in platelet count 5–10 days after exposure), if not absolute. Risk calculation tools help to judge the clinical probability and guide ordering of appropriate confirmatory tests [1]. The incidence of HIT is higher with unfractionated heparin (UFH) than with low-molecular-weight heparin (LMWH). A meta-analysis of 5 randomized or prospective nonrandomized trials indicated a risk of 2.6% (95% CI, 1.5%–3.8%) for UFH and 0.2% (95% CI, 0.1%–0.4%) for LMWH [2], though the analyzed studies were heavily weighted by studies of orthopedic surgery patients, a high-risk group. However, not all patients are at equal risk for HIT, suggesting that LMWH may not be necessary for all patients [3]. Unfortunately, LMWH is considerably more expensive for hospitals to purchase than UFH, raising costs for a prophylactic treatment that is widely utilized. However, the higher incidence of HIT and HITT associated with UFH can erode any cost savings because of the additional cost of diagnosing HIT and need for temporary or long-term treatment with even more expensive alternative anticoagulants. Indeed, a recent retrospective study suggested that the excess costs of evaluating and treating HIT were approximately $267,000 per year in Canadian dollars [4].But contrary data has also been reported. A retrospective study of the consequences of increased prophylactic UFH use found no increase in ordered HIT assays or in the results of HIT testing or of inferred positive cases despite a growth of 71% in the number of patients receiving UFH prophylaxis [5].

In 2013, the pharmacy and therapeutics committee made a decision to encourage the use of UFH over LMWH for VTE prophylaxis by making changes to order sets to favor UFH over LMWH (enoxaparin). Given the uncertainty about excess risk of HIT, a monitoring work group was created to assess for any increase of either HIT or HITT that might follow, including any patient readmitted with thrombosis within 30 days of a discharge. In this paper, we report the impact of a hospital-wide conversion to UFH for VTE prophylaxis on the incidence of VTE, HIT, and HITT and acquisition costs of UFH and LMWH and use of alternative prophylactic anticoagulant medications.

Methods

Setting

Anne Arundel Medical Center is a 383-bed acute care hospital with about 30,000 adult admissions and 10,000 inpatient surgeries annually. The average length of stay is approximately 3.6 days with a patient median age of 59 years. Caucasians comprise 75.3% of the admitted populations and African Americans 21.4%. Most patients are on Medicare (59%), while 29.5% have private insurance, 6.6% are on Medicaid, and 4.7% self-pay. The 9 most common medical principal diagnoses are sepsis, heart failure, chronic obstructive pulmonary disease, pneumonia, myocardial infarction, ischemic stroke, urinary tract infection, cardiac arrhythmia, and other infection. The 6 most common procedures include newborn delivery (with and without caesarean section), joint replacement surgery, bariatric procedures, cardiac catheterizations, other abdominal surgeries, and thoracotomy. The predominant medical care model is internal medicine and physician assistant acute care hospitalists attending both medicine and surgical patients. Obstetrical hospitalists care for admitted obstetric patients. Patients admitted to the intensive care units had only critical care trained physician specialists as attending physicians. No trainees cared for the patients described in this study.

P&T Committee

The P&T committee is a multidisciplinary group of health care professionals selected for appointment by the chairs of the committee (chair of medicine and director of pharmacy) and approved by the president of the medical staff. The committee has oversight responsibility for all medication policies, order sets involving medications, as well as the monitoring of clinical outcomes as they regard medications.

Electronic Medical Record and Best Practice Advisory

Throughout this study period both pre-and post-intervention, the EMR in use was Epic (Verona WI), used for all ordering and lab results. A best practice advisory was in place in the EMR that alerted providers to all cases of thrombocytopenia < 100,000/mm³ when there was concurrent order for any heparin. The best practice advisory highlighted the thrombocytopenia, advised the providers to consider HIT as a diagnosis and to order confirmation tests if clinically appropriate, providing a direct link to the HIT assay order screen. The best practice advisory did not access information from prior admissions where heparin might have been used nor determine the percentage drop from the baseline platelet count.

HIT Case Definition and Assays

The 2 laboratory tests for HIT on which this study is based are the heparin-induced platelet antibody test (also known as anti-PF4) and the serotonin release assay. The heparin-induced platelet antibody test is an enzyme-linked immunosorbent assay (ELISA) that detects IgG, IgM, and IgA antibodies against the platelet factor 4 (PF4/heparin complex). This test was reported as positive if the optical density was 0.4 or higher and generated an automatic request for a serotonin release assay (SRA), which is a functional assay that measures heparin-dependent platelet activation. The decision to order the SRA was therefore a “reflex” test and not made with any knowledge of clinical characteristics of the case. The HIT assays were performed by a reference lab, Quest Diagnostics, in the Chantilly, VA facility. HIT was said to be present when both a characteristic pattern of thrombocytopenia occurring after heparin use was seen [1]and when the confirmatory SRA was positive at a level of > 20% release.

 

Order Set Modifications

After the P&T committee decision to emphasize UFH for VTE prophylaxis in October 2013, the relevant electronic order sets were altered to highlight the fact that UFH was the first choice for VTE prophylaxis. The order sets still allowed LMWH (enoxaparin) or alternative anticoagulants at the prescribers’ discretion but indicated they were a second choice. Doses of UFH and LMWH in the order sets were standard based upon weight and estimates of creatinine clearance and, in the case of dosing frequency for UFH, based upon the risk of VTE. Order sets for the therapeutic treatment of VTE were not changed.

Data Collection and Analysis

The clinical research committee, the local oversight board for research and performance improvement analyses, reviewed this project and determined that it qualified as a performance improvement analysis based upon the standards of the U.S. Office of Human Research Protections. Some data were extracted from patient medical records and stored in a customized and password-protected database. Access to the database was limited to members of the analysis team and stripped of all patient identifiers under the HIPAA privacy rule standard for de-identification from 45 CFR 164.514(b) immediately following the collection of all data elements from the medical record.

An internal pharmacy database was used to determine the volume and actual acquisition cost of prophylactic anticoagulant doses administered during both pre- and post-intervention time periods. To determine if clinical suspicion for HIT increased after the intervention, a definitive listing of all ordered HIT assays was obtained from laboratory billing records for the 9 months (January 2013–September 2013) before the conversion and for 25 months after the intervention (beginning in November 2013 so as not to include the conversion month). To determine if the HIT assays were associated with a higher risk score, we identified all cases in which the HIT assay was ordered and retroactively measured the probability score known as the 4T score [1].Simultaneously, separate clinical work groups reviewed all cases of hospital-acquired thrombosis, whatever their cause, including patients readmitted with thrombosis up to 30 days after discharge and episodes of bleeding due to anti-coagulant use. A chi square analysis of the incidence of HIT pre- and post-intervention was performed.

Results

Heparin Use and Acquisition Costs

Annual hospital admissions and patient-days both increased 3% during the study time period to 29,975 annual adult admissions and 96,426 hospital days. However, the number of patients receiving any prophylactic anticoagulant increased 15% over the time period as a result of a simultaneous anti-VTE education campaign. After the modification in order sets there was a rapid increase in orders for prophylactic UFH and a corresponding decrease in the use of LMWH (Figure 1). A few patients received both at different times during their hospitalization. Prophylactic UFH orders increased from 43% of all prophylactic use pre-intervention to 86% by the second year of the intervention. There was a corresponding reduction in use of LMWH from 53% of all prophylaxis orders to 11% after the intervention. Other anticoagulants, fondaparinux and bivalrudin, together composed 5% and 4% of all prophylactic heparin in the pre- and post-intervention time periods, respectively. Pharmacy acquisition costs averaged $3.50 for daily doses of heparin and $24 for LMWH. With the shift in ordering pattern, acquisition costs of prophylactic LMWH fell by $165,000 annually with a concurrent offsetting increase in acquisition costs of UFH of $34,000 for a net savings of $131,000. The median duration of use was 3.5 days and 3.0 days for UFH and LMWH, respectively. Only 25% of treated patients had heparin exposure of > 5 days. Bleeding complications from any prophylaxis regimen were < 0.5% in both pre- and post-intervention time periods.

HIT Assays and Incidence of HIT and HITT

HIT assays results returned to the chart within a median of 4 days for anti-platelet factor 4 and 5 days for SRA. Data on the number of HIT assays ordered was missing for 1 month in 2013 and 1 month in 2014 due to missing laboratory billing data. On average, the best practice advisory for HIT fired 35% less frequently after the intervention (Figure 2). The number of ordered serum ELISA HIT assays decreased slightly after the intervention from a mean of 18.2 per month to 15.4 per month despite the increase in UFH use. The distribution of the retrospectively measured 4T scores for patients with HIT testing is shown in 

the Table. There were no differences in the retrospectively calculated scores between the pre- and post-intervention time periods. Most patients had a moderate risk 4T score prior to having HIT assay ordered.

In the 9 months pre-intervention, HIT and HITT occurred in zero of 6717 patients receiving at least 1 dose of VTE prophylaxis. In the 25 months of post-intervention follow-up, 44,240 patients received prophylaxis with either heparin. HIT (clinical suspicion with positive antibody and confirmatory SRA) occurred in 3 patients, 2 of whom had HITT, all after UFH. This incidence was not statistically significant using chi square analysis (P = 0.86).

 

Discussion

Because the efficacy of UFH and LMWH for VTE prophylaxis are equivalent [6],choosing between them involves many factors including patient-level risk factors such as renal function, risk of bleeding, as well as other considerations such as nursing time, patient preference, risk of HIT, and acquisition cost. Indeed, the most recent version of the American College of Chest Physicians guidelines for prophylaxis against VTE note that both drugs are recommended with an evidence grade of IB [7].Cost is among the considerations considered appropriate in choosing among agents. The difference in acquisition costs of > $20 per patient per day can have a major financial impact on hospital’s pharmacy budget and may be decisive. But a focus only on acquisition cost is short sighted as the 2 medications have different complication rates with regard to HIT. Thus the need to track HIT incidence after protocol changes are made is paramount.

In our study, we did not measure thrombocytopenia as an endpoint because acquired thrombocytopenia is too common and multifactorial to be a meaningful. Rather, we used the clinical suspicion for HIT as measured by both the number of times the BPA fired warnings of low platelets in the setting of recent heparin use and the number of times clinicians suspected HIT enough to order a HIT assay. We also used actual outcomes (clinically adjudicated cases of HIT and HITT). Our data shows substantial compliance among clinicians with the voluntary conversion to UFH with an immediate and sustained shift to UFH so that UFH was used in 86% of patients. Corresponding cost savings were achieved in heparin acquisition. Unlike some prior reports, there was a minimal burden of HIT as measured by the unchanged number of BPAs, monthly HIT assays and the unchanged clinical risk 4T scores among those patients in whom the test was ordered pre and post intervention. HIT rates were not statistically different after the order set conversion took effect.

Our results and study design are similar but not identical to that of Zhou et al, who found that a campaign to increase VTE prophylaxis resulted in 71% increase of UFH use over 5 years but no increase in number of HIT assays ordered or in the distribution of HIT assay results-both surrogate endpoints [5].But not all analyses of heparin order interventions show similar results. A recent study of a heparin avoidance program in a Canadian tertiary care hospital showed a reduction of 79% and 91% in adjudicated cases of HIT and HITT respectively [4].Moreover, hospital-related expenditures for HIT decreased by nearly $267,000 (Canadian dollars) per year though the additional acquisition costs of LMWH were not stated.A small retrospective heparin avoidance protocol among orthopedic surgery patients showed a reduction of HIT incidence from 5.2% with UFH to 0% with LMWH after universal substitution of LMWH for UFH [8].A recent systematic review identified only 3 prospective studies involving over 1398 postoperative surgical patients that measured HIT and HITT as outcomes [9].The review authors, in pooled analysis, found a lower incidence of HIT and HITT with LMWH postoperatively but downgraded the evidence to “low quality” due to methodologic issues and concerns over bias.A nested case-control study of adult medical patients found that HIT was 6 times more common with UFH than with LMWH and the cost of admissions associated with HIT was 3.5 times higher than for those without HIT, though this increase in costs are not necessarily due to the HIT diagnosis itself but may be markers of patients with more severe illness [10].The duration of heparin therapy was not stated.

There are several potential reasons that our data differs from some of the previous reports described above. We used a strict definition of HIT, requiring the serotonin release assay to be positive in the appropriate clinical setting and did not rely solely upon antibody tests to make the diagnosis, a less rigorous standard found in some studies. Furthermore, our results may differ from previously reports because of differences in patient risk and duration of therapy. Our institution does not perform cardiac surgery and the very large orthopedic surgery programs do not generally use heparin. Another potentially important difference in our study from prior studies is that many of the patients treated at this institution did not receive heparin long enough to be considered at risk; only a quarter were treated for longer than 5 days, generally considered a minumum [11].This is less than half of the duration of the patients in the studies included in the meta-analysis of HIT incidence [2].

 

We do not contend that UFH is as safe as LMWH with regard to HIT for all populatons, but rather that the increased risk is not manifest in all patient populations and settings and so the increased cost may not be justified in low-risk patients. Indeed while variability in HIT risk among patients is well documented [3,12], the guidelines for prophylaxis do not generally take this into account when recommending particular VTE prophylaxis strategies.Clinical practice guidelines do recommend different degrees of monitoring the platelet count based on risk of HIT however.

Our study had limitations, chief of which is the retrospective nature of the analysis; however, the methodology we used was similar to those of previous publications [4,5,8].We may have missed some cases of HIT if a clinician did not order the assay in all appropriate patients but there is no reason to think that likelihood was any different pre- and post-intervention. In addition, though we reviewed every case of hospital-acquired thrombosis, it is possible that the clinical reviewers may have missed cases of HITT, especially if the thrombosis occurred before a substantial drop in the platelet count, which is rare but possible. Here too the chance of missing actual cases did not change between the pre-and post-intervention. Our study examined prophylaxis with heparin use and not therapeutic uses. Finally, while noting the acquisition cost reduction achieved with conversion to UFH, we were not able to calculate any excess expense attributed to the rare case of HIT and HITT that occurred. We believe our results are generalizable to hospitals with similar patient profiles.

The idea that patients with different risk factors might do well with different prophylaxis strategies needs to be better appreciated. Such information could be used as a guide to more individualized prophylaxis strategy aided by clinical decision support embedded within the EMR. In this way the benefit of LMWH in avoiding HIT could be reserved for those patients at greatest risk of HIT while simultaneously allowing hospitals not to overspend for prophylaxis in patients who will not benefit from LMWH. Such a strategy would need to be tested prospectively before widespread adoption.

As a result of our internal analysis we have altered our EMR-based best practice alert to conform to the 2013 American Society of Hematology guidelines [15],which is more informative than our original BPA. Specifically, the old guideline only warned if the platelet count was < 100,000/mm³ in association with heparin. The revision notified if there is a > 30% fall regardless of the absolute count and informed prescribers of the 4T score to encourage more optimum use of the HIT assay, avoiding its use for low risk scores and encouraging its use for moderate to high risk scores. We are also strengthening the emphasis that moderate to high risk 4T patients receive alternative anticoagulation until results of the HIT assay are available as we found this not to be a be a universal practice. We recommend similar self-inspection to other institutions.

Corresponding author: Barry R. Meisenberg, MD, Anne Arundel Medical Center, 2001 Medical Parkway, Annapolis, MD 21401, [email protected].

Financial disclosures: None.

Author contributions: conception and design, JR, BRM; analysis and interpretation of data, KW, JR, BRM; drafting of article, JR, BRM; critical revision of the article, KW, JR, BRM; statistical expertise, KW, JR; administrative or technical support, JR; collection and assembly of data, KW, JR.

From the Anne Arundel Health System Research Institute, Annapolis, MD.

 

Abstract

• Objective: To measure clinical outcomes associated with heparin-induced thrombocytopenia (HIT) and acquisition costs of heparin after implementing a new order set promoting unfractionated heparin (UFH) use instead of low-molecular-weight heparin (LMWH) for venous thromboembolism (VTE) prophylaxis.
• Methods: This was single-center, retrospective, pre-post intervention analysis utilizing pharmacy, laboratory, and clinical data sources. Subjects were patients receiving VTE thromboprophyalxis with heparin at an acute care hospital. Usage rates for UFH and LMWH, acquisition costs for heparins, number of HIT assays, best practice advisories for HIT, and confirmed cases of HIT and HIT with thrombosis were assessed.
• Results: After order set intervention, UFH use increased from 43% of all prophylaxis orders to 86%. Net annual savings in acquisition costs for VTE prophylaxis was $131,000. After the intervention, HIT best practice advisories and number of monthly HIT assays fell 35% and 15%, respectively. In the 9-month pre-intervention period, HIT and HITT occurred in zero of 6717 patients receiving VTE prophylaxis. In the 25 months of post-intervention follow-up, HIT occurred in 3 of 44,240 patients (P = 0.86) receiving VTE prophylaxis, 2 of whom had HITT, all after receiving UFH. The median duration of UFH and LMWH use was 3.0 and 3.5 days, respectively.
• Conclusion: UFH use in hospitals can be safely maintained or increased among patient subpopulations that are not at high risk for HIT. A more nuanced approach to prophylaxis, taking into account individual patient risk and expected duration of therapy, may provide desired cost savings without provoking HIT.

Key words: heparin; heparin-induced thrombocytopenia; venous thromboembolism prophylaxis; cost-effectiveness.

 

Heparin-induced thrombocytopenia (HIT) and its more severe clinical complication, HIT with thrombosis (HITT), complicate the use of heparin products for venous thromboembolic (VTE) prophylaxis. The clinical characteristics and time course of thrombocytopenia in relation to heparin are well characterized (typically 30%–50% drop in platelet count 5–10 days after exposure), if not absolute. Risk calculation tools help to judge the clinical probability and guide ordering of appropriate confirmatory tests [1]. The incidence of HIT is higher with unfractionated heparin (UFH) than with low-molecular-weight heparin (LMWH). A meta-analysis of 5 randomized or prospective nonrandomized trials indicated a risk of 2.6% (95% CI, 1.5%–3.8%) for UFH and 0.2% (95% CI, 0.1%–0.4%) for LMWH [2], though the analyzed studies were heavily weighted by studies of orthopedic surgery patients, a high-risk group. However, not all patients are at equal risk for HIT, suggesting that LMWH may not be necessary for all patients [3]. Unfortunately, LMWH is considerably more expensive for hospitals to purchase than UFH, raising costs for a prophylactic treatment that is widely utilized. However, the higher incidence of HIT and HITT associated with UFH can erode any cost savings because of the additional cost of diagnosing HIT and need for temporary or long-term treatment with even more expensive alternative anticoagulants. Indeed, a recent retrospective study suggested that the excess costs of evaluating and treating HIT were approximately $267,000 per year in Canadian dollars [4].But contrary data has also been reported. A retrospective study of the consequences of increased prophylactic UFH use found no increase in ordered HIT assays or in the results of HIT testing or of inferred positive cases despite a growth of 71% in the number of patients receiving UFH prophylaxis [5].

In 2013, the pharmacy and therapeutics committee made a decision to encourage the use of UFH over LMWH for VTE prophylaxis by making changes to order sets to favor UFH over LMWH (enoxaparin). Given the uncertainty about excess risk of HIT, a monitoring work group was created to assess for any increase of either HIT or HITT that might follow, including any patient readmitted with thrombosis within 30 days of a discharge. In this paper, we report the impact of a hospital-wide conversion to UFH for VTE prophylaxis on the incidence of VTE, HIT, and HITT and acquisition costs of UFH and LMWH and use of alternative prophylactic anticoagulant medications.

Methods

Setting

Anne Arundel Medical Center is a 383-bed acute care hospital with about 30,000 adult admissions and 10,000 inpatient surgeries annually. The average length of stay is approximately 3.6 days with a patient median age of 59 years. Caucasians comprise 75.3% of the admitted populations and African Americans 21.4%. Most patients are on Medicare (59%), while 29.5% have private insurance, 6.6% are on Medicaid, and 4.7% self-pay. The 9 most common medical principal diagnoses are sepsis, heart failure, chronic obstructive pulmonary disease, pneumonia, myocardial infarction, ischemic stroke, urinary tract infection, cardiac arrhythmia, and other infection. The 6 most common procedures include newborn delivery (with and without caesarean section), joint replacement surgery, bariatric procedures, cardiac catheterizations, other abdominal surgeries, and thoracotomy. The predominant medical care model is internal medicine and physician assistant acute care hospitalists attending both medicine and surgical patients. Obstetrical hospitalists care for admitted obstetric patients. Patients admitted to the intensive care units had only critical care trained physician specialists as attending physicians. No trainees cared for the patients described in this study.

P&T Committee

The P&T committee is a multidisciplinary group of health care professionals selected for appointment by the chairs of the committee (chair of medicine and director of pharmacy) and approved by the president of the medical staff. The committee has oversight responsibility for all medication policies, order sets involving medications, as well as the monitoring of clinical outcomes as they regard medications.

Electronic Medical Record and Best Practice Advisory

Throughout this study period both pre-and post-intervention, the EMR in use was Epic (Verona WI), used for all ordering and lab results. A best practice advisory was in place in the EMR that alerted providers to all cases of thrombocytopenia < 100,000/mm³ when there was concurrent order for any heparin. The best practice advisory highlighted the thrombocytopenia, advised the providers to consider HIT as a diagnosis and to order confirmation tests if clinically appropriate, providing a direct link to the HIT assay order screen. The best practice advisory did not access information from prior admissions where heparin might have been used nor determine the percentage drop from the baseline platelet count.

HIT Case Definition and Assays

The 2 laboratory tests for HIT on which this study is based are the heparin-induced platelet antibody test (also known as anti-PF4) and the serotonin release assay. The heparin-induced platelet antibody test is an enzyme-linked immunosorbent assay (ELISA) that detects IgG, IgM, and IgA antibodies against the platelet factor 4 (PF4/heparin complex). This test was reported as positive if the optical density was 0.4 or higher and generated an automatic request for a serotonin release assay (SRA), which is a functional assay that measures heparin-dependent platelet activation. The decision to order the SRA was therefore a “reflex” test and not made with any knowledge of clinical characteristics of the case. The HIT assays were performed by a reference lab, Quest Diagnostics, in the Chantilly, VA facility. HIT was said to be present when both a characteristic pattern of thrombocytopenia occurring after heparin use was seen [1]and when the confirmatory SRA was positive at a level of > 20% release.

 

Order Set Modifications

After the P&T committee decision to emphasize UFH for VTE prophylaxis in October 2013, the relevant electronic order sets were altered to highlight the fact that UFH was the first choice for VTE prophylaxis. The order sets still allowed LMWH (enoxaparin) or alternative anticoagulants at the prescribers’ discretion but indicated they were a second choice. Doses of UFH and LMWH in the order sets were standard based upon weight and estimates of creatinine clearance and, in the case of dosing frequency for UFH, based upon the risk of VTE. Order sets for the therapeutic treatment of VTE were not changed.

Data Collection and Analysis

The clinical research committee, the local oversight board for research and performance improvement analyses, reviewed this project and determined that it qualified as a performance improvement analysis based upon the standards of the U.S. Office of Human Research Protections. Some data were extracted from patient medical records and stored in a customized and password-protected database. Access to the database was limited to members of the analysis team and stripped of all patient identifiers under the HIPAA privacy rule standard for de-identification from 45 CFR 164.514(b) immediately following the collection of all data elements from the medical record.

An internal pharmacy database was used to determine the volume and actual acquisition cost of prophylactic anticoagulant doses administered during both pre- and post-intervention time periods. To determine if clinical suspicion for HIT increased after the intervention, a definitive listing of all ordered HIT assays was obtained from laboratory billing records for the 9 months (January 2013–September 2013) before the conversion and for 25 months after the intervention (beginning in November 2013 so as not to include the conversion month). To determine if the HIT assays were associated with a higher risk score, we identified all cases in which the HIT assay was ordered and retroactively measured the probability score known as the 4T score [1].Simultaneously, separate clinical work groups reviewed all cases of hospital-acquired thrombosis, whatever their cause, including patients readmitted with thrombosis up to 30 days after discharge and episodes of bleeding due to anti-coagulant use. A chi square analysis of the incidence of HIT pre- and post-intervention was performed.

Results

Heparin Use and Acquisition Costs

Annual hospital admissions and patient-days both increased 3% during the study time period to 29,975 annual adult admissions and 96,426 hospital days. However, the number of patients receiving any prophylactic anticoagulant increased 15% over the time period as a result of a simultaneous anti-VTE education campaign. After the modification in order sets there was a rapid increase in orders for prophylactic UFH and a corresponding decrease in the use of LMWH (Figure 1). A few patients received both at different times during their hospitalization. Prophylactic UFH orders increased from 43% of all prophylactic use pre-intervention to 86% by the second year of the intervention. There was a corresponding reduction in use of LMWH from 53% of all prophylaxis orders to 11% after the intervention. Other anticoagulants, fondaparinux and bivalrudin, together composed 5% and 4% of all prophylactic heparin in the pre- and post-intervention time periods, respectively. Pharmacy acquisition costs averaged $3.50 for daily doses of heparin and $24 for LMWH. With the shift in ordering pattern, acquisition costs of prophylactic LMWH fell by $165,000 annually with a concurrent offsetting increase in acquisition costs of UFH of $34,000 for a net savings of $131,000. The median duration of use was 3.5 days and 3.0 days for UFH and LMWH, respectively. Only 25% of treated patients had heparin exposure of > 5 days. Bleeding complications from any prophylaxis regimen were < 0.5% in both pre- and post-intervention time periods.

HIT Assays and Incidence of HIT and HITT

HIT assays results returned to the chart within a median of 4 days for anti-platelet factor 4 and 5 days for SRA. Data on the number of HIT assays ordered was missing for 1 month in 2013 and 1 month in 2014 due to missing laboratory billing data. On average, the best practice advisory for HIT fired 35% less frequently after the intervention (Figure 2). The number of ordered serum ELISA HIT assays decreased slightly after the intervention from a mean of 18.2 per month to 15.4 per month despite the increase in UFH use. The distribution of the retrospectively measured 4T scores for patients with HIT testing is shown in 

the Table. There were no differences in the retrospectively calculated scores between the pre- and post-intervention time periods. Most patients had a moderate risk 4T score prior to having HIT assay ordered.

In the 9 months pre-intervention, HIT and HITT occurred in zero of 6717 patients receiving at least 1 dose of VTE prophylaxis. In the 25 months of post-intervention follow-up, 44,240 patients received prophylaxis with either heparin. HIT (clinical suspicion with positive antibody and confirmatory SRA) occurred in 3 patients, 2 of whom had HITT, all after UFH. This incidence was not statistically significant using chi square analysis (P = 0.86).

 

Discussion

Because the efficacy of UFH and LMWH for VTE prophylaxis are equivalent [6],choosing between them involves many factors including patient-level risk factors such as renal function, risk of bleeding, as well as other considerations such as nursing time, patient preference, risk of HIT, and acquisition cost. Indeed, the most recent version of the American College of Chest Physicians guidelines for prophylaxis against VTE note that both drugs are recommended with an evidence grade of IB [7].Cost is among the considerations considered appropriate in choosing among agents. The difference in acquisition costs of > $20 per patient per day can have a major financial impact on hospital’s pharmacy budget and may be decisive. But a focus only on acquisition cost is short sighted as the 2 medications have different complication rates with regard to HIT. Thus the need to track HIT incidence after protocol changes are made is paramount.

In our study, we did not measure thrombocytopenia as an endpoint because acquired thrombocytopenia is too common and multifactorial to be a meaningful. Rather, we used the clinical suspicion for HIT as measured by both the number of times the BPA fired warnings of low platelets in the setting of recent heparin use and the number of times clinicians suspected HIT enough to order a HIT assay. We also used actual outcomes (clinically adjudicated cases of HIT and HITT). Our data shows substantial compliance among clinicians with the voluntary conversion to UFH with an immediate and sustained shift to UFH so that UFH was used in 86% of patients. Corresponding cost savings were achieved in heparin acquisition. Unlike some prior reports, there was a minimal burden of HIT as measured by the unchanged number of BPAs, monthly HIT assays and the unchanged clinical risk 4T scores among those patients in whom the test was ordered pre and post intervention. HIT rates were not statistically different after the order set conversion took effect.

Our results and study design are similar but not identical to that of Zhou et al, who found that a campaign to increase VTE prophylaxis resulted in 71% increase of UFH use over 5 years but no increase in number of HIT assays ordered or in the distribution of HIT assay results-both surrogate endpoints [5].But not all analyses of heparin order interventions show similar results. A recent study of a heparin avoidance program in a Canadian tertiary care hospital showed a reduction of 79% and 91% in adjudicated cases of HIT and HITT respectively [4].Moreover, hospital-related expenditures for HIT decreased by nearly $267,000 (Canadian dollars) per year though the additional acquisition costs of LMWH were not stated.A small retrospective heparin avoidance protocol among orthopedic surgery patients showed a reduction of HIT incidence from 5.2% with UFH to 0% with LMWH after universal substitution of LMWH for UFH [8].A recent systematic review identified only 3 prospective studies involving over 1398 postoperative surgical patients that measured HIT and HITT as outcomes [9].The review authors, in pooled analysis, found a lower incidence of HIT and HITT with LMWH postoperatively but downgraded the evidence to “low quality” due to methodologic issues and concerns over bias.A nested case-control study of adult medical patients found that HIT was 6 times more common with UFH than with LMWH and the cost of admissions associated with HIT was 3.5 times higher than for those without HIT, though this increase in costs are not necessarily due to the HIT diagnosis itself but may be markers of patients with more severe illness [10].The duration of heparin therapy was not stated.

There are several potential reasons that our data differs from some of the previous reports described above. We used a strict definition of HIT, requiring the serotonin release assay to be positive in the appropriate clinical setting and did not rely solely upon antibody tests to make the diagnosis, a less rigorous standard found in some studies. Furthermore, our results may differ from previously reports because of differences in patient risk and duration of therapy. Our institution does not perform cardiac surgery and the very large orthopedic surgery programs do not generally use heparin. Another potentially important difference in our study from prior studies is that many of the patients treated at this institution did not receive heparin long enough to be considered at risk; only a quarter were treated for longer than 5 days, generally considered a minumum [11].This is less than half of the duration of the patients in the studies included in the meta-analysis of HIT incidence [2].

 

We do not contend that UFH is as safe as LMWH with regard to HIT for all populatons, but rather that the increased risk is not manifest in all patient populations and settings and so the increased cost may not be justified in low-risk patients. Indeed while variability in HIT risk among patients is well documented [3,12], the guidelines for prophylaxis do not generally take this into account when recommending particular VTE prophylaxis strategies.Clinical practice guidelines do recommend different degrees of monitoring the platelet count based on risk of HIT however.

Our study had limitations, chief of which is the retrospective nature of the analysis; however, the methodology we used was similar to those of previous publications [4,5,8].We may have missed some cases of HIT if a clinician did not order the assay in all appropriate patients but there is no reason to think that likelihood was any different pre- and post-intervention. In addition, though we reviewed every case of hospital-acquired thrombosis, it is possible that the clinical reviewers may have missed cases of HITT, especially if the thrombosis occurred before a substantial drop in the platelet count, which is rare but possible. Here too the chance of missing actual cases did not change between the pre-and post-intervention. Our study examined prophylaxis with heparin use and not therapeutic uses. Finally, while noting the acquisition cost reduction achieved with conversion to UFH, we were not able to calculate any excess expense attributed to the rare case of HIT and HITT that occurred. We believe our results are generalizable to hospitals with similar patient profiles.

The idea that patients with different risk factors might do well with different prophylaxis strategies needs to be better appreciated. Such information could be used as a guide to more individualized prophylaxis strategy aided by clinical decision support embedded within the EMR. In this way the benefit of LMWH in avoiding HIT could be reserved for those patients at greatest risk of HIT while simultaneously allowing hospitals not to overspend for prophylaxis in patients who will not benefit from LMWH. Such a strategy would need to be tested prospectively before widespread adoption.

As a result of our internal analysis we have altered our EMR-based best practice alert to conform to the 2013 American Society of Hematology guidelines [15],which is more informative than our original BPA. Specifically, the old guideline only warned if the platelet count was < 100,000/mm³ in association with heparin. The revision notified if there is a > 30% fall regardless of the absolute count and informed prescribers of the 4T score to encourage more optimum use of the HIT assay, avoiding its use for low risk scores and encouraging its use for moderate to high risk scores. We are also strengthening the emphasis that moderate to high risk 4T patients receive alternative anticoagulation until results of the HIT assay are available as we found this not to be a be a universal practice. We recommend similar self-inspection to other institutions.

Corresponding author: Barry R. Meisenberg, MD, Anne Arundel Medical Center, 2001 Medical Parkway, Annapolis, MD 21401, [email protected].

Financial disclosures: None.

Author contributions: conception and design, JR, BRM; analysis and interpretation of data, KW, JR, BRM; drafting of article, JR, BRM; critical revision of the article, KW, JR, BRM; statistical expertise, KW, JR; administrative or technical support, JR; collection and assembly of data, KW, JR.

From the Anne Arundel Health System Research Institute, Annapolis, MD.

 

Abstract

• Objective: To measure clinical outcomes associated with heparin-induced thrombocytopenia (HIT) and acquisition costs of heparin after implementing a new order set promoting unfractionated heparin (UFH) use instead of low-molecular-weight heparin (LMWH) for venous thromboembolism (VTE) prophylaxis.
• Methods: This was single-center, retrospective, pre-post intervention analysis utilizing pharmacy, laboratory, and clinical data sources. Subjects were patients receiving VTE thromboprophyalxis with heparin at an acute care hospital. Usage rates for UFH and LMWH, acquisition costs for heparins, number of HIT assays, best practice advisories for HIT, and confirmed cases of HIT and HIT with thrombosis were assessed.
• Results: After order set intervention, UFH use increased from 43% of all prophylaxis orders to 86%. Net annual savings in acquisition costs for VTE prophylaxis was $131,000. After the intervention, HIT best practice advisories and number of monthly HIT assays fell 35% and 15%, respectively. In the 9-month pre-intervention period, HIT and HITT occurred in zero of 6717 patients receiving VTE prophylaxis. In the 25 months of post-intervention follow-up, HIT occurred in 3 of 44,240 patients (P = 0.86) receiving VTE prophylaxis, 2 of whom had HITT, all after receiving UFH. The median duration of UFH and LMWH use was 3.0 and 3.5 days, respectively.
• Conclusion: UFH use in hospitals can be safely maintained or increased among patient subpopulations that are not at high risk for HIT. A more nuanced approach to prophylaxis, taking into account individual patient risk and expected duration of therapy, may provide desired cost savings without provoking HIT.

Key words: heparin; heparin-induced thrombocytopenia; venous thromboembolism prophylaxis; cost-effectiveness.

 

Heparin-induced thrombocytopenia (HIT) and its more severe clinical complication, HIT with thrombosis (HITT), complicate the use of heparin products for venous thromboembolic (VTE) prophylaxis. The clinical characteristics and time course of thrombocytopenia in relation to heparin are well characterized (typically 30%–50% drop in platelet count 5–10 days after exposure), if not absolute. Risk calculation tools help to judge the clinical probability and guide ordering of appropriate confirmatory tests [1]. The incidence of HIT is higher with unfractionated heparin (UFH) than with low-molecular-weight heparin (LMWH). A meta-analysis of 5 randomized or prospective nonrandomized trials indicated a risk of 2.6% (95% CI, 1.5%–3.8%) for UFH and 0.2% (95% CI, 0.1%–0.4%) for LMWH [2], though the analyzed studies were heavily weighted by studies of orthopedic surgery patients, a high-risk group. However, not all patients are at equal risk for HIT, suggesting that LMWH may not be necessary for all patients [3]. Unfortunately, LMWH is considerably more expensive for hospitals to purchase than UFH, raising costs for a prophylactic treatment that is widely utilized. However, the higher incidence of HIT and HITT associated with UFH can erode any cost savings because of the additional cost of diagnosing HIT and need for temporary or long-term treatment with even more expensive alternative anticoagulants. Indeed, a recent retrospective study suggested that the excess costs of evaluating and treating HIT were approximately $267,000 per year in Canadian dollars [4].But contrary data has also been reported. A retrospective study of the consequences of increased prophylactic UFH use found no increase in ordered HIT assays or in the results of HIT testing or of inferred positive cases despite a growth of 71% in the number of patients receiving UFH prophylaxis [5].

In 2013, the pharmacy and therapeutics committee made a decision to encourage the use of UFH over LMWH for VTE prophylaxis by making changes to order sets to favor UFH over LMWH (enoxaparin). Given the uncertainty about excess risk of HIT, a monitoring work group was created to assess for any increase of either HIT or HITT that might follow, including any patient readmitted with thrombosis within 30 days of a discharge. In this paper, we report the impact of a hospital-wide conversion to UFH for VTE prophylaxis on the incidence of VTE, HIT, and HITT and acquisition costs of UFH and LMWH and use of alternative prophylactic anticoagulant medications.

Methods

Setting

Anne Arundel Medical Center is a 383-bed acute care hospital with about 30,000 adult admissions and 10,000 inpatient surgeries annually. The average length of stay is approximately 3.6 days with a patient median age of 59 years. Caucasians comprise 75.3% of the admitted populations and African Americans 21.4%. Most patients are on Medicare (59%), while 29.5% have private insurance, 6.6% are on Medicaid, and 4.7% self-pay. The 9 most common medical principal diagnoses are sepsis, heart failure, chronic obstructive pulmonary disease, pneumonia, myocardial infarction, ischemic stroke, urinary tract infection, cardiac arrhythmia, and other infection. The 6 most common procedures include newborn delivery (with and without caesarean section), joint replacement surgery, bariatric procedures, cardiac catheterizations, other abdominal surgeries, and thoracotomy. The predominant medical care model is internal medicine and physician assistant acute care hospitalists attending both medicine and surgical patients. Obstetrical hospitalists care for admitted obstetric patients. Patients admitted to the intensive care units had only critical care trained physician specialists as attending physicians. No trainees cared for the patients described in this study.

P&T Committee

The P&T committee is a multidisciplinary group of health care professionals selected for appointment by the chairs of the committee (chair of medicine and director of pharmacy) and approved by the president of the medical staff. The committee has oversight responsibility for all medication policies, order sets involving medications, as well as the monitoring of clinical outcomes as they regard medications.

Electronic Medical Record and Best Practice Advisory

Throughout this study period both pre-and post-intervention, the EMR in use was Epic (Verona WI), used for all ordering and lab results. A best practice advisory was in place in the EMR that alerted providers to all cases of thrombocytopenia < 100,000/mm³ when there was concurrent order for any heparin. The best practice advisory highlighted the thrombocytopenia, advised the providers to consider HIT as a diagnosis and to order confirmation tests if clinically appropriate, providing a direct link to the HIT assay order screen. The best practice advisory did not access information from prior admissions where heparin might have been used nor determine the percentage drop from the baseline platelet count.

HIT Case Definition and Assays

The 2 laboratory tests for HIT on which this study is based are the heparin-induced platelet antibody test (also known as anti-PF4) and the serotonin release assay. The heparin-induced platelet antibody test is an enzyme-linked immunosorbent assay (ELISA) that detects IgG, IgM, and IgA antibodies against the platelet factor 4 (PF4/heparin complex). This test was reported as positive if the optical density was 0.4 or higher and generated an automatic request for a serotonin release assay (SRA), which is a functional assay that measures heparin-dependent platelet activation. The decision to order the SRA was therefore a “reflex” test and not made with any knowledge of clinical characteristics of the case. The HIT assays were performed by a reference lab, Quest Diagnostics, in the Chantilly, VA facility. HIT was said to be present when both a characteristic pattern of thrombocytopenia occurring after heparin use was seen [1]and when the confirmatory SRA was positive at a level of > 20% release.

 

Order Set Modifications

After the P&T committee decision to emphasize UFH for VTE prophylaxis in October 2013, the relevant electronic order sets were altered to highlight the fact that UFH was the first choice for VTE prophylaxis. The order sets still allowed LMWH (enoxaparin) or alternative anticoagulants at the prescribers’ discretion but indicated they were a second choice. Doses of UFH and LMWH in the order sets were standard based upon weight and estimates of creatinine clearance and, in the case of dosing frequency for UFH, based upon the risk of VTE. Order sets for the therapeutic treatment of VTE were not changed.

Data Collection and Analysis

The clinical research committee, the local oversight board for research and performance improvement analyses, reviewed this project and determined that it qualified as a performance improvement analysis based upon the standards of the U.S. Office of Human Research Protections. Some data were extracted from patient medical records and stored in a customized and password-protected database. Access to the database was limited to members of the analysis team and stripped of all patient identifiers under the HIPAA privacy rule standard for de-identification from 45 CFR 164.514(b) immediately following the collection of all data elements from the medical record.

An internal pharmacy database was used to determine the volume and actual acquisition cost of prophylactic anticoagulant doses administered during both pre- and post-intervention time periods. To determine if clinical suspicion for HIT increased after the intervention, a definitive listing of all ordered HIT assays was obtained from laboratory billing records for the 9 months (January 2013–September 2013) before the conversion and for 25 months after the intervention (beginning in November 2013 so as not to include the conversion month). To determine if the HIT assays were associated with a higher risk score, we identified all cases in which the HIT assay was ordered and retroactively measured the probability score known as the 4T score [1].Simultaneously, separate clinical work groups reviewed all cases of hospital-acquired thrombosis, whatever their cause, including patients readmitted with thrombosis up to 30 days after discharge and episodes of bleeding due to anti-coagulant use. A chi square analysis of the incidence of HIT pre- and post-intervention was performed.

Results

Heparin Use and Acquisition Costs

Annual hospital admissions and patient-days both increased 3% during the study time period to 29,975 annual adult admissions and 96,426 hospital days. However, the number of patients receiving any prophylactic anticoagulant increased 15% over the time period as a result of a simultaneous anti-VTE education campaign. After the modification in order sets there was a rapid increase in orders for prophylactic UFH and a corresponding decrease in the use of LMWH (Figure 1). A few patients received both at different times during their hospitalization. Prophylactic UFH orders increased from 43% of all prophylactic use pre-intervention to 86% by the second year of the intervention. There was a corresponding reduction in use of LMWH from 53% of all prophylaxis orders to 11% after the intervention. Other anticoagulants, fondaparinux and bivalrudin, together composed 5% and 4% of all prophylactic heparin in the pre- and post-intervention time periods, respectively. Pharmacy acquisition costs averaged $3.50 for daily doses of heparin and $24 for LMWH. With the shift in ordering pattern, acquisition costs of prophylactic LMWH fell by $165,000 annually with a concurrent offsetting increase in acquisition costs of UFH of $34,000 for a net savings of $131,000. The median duration of use was 3.5 days and 3.0 days for UFH and LMWH, respectively. Only 25% of treated patients had heparin exposure of > 5 days. Bleeding complications from any prophylaxis regimen were < 0.5% in both pre- and post-intervention time periods.

HIT Assays and Incidence of HIT and HITT

HIT assays results returned to the chart within a median of 4 days for anti-platelet factor 4 and 5 days for SRA. Data on the number of HIT assays ordered was missing for 1 month in 2013 and 1 month in 2014 due to missing laboratory billing data. On average, the best practice advisory for HIT fired 35% less frequently after the intervention (Figure 2). The number of ordered serum ELISA HIT assays decreased slightly after the intervention from a mean of 18.2 per month to 15.4 per month despite the increase in UFH use. The distribution of the retrospectively measured 4T scores for patients with HIT testing is shown in 

the Table. There were no differences in the retrospectively calculated scores between the pre- and post-intervention time periods. Most patients had a moderate risk 4T score prior to having HIT assay ordered.

In the 9 months pre-intervention, HIT and HITT occurred in zero of 6717 patients receiving at least 1 dose of VTE prophylaxis. In the 25 months of post-intervention follow-up, 44,240 patients received prophylaxis with either heparin. HIT (clinical suspicion with positive antibody and confirmatory SRA) occurred in 3 patients, 2 of whom had HITT, all after UFH. This incidence was not statistically significant using chi square analysis (P = 0.86).

 

Discussion

Because the efficacy of UFH and LMWH for VTE prophylaxis are equivalent [6],choosing between them involves many factors including patient-level risk factors such as renal function, risk of bleeding, as well as other considerations such as nursing time, patient preference, risk of HIT, and acquisition cost. Indeed, the most recent version of the American College of Chest Physicians guidelines for prophylaxis against VTE note that both drugs are recommended with an evidence grade of IB [7].Cost is among the considerations considered appropriate in choosing among agents. The difference in acquisition costs of > $20 per patient per day can have a major financial impact on hospital’s pharmacy budget and may be decisive. But a focus only on acquisition cost is short sighted as the 2 medications have different complication rates with regard to HIT. Thus the need to track HIT incidence after protocol changes are made is paramount.

In our study, we did not measure thrombocytopenia as an endpoint because acquired thrombocytopenia is too common and multifactorial to be a meaningful. Rather, we used the clinical suspicion for HIT as measured by both the number of times the BPA fired warnings of low platelets in the setting of recent heparin use and the number of times clinicians suspected HIT enough to order a HIT assay. We also used actual outcomes (clinically adjudicated cases of HIT and HITT). Our data shows substantial compliance among clinicians with the voluntary conversion to UFH with an immediate and sustained shift to UFH so that UFH was used in 86% of patients. Corresponding cost savings were achieved in heparin acquisition. Unlike some prior reports, there was a minimal burden of HIT as measured by the unchanged number of BPAs, monthly HIT assays and the unchanged clinical risk 4T scores among those patients in whom the test was ordered pre and post intervention. HIT rates were not statistically different after the order set conversion took effect.

Our results and study design are similar but not identical to that of Zhou et al, who found that a campaign to increase VTE prophylaxis resulted in 71% increase of UFH use over 5 years but no increase in number of HIT assays ordered or in the distribution of HIT assay results-both surrogate endpoints [5].But not all analyses of heparin order interventions show similar results. A recent study of a heparin avoidance program in a Canadian tertiary care hospital showed a reduction of 79% and 91% in adjudicated cases of HIT and HITT respectively [4].Moreover, hospital-related expenditures for HIT decreased by nearly $267,000 (Canadian dollars) per year though the additional acquisition costs of LMWH were not stated.A small retrospective heparin avoidance protocol among orthopedic surgery patients showed a reduction of HIT incidence from 5.2% with UFH to 0% with LMWH after universal substitution of LMWH for UFH [8].A recent systematic review identified only 3 prospective studies involving over 1398 postoperative surgical patients that measured HIT and HITT as outcomes [9].The review authors, in pooled analysis, found a lower incidence of HIT and HITT with LMWH postoperatively but downgraded the evidence to “low quality” due to methodologic issues and concerns over bias.A nested case-control study of adult medical patients found that HIT was 6 times more common with UFH than with LMWH and the cost of admissions associated with HIT was 3.5 times higher than for those without HIT, though this increase in costs are not necessarily due to the HIT diagnosis itself but may be markers of patients with more severe illness [10].The duration of heparin therapy was not stated.

There are several potential reasons that our data differs from some of the previous reports described above. We used a strict definition of HIT, requiring the serotonin release assay to be positive in the appropriate clinical setting and did not rely solely upon antibody tests to make the diagnosis, a less rigorous standard found in some studies. Furthermore, our results may differ from previously reports because of differences in patient risk and duration of therapy. Our institution does not perform cardiac surgery and the very large orthopedic surgery programs do not generally use heparin. Another potentially important difference in our study from prior studies is that many of the patients treated at this institution did not receive heparin long enough to be considered at risk; only a quarter were treated for longer than 5 days, generally considered a minumum [11].This is less than half of the duration of the patients in the studies included in the meta-analysis of HIT incidence [2].

 

We do not contend that UFH is as safe as LMWH with regard to HIT for all populatons, but rather that the increased risk is not manifest in all patient populations and settings and so the increased cost may not be justified in low-risk patients. Indeed while variability in HIT risk among patients is well documented [3,12], the guidelines for prophylaxis do not generally take this into account when recommending particular VTE prophylaxis strategies.Clinical practice guidelines do recommend different degrees of monitoring the platelet count based on risk of HIT however.

Our study had limitations, chief of which is the retrospective nature of the analysis; however, the methodology we used was similar to those of previous publications [4,5,8].We may have missed some cases of HIT if a clinician did not order the assay in all appropriate patients but there is no reason to think that likelihood was any different pre- and post-intervention. In addition, though we reviewed every case of hospital-acquired thrombosis, it is possible that the clinical reviewers may have missed cases of HITT, especially if the thrombosis occurred before a substantial drop in the platelet count, which is rare but possible. Here too the chance of missing actual cases did not change between the pre-and post-intervention. Our study examined prophylaxis with heparin use and not therapeutic uses. Finally, while noting the acquisition cost reduction achieved with conversion to UFH, we were not able to calculate any excess expense attributed to the rare case of HIT and HITT that occurred. We believe our results are generalizable to hospitals with similar patient profiles.

The idea that patients with different risk factors might do well with different prophylaxis strategies needs to be better appreciated. Such information could be used as a guide to more individualized prophylaxis strategy aided by clinical decision support embedded within the EMR. In this way the benefit of LMWH in avoiding HIT could be reserved for those patients at greatest risk of HIT while simultaneously allowing hospitals not to overspend for prophylaxis in patients who will not benefit from LMWH. Such a strategy would need to be tested prospectively before widespread adoption.

As a result of our internal analysis we have altered our EMR-based best practice alert to conform to the 2013 American Society of Hematology guidelines [15],which is more informative than our original BPA. Specifically, the old guideline only warned if the platelet count was < 100,000/mm³ in association with heparin. The revision notified if there is a > 30% fall regardless of the absolute count and informed prescribers of the 4T score to encourage more optimum use of the HIT assay, avoiding its use for low risk scores and encouraging its use for moderate to high risk scores. We are also strengthening the emphasis that moderate to high risk 4T patients receive alternative anticoagulation until results of the HIT assay are available as we found this not to be a be a universal practice. We recommend similar self-inspection to other institutions.

Corresponding author: Barry R. Meisenberg, MD, Anne Arundel Medical Center, 2001 Medical Parkway, Annapolis, MD 21401, [email protected].

Financial disclosures: None.

Author contributions: conception and design, JR, BRM; analysis and interpretation of data, KW, JR, BRM; drafting of article, JR, BRM; critical revision of the article, KW, JR, BRM; statistical expertise, KW, JR; administrative or technical support, JR; collection and assembly of data, KW, JR.

BACKGROUND

Physiologic monitors are intended to prevent cardiac and respiratory arrest by generating alarms to alert clinicians to signs of instability. To minimize the probability that monitors will miss signs of deterioration, alarm algorithms and default parameters are often set to maximize sensitivity while sacrificing specificity.¹ As a result, monitors generate large numbers of nonactionable alarms—alarms that are either invalid and do not accurately represent the physiologic status of the patient or are valid but do not warrant clinical intervention.² Prior research has demonstrated that the pediatric intensive care unit (PICU) is responsible for a higher proportion of alarms than pediatric wards³ and a large proportion of these alarms, 87% - 97%, are nonactionable.^4-8 In national surveys of healthcare staff, respondents report that high alarm rates interrupt patient care and can lead clinicians to disable alarms entirely.⁹ Recent research has supported this, demonstrating that nurses who are exposed to higher numbers of alarms have slower response times to alarms.^4,10 In an attempt to mitigate safety risks, the Joint Commission in 2012 issued recommendations for hospitals to (a) establish guidelines for tailoring alarm settings and limits for individual patients and (b) identify situations in which alarms are not clinically necessary.¹¹

In order to address these recommendations within our PICU, we sought to evaluate the impact of a focused physiologic monitor alarm reduction intervention integrated into safety huddles. Safety huddles are brief, structured discussions among physicians, nurses, and other staff aiming to identify safety concerns.¹² Huddles offer an appropriate forum for reviewing alarm data and identifying patients whose high alarm rates may necessitate safe tailoring of alarm limits. Pilot data demonstrating high alarm rates among low-acuity PICU patients led us to hypothesize that low-acuity, high-alarm PICU patients would be a safe and effective target for an alarm huddle-based intervention.

In this study, we aimed to measure the impact of a structured safety huddle review of low-acuity PICU patients with high rates of priority alarms who were randomized to intervention compared with other low-acuity, high-alarm, concurrent, and historical control patients in the PICU.

METHODS

Study Definitions

Priority alarm activation rate. We conceptualized priority alarms as any alarm for a clinical condition that requires a timely response to determine if intervention is necessary to save a patient’s life,⁴ yet little empirical data support its existence in the hospital. We operationally defined these alarms on the General Electric Solar physiologic monitoring devices as any potentially life-threatening events including lethal arrhythmias (asystole, ventricular tachycardia, and ventricular fibrillation) and alarms for vital signs (heart rate, respiratory rate, and oxygen saturation) outside of the set parameter limits. These alarms produced audible tones in the patient room and automatically sent text messages to the nurse’s phone and had the potential to contribute to alarm fatigue regardless of the nurse’s location.

High-alarm patients. High-alarm patients were those who had more than 40 priority alarms in the preceding 4 hours, representing the top 20% of alarm rates in the PICU according to prior quality improvement projects completed in our PICU.

Low-acuity patients. Prior to and during this study, patient acuity was determined using the OptiLink Patient Classification System (OptiLink Healthcare Management Systems, Inc.; Tigard, OR; www.optilinkhealthcare.com; see Appendix 1) for the PICU twice daily. Low-acuity patients comprised on average 16% of the PICU patients.

Setting and Subjects

This study was performed in the PICU at The Children’s Hospital of Philadelphia.

The PICU is made up of 3 separate wings: east, south, and west. Bed availability was the only factor determining patient placement on the east, south, or west wing; the physical bed location was not preferentially assigned based on diagnosis or disease severity. The east wing was the intervention unit where the huddles occurred.

The PICU is composed of 3 different geographical teams. Two of the teams are composed of 4 to 5 pediatric or emergency medicine residents, 1 fellow, and 1 attending covering the south and west wings. The third team, located on the east wing, is composed of 1 to 2 pediatric residents, 2 to 3 nurse practitioners, 1 fellow, and 1 attending. Bedside family-centered rounds are held at each patient room, with the bedside nurse participating by reading a nursing rounding script that includes vital signs, vascular access, continuous medications, and additional questions or concerns.

Control subjects were any monitored patients on any of the 3 wings of the PICU between April 1, 2015, and October 31, 2015. The control patients were in 2 categories: historical controls from April 1, 2015, to May 31, 2015, and concurrent controls from June 1, 2015, to October 31, 2015, who were located anywhere in the PICU. On each nonholiday weekday beginning June 1, 2015, we randomly selected up to 2 patients to receive the intervention. These were high-alarm, low-acuity patients on the east wing to be discussed in the daily morning huddle. If more than 2 high-alarm, low-acuity patients were eligible for intervention, they were randomly selected by using the RAND function in Microsoft Excel. The other low-acuity, high-alarm patients in the PICU were included as control patients. Patients were eligible for the study if they were present for the 4 hours prior to huddle and present past noon on the day of huddle. If patients met criteria as high-alarm, low-acuity patients on multiple days, they could be enrolled as intervention or control patients multiple times. Patients’ alarm rates were calculated by dividing the number of alarms by their length of stay to the minute. There was no adjustment made for patients enrolled more than once.

 

Human Subjects Protection

The Institutional Review Board of The Children’s Hospital of Philadelphia approved this study with a waiver of informed consent.

Alarm Capture

We used BedMasterEx (Excel Medical Electronics; Jupiter, FL, http://excel-medical.com/products/bedmaster-ex) software connected to the General Electric monitor network to measure alarm rates. The software captured, in near real time, every alarm that occurred on every monitor in the PICU. Alarm rates over the preceding 4 hours for all PICU patients were exported and summarized by alarm type and level as set by hospital policy (crisis, warning, advisory, and system warning). Crisis and warning alarms were included as they represented potential life-threatening events meeting the definition of priority alarms. Physicians used an order within the PICU admission order-set to order monitoring based on preset age parameters (see online Appendix 1 for default settings). Physician orders were required for nurses to change alarm parameters. Daily electrode changes to reduce false alarms were standard of care.

Primary Outcome

The primary outcome was the change in priority alarm activation rate (the number of priority alarms per day) from prehuddle period (24 hours before morning huddle) to posthuddle period (the 24 hours following morning huddle) for intervention cases as compared with controls.

Primary Intervention

The intervention consisted of integrating a short script to facilitate the discussion of the alarm data during existing safety huddle and rounding workflows. The discussion and subsequent workflow proceeded as follows: A member of the research team who was not involved in patient care brought an alarm data sheet for each randomly selected intervention patient on the east wing to each safety huddle. The huddles were attended by the outgoing night charge nurse, the day charge nurse, and all bedside nurses working on the east wing that day. The alarm data sheet provided to the charge nurse displayed data on the 1 to 2 alarm parameters (respiratory rate, heart rate, or pulse oximetry) that generated the highest number of alarms. The charge nurse listed the high-alarm patients by room number during huddle, and the alarm data sheet was given to the bedside nurse responsible for the patient to facilitate further scripted discussion during bedside rounds with patient-specific information to reduce the alarm rates of individual patients throughout the adjustment of physiologic monitor parameters (see Appendix 2 for sample data sheet and script).

Data Collection

Intervention patients were high-alarm, low-acuity patients on the east wing from June 1, 2015, through October 31, 2015. Two months of baseline data were gathered prior to intervention on all 3 wings; therefore, control patients were high-alarm, low-acuity patients throughout the PICU from April 1, 2015, to May 31, 2015, as historical controls and from June 1, 2015, to October 31, 2015, as concurrent controls. Alarm rates for the 24 hours prior to huddle and the 24 hours following huddle were collected and analyzed. See Figure 1 for schematic of study design.

We collected data on patient characteristics, including patient location, age, sex, and intervention date. Information regarding changes to monitor alarm parameters for both intervention and control patients during the posthuddle period (the period following morning huddle until noon on intervention day) was also collected. We monitored for code blue events and unexpected changes in acuity until discharge or transfer out of the PICU.

Data Analysis

We compared the priority alarm activation rates of individual patients in the 24 hours before and the 24 hours after the huddle intervention and contrasted the differences in rates between intervention and control patients, both concurrent and historical controls. We also divided the intervention and control groups into 2 additional groups each—those patients whose alarm parameters were changed, compared with those whose parameters did not change. We evaluated for possible contamination by comparing alarm rates of historical and concurrent controls, as well as evaluating alarm rates by location. We used mixed-effects regression models to evaluate the effect of the intervention and control type (historical or concurrent) on alarm rates, adjusted for patient age and sex. Analysis was performed using Stata version 10.3 (StataCorp, LLC, College Station, TX) and SAS version 9.4 (SAS Institute Inc., Cary, NC).

RESULTS

Because patients could be enrolled more than once, we refer to the instances when they were included in the study as “events” (huddle discussions for intervention patients and huddle opportunities for controls) below. We identified 49 historical control events between April 1, 2015, and May 31, 2015. During the intervention period, we identified 88 intervention events and 163 concurrent control events between June 1, 2015, and October 31, 2015 (total n = 300; see Table 1 for event characteristics). A total of 6 patients were enrolled more than once as either intervention or control patients.

 

UNADJUSTED ANALYSIS OF CHANGES IN ALARM RATES

The average priority alarm activation rate for intervention patients was 433 alarms (95% confidence interval [CI], 392-472) per day in the 24 hours leading up to the intervention and 223 alarms (95% CI, 182-265) per day in the 24 hours following the intervention, a 48.5% unadjusted decrease (95% CI, 38.1%-58.9%). In contrast, priority alarm activation rates for concurrent control patients averaged 412 alarms (95% CI, 383-442) per day in the 24 hours leading up to the morning huddle and 323 alarms (95% CI, 270-375) per day in the 24 hours following huddle, a 21.6% unadjusted decrease (95% CI, 15.3%-27.9%). For historical controls, priority alarm activation rates averaged 369 alarms (95% CI, 339-399) per day in the 24 hours leading up to the morning huddle and 242 alarms (95% CI, 164-320) per day in the 24 hours following huddle, a 34.4% unadjusted decrease (95% CI, 13.5%-55.0%). When we compared historical versus concurrent controls in the unadjusted analysis, concurrent controls had 37 more alarms per day (95% CI, 59 fewer to 134 more; P = 0.45) than historical controls. There was no significant difference between concurrent and historical controls, demonstrating no evidence of contamination.

Adjusted Analysis of Changes in Alarm Rates

The overall estimate of the effect of the intervention adjusted for age and sex compared with concurrent controls was a reduction of 116 priority alarms per day (95% CI, 37-194; P = 0.004, Table 2). The adjusted percent decrease was 29.0% (95% CI, 12.1%-46.0%). There were no unexpected changes in patient acuity or code blue events related to the intervention.

Fidelity Analysis

We tracked changes in alarm parameter settings for evidence of intervention fidelity to determine if the team carried out the recommendations made. We found that 42% of intervention patients and 24% of combined control patients had alarm parameters changed during the posthuddle period (P = 0.002).

For those intervention patients who had parameters changed during the posthuddle period (N = 37), the mean effect was greater at a 54.9% decrease (95% CI, 38.8%-70.8%) in priority alarms as compared with control patients who had parameters adjusted during the posthuddle period (n = 50), having a mean decrease of only 12.2% (95% CI, –18.1%-42.3%). There was a 43.2% decrease (95% CI, 29.3%-57.0%) for intervention patients who were discussed but did not have parameters adjusted during the time window of observation (n = 51), as compared with combined control patients who did not have parameters adjusted (N = 162) who had a 28.1% decrease (95% CI, 16.8%-39.1%); see Figure 2.

DISCUSSION

This study is the first to demonstrate a successful and safe intervention to reduce the alarm rates of PICU patients. In addition, we observed a more significant reduction in priority alarm activation rates for intervention patients who had their alarm parameters changed during the monitored time period, leading us to hypothesize that providing patient-specific data regarding types of alarms was a key component of the intervention.

In control patients, we observed a reduction in alarm rates over time as well. There are 2 potential explanations for this. First, it is possible that as patients stabilize in the PICU, their vital signs become less extreme and generate fewer alarms even if the alarm parameters are not changed. The second is that parameters were changed within or outside of the time windows during which we evaluated for alarm parameter changes. Nevertheless, the decline over time observed in the intervention patients was greater than in both control groups. This change was even more noticeable in the intervention patients who had their alarm parameters changed during the posthuddle period as compared with controls who had their alarm parameters changed following the posthuddle period. This may have been due to the data provided during the huddle intervention, pointing the team to the cause of the high alarm rate.

Prior successful research regarding reduction of pediatric alarms has often shown decreased use of physiological monitors as 1 approach to reducing unnecessary alarms. The single prior pediatric alarm intervention study conducted on a pediatric ward involved instituting a cardiac monitor care process that included the ordering of age-based parameters, daily replacement of electrodes, individualized assessment of parameters, and a reliable method to discontinue monitoring.¹³ Because most patients in the PICU are critically ill, the reliance on monitor discontinuation as a main approach to decreasing alarms is not feasible in this setting. Instead, the use of targeted alarm parameter adjustments for low-acuity patients demonstrated a safe and feasible approach to decreasing alarms in PICU patients. The daily electrode change and age-based parameters were already in place at our institution.

There are a few limitations to this study. First, we focused only on low-acuity PICU patients. We believe that focusing on low-acuity patients allows for reduction in nonactionable alarms with limited potential for adverse events; however, this approach excludes many critically ill patients who might be at highest risk for harm from alarm fatigue if important alarms are ignored. Second, many of our patients were not present for the full 24 hours pre- and posthuddle due to their low acuity limiting our ability to follow alarm rates over time. Third, changes in alarm parameters were only monitored for a set period of 5 hours following the huddle to determine the effect of the recommended rounding script on changes to alarms. It is possible the changes to alarm parameters outside of the observed posthuddle period affected the alarm rates of both intervention and control patients. Lastly, the balancing metrics of unexpected changes in OptiLink status and code blue events are rare events, and therefore we may have been underpowered to find them. The effects of the huddle intervention on safety huddle length and rounding length were not measured.

 

CONCLUSION

Integrating a data-driven monitor alarm discussion into safety huddles was a safe and effective approach to reduce alarms in low-acuity, high-alarm PICU patients. Innovative approaches to make data-driven alarm decisions using informatics tools integrated into monitoring systems and electronic health records have the potential to facilitate cost-effective spread of this intervention.

Disclosure

This work was supported by a pilot grant from the Center for Pediatric Clinical Effectiveness, The Children’s Hospital of Philadelphia. Dr. Bonafide is supported by a Mentored Patient-Oriented Research Career Development Award from the National Heart, Lung, and Blood Institute of the National Institutes of Health under Award Number K23HL116427. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding organizations or employers. The funding organizations had no role in the design, preparation, review, or approval of this paper, nor the decision to submit for publication.

BACKGROUND

Physiologic monitors are intended to prevent cardiac and respiratory arrest by generating alarms to alert clinicians to signs of instability. To minimize the probability that monitors will miss signs of deterioration, alarm algorithms and default parameters are often set to maximize sensitivity while sacrificing specificity.¹ As a result, monitors generate large numbers of nonactionable alarms—alarms that are either invalid and do not accurately represent the physiologic status of the patient or are valid but do not warrant clinical intervention.² Prior research has demonstrated that the pediatric intensive care unit (PICU) is responsible for a higher proportion of alarms than pediatric wards³ and a large proportion of these alarms, 87% - 97%, are nonactionable.^4-8 In national surveys of healthcare staff, respondents report that high alarm rates interrupt patient care and can lead clinicians to disable alarms entirely.⁹ Recent research has supported this, demonstrating that nurses who are exposed to higher numbers of alarms have slower response times to alarms.^4,10 In an attempt to mitigate safety risks, the Joint Commission in 2012 issued recommendations for hospitals to (a) establish guidelines for tailoring alarm settings and limits for individual patients and (b) identify situations in which alarms are not clinically necessary.¹¹

In order to address these recommendations within our PICU, we sought to evaluate the impact of a focused physiologic monitor alarm reduction intervention integrated into safety huddles. Safety huddles are brief, structured discussions among physicians, nurses, and other staff aiming to identify safety concerns.¹² Huddles offer an appropriate forum for reviewing alarm data and identifying patients whose high alarm rates may necessitate safe tailoring of alarm limits. Pilot data demonstrating high alarm rates among low-acuity PICU patients led us to hypothesize that low-acuity, high-alarm PICU patients would be a safe and effective target for an alarm huddle-based intervention.

In this study, we aimed to measure the impact of a structured safety huddle review of low-acuity PICU patients with high rates of priority alarms who were randomized to intervention compared with other low-acuity, high-alarm, concurrent, and historical control patients in the PICU.

METHODS

Study Definitions

Priority alarm activation rate. We conceptualized priority alarms as any alarm for a clinical condition that requires a timely response to determine if intervention is necessary to save a patient’s life,⁴ yet little empirical data support its existence in the hospital. We operationally defined these alarms on the General Electric Solar physiologic monitoring devices as any potentially life-threatening events including lethal arrhythmias (asystole, ventricular tachycardia, and ventricular fibrillation) and alarms for vital signs (heart rate, respiratory rate, and oxygen saturation) outside of the set parameter limits. These alarms produced audible tones in the patient room and automatically sent text messages to the nurse’s phone and had the potential to contribute to alarm fatigue regardless of the nurse’s location.

High-alarm patients. High-alarm patients were those who had more than 40 priority alarms in the preceding 4 hours, representing the top 20% of alarm rates in the PICU according to prior quality improvement projects completed in our PICU.

Low-acuity patients. Prior to and during this study, patient acuity was determined using the OptiLink Patient Classification System (OptiLink Healthcare Management Systems, Inc.; Tigard, OR; www.optilinkhealthcare.com; see Appendix 1) for the PICU twice daily. Low-acuity patients comprised on average 16% of the PICU patients.

Setting and Subjects

This study was performed in the PICU at The Children’s Hospital of Philadelphia.

The PICU is made up of 3 separate wings: east, south, and west. Bed availability was the only factor determining patient placement on the east, south, or west wing; the physical bed location was not preferentially assigned based on diagnosis or disease severity. The east wing was the intervention unit where the huddles occurred.

The PICU is composed of 3 different geographical teams. Two of the teams are composed of 4 to 5 pediatric or emergency medicine residents, 1 fellow, and 1 attending covering the south and west wings. The third team, located on the east wing, is composed of 1 to 2 pediatric residents, 2 to 3 nurse practitioners, 1 fellow, and 1 attending. Bedside family-centered rounds are held at each patient room, with the bedside nurse participating by reading a nursing rounding script that includes vital signs, vascular access, continuous medications, and additional questions or concerns.

Control subjects were any monitored patients on any of the 3 wings of the PICU between April 1, 2015, and October 31, 2015. The control patients were in 2 categories: historical controls from April 1, 2015, to May 31, 2015, and concurrent controls from June 1, 2015, to October 31, 2015, who were located anywhere in the PICU. On each nonholiday weekday beginning June 1, 2015, we randomly selected up to 2 patients to receive the intervention. These were high-alarm, low-acuity patients on the east wing to be discussed in the daily morning huddle. If more than 2 high-alarm, low-acuity patients were eligible for intervention, they were randomly selected by using the RAND function in Microsoft Excel. The other low-acuity, high-alarm patients in the PICU were included as control patients. Patients were eligible for the study if they were present for the 4 hours prior to huddle and present past noon on the day of huddle. If patients met criteria as high-alarm, low-acuity patients on multiple days, they could be enrolled as intervention or control patients multiple times. Patients’ alarm rates were calculated by dividing the number of alarms by their length of stay to the minute. There was no adjustment made for patients enrolled more than once.

 

Human Subjects Protection

The Institutional Review Board of The Children’s Hospital of Philadelphia approved this study with a waiver of informed consent.

Alarm Capture

We used BedMasterEx (Excel Medical Electronics; Jupiter, FL, http://excel-medical.com/products/bedmaster-ex) software connected to the General Electric monitor network to measure alarm rates. The software captured, in near real time, every alarm that occurred on every monitor in the PICU. Alarm rates over the preceding 4 hours for all PICU patients were exported and summarized by alarm type and level as set by hospital policy (crisis, warning, advisory, and system warning). Crisis and warning alarms were included as they represented potential life-threatening events meeting the definition of priority alarms. Physicians used an order within the PICU admission order-set to order monitoring based on preset age parameters (see online Appendix 1 for default settings). Physician orders were required for nurses to change alarm parameters. Daily electrode changes to reduce false alarms were standard of care.

Primary Outcome

The primary outcome was the change in priority alarm activation rate (the number of priority alarms per day) from prehuddle period (24 hours before morning huddle) to posthuddle period (the 24 hours following morning huddle) for intervention cases as compared with controls.

Primary Intervention

The intervention consisted of integrating a short script to facilitate the discussion of the alarm data during existing safety huddle and rounding workflows. The discussion and subsequent workflow proceeded as follows: A member of the research team who was not involved in patient care brought an alarm data sheet for each randomly selected intervention patient on the east wing to each safety huddle. The huddles were attended by the outgoing night charge nurse, the day charge nurse, and all bedside nurses working on the east wing that day. The alarm data sheet provided to the charge nurse displayed data on the 1 to 2 alarm parameters (respiratory rate, heart rate, or pulse oximetry) that generated the highest number of alarms. The charge nurse listed the high-alarm patients by room number during huddle, and the alarm data sheet was given to the bedside nurse responsible for the patient to facilitate further scripted discussion during bedside rounds with patient-specific information to reduce the alarm rates of individual patients throughout the adjustment of physiologic monitor parameters (see Appendix 2 for sample data sheet and script).

Data Collection

Intervention patients were high-alarm, low-acuity patients on the east wing from June 1, 2015, through October 31, 2015. Two months of baseline data were gathered prior to intervention on all 3 wings; therefore, control patients were high-alarm, low-acuity patients throughout the PICU from April 1, 2015, to May 31, 2015, as historical controls and from June 1, 2015, to October 31, 2015, as concurrent controls. Alarm rates for the 24 hours prior to huddle and the 24 hours following huddle were collected and analyzed. See Figure 1 for schematic of study design.

We collected data on patient characteristics, including patient location, age, sex, and intervention date. Information regarding changes to monitor alarm parameters for both intervention and control patients during the posthuddle period (the period following morning huddle until noon on intervention day) was also collected. We monitored for code blue events and unexpected changes in acuity until discharge or transfer out of the PICU.

Data Analysis

We compared the priority alarm activation rates of individual patients in the 24 hours before and the 24 hours after the huddle intervention and contrasted the differences in rates between intervention and control patients, both concurrent and historical controls. We also divided the intervention and control groups into 2 additional groups each—those patients whose alarm parameters were changed, compared with those whose parameters did not change. We evaluated for possible contamination by comparing alarm rates of historical and concurrent controls, as well as evaluating alarm rates by location. We used mixed-effects regression models to evaluate the effect of the intervention and control type (historical or concurrent) on alarm rates, adjusted for patient age and sex. Analysis was performed using Stata version 10.3 (StataCorp, LLC, College Station, TX) and SAS version 9.4 (SAS Institute Inc., Cary, NC).

RESULTS

Because patients could be enrolled more than once, we refer to the instances when they were included in the study as “events” (huddle discussions for intervention patients and huddle opportunities for controls) below. We identified 49 historical control events between April 1, 2015, and May 31, 2015. During the intervention period, we identified 88 intervention events and 163 concurrent control events between June 1, 2015, and October 31, 2015 (total n = 300; see Table 1 for event characteristics). A total of 6 patients were enrolled more than once as either intervention or control patients.

 

UNADJUSTED ANALYSIS OF CHANGES IN ALARM RATES

The average priority alarm activation rate for intervention patients was 433 alarms (95% confidence interval [CI], 392-472) per day in the 24 hours leading up to the intervention and 223 alarms (95% CI, 182-265) per day in the 24 hours following the intervention, a 48.5% unadjusted decrease (95% CI, 38.1%-58.9%). In contrast, priority alarm activation rates for concurrent control patients averaged 412 alarms (95% CI, 383-442) per day in the 24 hours leading up to the morning huddle and 323 alarms (95% CI, 270-375) per day in the 24 hours following huddle, a 21.6% unadjusted decrease (95% CI, 15.3%-27.9%). For historical controls, priority alarm activation rates averaged 369 alarms (95% CI, 339-399) per day in the 24 hours leading up to the morning huddle and 242 alarms (95% CI, 164-320) per day in the 24 hours following huddle, a 34.4% unadjusted decrease (95% CI, 13.5%-55.0%). When we compared historical versus concurrent controls in the unadjusted analysis, concurrent controls had 37 more alarms per day (95% CI, 59 fewer to 134 more; P = 0.45) than historical controls. There was no significant difference between concurrent and historical controls, demonstrating no evidence of contamination.

Adjusted Analysis of Changes in Alarm Rates

The overall estimate of the effect of the intervention adjusted for age and sex compared with concurrent controls was a reduction of 116 priority alarms per day (95% CI, 37-194; P = 0.004, Table 2). The adjusted percent decrease was 29.0% (95% CI, 12.1%-46.0%). There were no unexpected changes in patient acuity or code blue events related to the intervention.

Fidelity Analysis

We tracked changes in alarm parameter settings for evidence of intervention fidelity to determine if the team carried out the recommendations made. We found that 42% of intervention patients and 24% of combined control patients had alarm parameters changed during the posthuddle period (P = 0.002).

For those intervention patients who had parameters changed during the posthuddle period (N = 37), the mean effect was greater at a 54.9% decrease (95% CI, 38.8%-70.8%) in priority alarms as compared with control patients who had parameters adjusted during the posthuddle period (n = 50), having a mean decrease of only 12.2% (95% CI, –18.1%-42.3%). There was a 43.2% decrease (95% CI, 29.3%-57.0%) for intervention patients who were discussed but did not have parameters adjusted during the time window of observation (n = 51), as compared with combined control patients who did not have parameters adjusted (N = 162) who had a 28.1% decrease (95% CI, 16.8%-39.1%); see Figure 2.

DISCUSSION

This study is the first to demonstrate a successful and safe intervention to reduce the alarm rates of PICU patients. In addition, we observed a more significant reduction in priority alarm activation rates for intervention patients who had their alarm parameters changed during the monitored time period, leading us to hypothesize that providing patient-specific data regarding types of alarms was a key component of the intervention.

In control patients, we observed a reduction in alarm rates over time as well. There are 2 potential explanations for this. First, it is possible that as patients stabilize in the PICU, their vital signs become less extreme and generate fewer alarms even if the alarm parameters are not changed. The second is that parameters were changed within or outside of the time windows during which we evaluated for alarm parameter changes. Nevertheless, the decline over time observed in the intervention patients was greater than in both control groups. This change was even more noticeable in the intervention patients who had their alarm parameters changed during the posthuddle period as compared with controls who had their alarm parameters changed following the posthuddle period. This may have been due to the data provided during the huddle intervention, pointing the team to the cause of the high alarm rate.

Prior successful research regarding reduction of pediatric alarms has often shown decreased use of physiological monitors as 1 approach to reducing unnecessary alarms. The single prior pediatric alarm intervention study conducted on a pediatric ward involved instituting a cardiac monitor care process that included the ordering of age-based parameters, daily replacement of electrodes, individualized assessment of parameters, and a reliable method to discontinue monitoring.¹³ Because most patients in the PICU are critically ill, the reliance on monitor discontinuation as a main approach to decreasing alarms is not feasible in this setting. Instead, the use of targeted alarm parameter adjustments for low-acuity patients demonstrated a safe and feasible approach to decreasing alarms in PICU patients. The daily electrode change and age-based parameters were already in place at our institution.

There are a few limitations to this study. First, we focused only on low-acuity PICU patients. We believe that focusing on low-acuity patients allows for reduction in nonactionable alarms with limited potential for adverse events; however, this approach excludes many critically ill patients who might be at highest risk for harm from alarm fatigue if important alarms are ignored. Second, many of our patients were not present for the full 24 hours pre- and posthuddle due to their low acuity limiting our ability to follow alarm rates over time. Third, changes in alarm parameters were only monitored for a set period of 5 hours following the huddle to determine the effect of the recommended rounding script on changes to alarms. It is possible the changes to alarm parameters outside of the observed posthuddle period affected the alarm rates of both intervention and control patients. Lastly, the balancing metrics of unexpected changes in OptiLink status and code blue events are rare events, and therefore we may have been underpowered to find them. The effects of the huddle intervention on safety huddle length and rounding length were not measured.

 

CONCLUSION

Integrating a data-driven monitor alarm discussion into safety huddles was a safe and effective approach to reduce alarms in low-acuity, high-alarm PICU patients. Innovative approaches to make data-driven alarm decisions using informatics tools integrated into monitoring systems and electronic health records have the potential to facilitate cost-effective spread of this intervention.

Disclosure

This work was supported by a pilot grant from the Center for Pediatric Clinical Effectiveness, The Children’s Hospital of Philadelphia. Dr. Bonafide is supported by a Mentored Patient-Oriented Research Career Development Award from the National Heart, Lung, and Blood Institute of the National Institutes of Health under Award Number K23HL116427. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding organizations or employers. The funding organizations had no role in the design, preparation, review, or approval of this paper, nor the decision to submit for publication.

BACKGROUND

Physiologic monitors are intended to prevent cardiac and respiratory arrest by generating alarms to alert clinicians to signs of instability. To minimize the probability that monitors will miss signs of deterioration, alarm algorithms and default parameters are often set to maximize sensitivity while sacrificing specificity.¹ As a result, monitors generate large numbers of nonactionable alarms—alarms that are either invalid and do not accurately represent the physiologic status of the patient or are valid but do not warrant clinical intervention.² Prior research has demonstrated that the pediatric intensive care unit (PICU) is responsible for a higher proportion of alarms than pediatric wards³ and a large proportion of these alarms, 87% - 97%, are nonactionable.^4-8 In national surveys of healthcare staff, respondents report that high alarm rates interrupt patient care and can lead clinicians to disable alarms entirely.⁹ Recent research has supported this, demonstrating that nurses who are exposed to higher numbers of alarms have slower response times to alarms.^4,10 In an attempt to mitigate safety risks, the Joint Commission in 2012 issued recommendations for hospitals to (a) establish guidelines for tailoring alarm settings and limits for individual patients and (b) identify situations in which alarms are not clinically necessary.¹¹

In order to address these recommendations within our PICU, we sought to evaluate the impact of a focused physiologic monitor alarm reduction intervention integrated into safety huddles. Safety huddles are brief, structured discussions among physicians, nurses, and other staff aiming to identify safety concerns.¹² Huddles offer an appropriate forum for reviewing alarm data and identifying patients whose high alarm rates may necessitate safe tailoring of alarm limits. Pilot data demonstrating high alarm rates among low-acuity PICU patients led us to hypothesize that low-acuity, high-alarm PICU patients would be a safe and effective target for an alarm huddle-based intervention.

In this study, we aimed to measure the impact of a structured safety huddle review of low-acuity PICU patients with high rates of priority alarms who were randomized to intervention compared with other low-acuity, high-alarm, concurrent, and historical control patients in the PICU.

METHODS

Study Definitions

Priority alarm activation rate. We conceptualized priority alarms as any alarm for a clinical condition that requires a timely response to determine if intervention is necessary to save a patient’s life,⁴ yet little empirical data support its existence in the hospital. We operationally defined these alarms on the General Electric Solar physiologic monitoring devices as any potentially life-threatening events including lethal arrhythmias (asystole, ventricular tachycardia, and ventricular fibrillation) and alarms for vital signs (heart rate, respiratory rate, and oxygen saturation) outside of the set parameter limits. These alarms produced audible tones in the patient room and automatically sent text messages to the nurse’s phone and had the potential to contribute to alarm fatigue regardless of the nurse’s location.

High-alarm patients. High-alarm patients were those who had more than 40 priority alarms in the preceding 4 hours, representing the top 20% of alarm rates in the PICU according to prior quality improvement projects completed in our PICU.

Low-acuity patients. Prior to and during this study, patient acuity was determined using the OptiLink Patient Classification System (OptiLink Healthcare Management Systems, Inc.; Tigard, OR; www.optilinkhealthcare.com; see Appendix 1) for the PICU twice daily. Low-acuity patients comprised on average 16% of the PICU patients.

Setting and Subjects

This study was performed in the PICU at The Children’s Hospital of Philadelphia.

The PICU is made up of 3 separate wings: east, south, and west. Bed availability was the only factor determining patient placement on the east, south, or west wing; the physical bed location was not preferentially assigned based on diagnosis or disease severity. The east wing was the intervention unit where the huddles occurred.

The PICU is composed of 3 different geographical teams. Two of the teams are composed of 4 to 5 pediatric or emergency medicine residents, 1 fellow, and 1 attending covering the south and west wings. The third team, located on the east wing, is composed of 1 to 2 pediatric residents, 2 to 3 nurse practitioners, 1 fellow, and 1 attending. Bedside family-centered rounds are held at each patient room, with the bedside nurse participating by reading a nursing rounding script that includes vital signs, vascular access, continuous medications, and additional questions or concerns.

Control subjects were any monitored patients on any of the 3 wings of the PICU between April 1, 2015, and October 31, 2015. The control patients were in 2 categories: historical controls from April 1, 2015, to May 31, 2015, and concurrent controls from June 1, 2015, to October 31, 2015, who were located anywhere in the PICU. On each nonholiday weekday beginning June 1, 2015, we randomly selected up to 2 patients to receive the intervention. These were high-alarm, low-acuity patients on the east wing to be discussed in the daily morning huddle. If more than 2 high-alarm, low-acuity patients were eligible for intervention, they were randomly selected by using the RAND function in Microsoft Excel. The other low-acuity, high-alarm patients in the PICU were included as control patients. Patients were eligible for the study if they were present for the 4 hours prior to huddle and present past noon on the day of huddle. If patients met criteria as high-alarm, low-acuity patients on multiple days, they could be enrolled as intervention or control patients multiple times. Patients’ alarm rates were calculated by dividing the number of alarms by their length of stay to the minute. There was no adjustment made for patients enrolled more than once.

 

Human Subjects Protection

The Institutional Review Board of The Children’s Hospital of Philadelphia approved this study with a waiver of informed consent.

Alarm Capture

We used BedMasterEx (Excel Medical Electronics; Jupiter, FL, http://excel-medical.com/products/bedmaster-ex) software connected to the General Electric monitor network to measure alarm rates. The software captured, in near real time, every alarm that occurred on every monitor in the PICU. Alarm rates over the preceding 4 hours for all PICU patients were exported and summarized by alarm type and level as set by hospital policy (crisis, warning, advisory, and system warning). Crisis and warning alarms were included as they represented potential life-threatening events meeting the definition of priority alarms. Physicians used an order within the PICU admission order-set to order monitoring based on preset age parameters (see online Appendix 1 for default settings). Physician orders were required for nurses to change alarm parameters. Daily electrode changes to reduce false alarms were standard of care.

Primary Outcome

The primary outcome was the change in priority alarm activation rate (the number of priority alarms per day) from prehuddle period (24 hours before morning huddle) to posthuddle period (the 24 hours following morning huddle) for intervention cases as compared with controls.

Primary Intervention

The intervention consisted of integrating a short script to facilitate the discussion of the alarm data during existing safety huddle and rounding workflows. The discussion and subsequent workflow proceeded as follows: A member of the research team who was not involved in patient care brought an alarm data sheet for each randomly selected intervention patient on the east wing to each safety huddle. The huddles were attended by the outgoing night charge nurse, the day charge nurse, and all bedside nurses working on the east wing that day. The alarm data sheet provided to the charge nurse displayed data on the 1 to 2 alarm parameters (respiratory rate, heart rate, or pulse oximetry) that generated the highest number of alarms. The charge nurse listed the high-alarm patients by room number during huddle, and the alarm data sheet was given to the bedside nurse responsible for the patient to facilitate further scripted discussion during bedside rounds with patient-specific information to reduce the alarm rates of individual patients throughout the adjustment of physiologic monitor parameters (see Appendix 2 for sample data sheet and script).

Data Collection

Intervention patients were high-alarm, low-acuity patients on the east wing from June 1, 2015, through October 31, 2015. Two months of baseline data were gathered prior to intervention on all 3 wings; therefore, control patients were high-alarm, low-acuity patients throughout the PICU from April 1, 2015, to May 31, 2015, as historical controls and from June 1, 2015, to October 31, 2015, as concurrent controls. Alarm rates for the 24 hours prior to huddle and the 24 hours following huddle were collected and analyzed. See Figure 1 for schematic of study design.

We collected data on patient characteristics, including patient location, age, sex, and intervention date. Information regarding changes to monitor alarm parameters for both intervention and control patients during the posthuddle period (the period following morning huddle until noon on intervention day) was also collected. We monitored for code blue events and unexpected changes in acuity until discharge or transfer out of the PICU.

Data Analysis

We compared the priority alarm activation rates of individual patients in the 24 hours before and the 24 hours after the huddle intervention and contrasted the differences in rates between intervention and control patients, both concurrent and historical controls. We also divided the intervention and control groups into 2 additional groups each—those patients whose alarm parameters were changed, compared with those whose parameters did not change. We evaluated for possible contamination by comparing alarm rates of historical and concurrent controls, as well as evaluating alarm rates by location. We used mixed-effects regression models to evaluate the effect of the intervention and control type (historical or concurrent) on alarm rates, adjusted for patient age and sex. Analysis was performed using Stata version 10.3 (StataCorp, LLC, College Station, TX) and SAS version 9.4 (SAS Institute Inc., Cary, NC).

RESULTS

Because patients could be enrolled more than once, we refer to the instances when they were included in the study as “events” (huddle discussions for intervention patients and huddle opportunities for controls) below. We identified 49 historical control events between April 1, 2015, and May 31, 2015. During the intervention period, we identified 88 intervention events and 163 concurrent control events between June 1, 2015, and October 31, 2015 (total n = 300; see Table 1 for event characteristics). A total of 6 patients were enrolled more than once as either intervention or control patients.

 

UNADJUSTED ANALYSIS OF CHANGES IN ALARM RATES

The average priority alarm activation rate for intervention patients was 433 alarms (95% confidence interval [CI], 392-472) per day in the 24 hours leading up to the intervention and 223 alarms (95% CI, 182-265) per day in the 24 hours following the intervention, a 48.5% unadjusted decrease (95% CI, 38.1%-58.9%). In contrast, priority alarm activation rates for concurrent control patients averaged 412 alarms (95% CI, 383-442) per day in the 24 hours leading up to the morning huddle and 323 alarms (95% CI, 270-375) per day in the 24 hours following huddle, a 21.6% unadjusted decrease (95% CI, 15.3%-27.9%). For historical controls, priority alarm activation rates averaged 369 alarms (95% CI, 339-399) per day in the 24 hours leading up to the morning huddle and 242 alarms (95% CI, 164-320) per day in the 24 hours following huddle, a 34.4% unadjusted decrease (95% CI, 13.5%-55.0%). When we compared historical versus concurrent controls in the unadjusted analysis, concurrent controls had 37 more alarms per day (95% CI, 59 fewer to 134 more; P = 0.45) than historical controls. There was no significant difference between concurrent and historical controls, demonstrating no evidence of contamination.

Adjusted Analysis of Changes in Alarm Rates

The overall estimate of the effect of the intervention adjusted for age and sex compared with concurrent controls was a reduction of 116 priority alarms per day (95% CI, 37-194; P = 0.004, Table 2). The adjusted percent decrease was 29.0% (95% CI, 12.1%-46.0%). There were no unexpected changes in patient acuity or code blue events related to the intervention.

Fidelity Analysis

We tracked changes in alarm parameter settings for evidence of intervention fidelity to determine if the team carried out the recommendations made. We found that 42% of intervention patients and 24% of combined control patients had alarm parameters changed during the posthuddle period (P = 0.002).

For those intervention patients who had parameters changed during the posthuddle period (N = 37), the mean effect was greater at a 54.9% decrease (95% CI, 38.8%-70.8%) in priority alarms as compared with control patients who had parameters adjusted during the posthuddle period (n = 50), having a mean decrease of only 12.2% (95% CI, –18.1%-42.3%). There was a 43.2% decrease (95% CI, 29.3%-57.0%) for intervention patients who were discussed but did not have parameters adjusted during the time window of observation (n = 51), as compared with combined control patients who did not have parameters adjusted (N = 162) who had a 28.1% decrease (95% CI, 16.8%-39.1%); see Figure 2.

DISCUSSION

This study is the first to demonstrate a successful and safe intervention to reduce the alarm rates of PICU patients. In addition, we observed a more significant reduction in priority alarm activation rates for intervention patients who had their alarm parameters changed during the monitored time period, leading us to hypothesize that providing patient-specific data regarding types of alarms was a key component of the intervention.

In control patients, we observed a reduction in alarm rates over time as well. There are 2 potential explanations for this. First, it is possible that as patients stabilize in the PICU, their vital signs become less extreme and generate fewer alarms even if the alarm parameters are not changed. The second is that parameters were changed within or outside of the time windows during which we evaluated for alarm parameter changes. Nevertheless, the decline over time observed in the intervention patients was greater than in both control groups. This change was even more noticeable in the intervention patients who had their alarm parameters changed during the posthuddle period as compared with controls who had their alarm parameters changed following the posthuddle period. This may have been due to the data provided during the huddle intervention, pointing the team to the cause of the high alarm rate.

Prior successful research regarding reduction of pediatric alarms has often shown decreased use of physiological monitors as 1 approach to reducing unnecessary alarms. The single prior pediatric alarm intervention study conducted on a pediatric ward involved instituting a cardiac monitor care process that included the ordering of age-based parameters, daily replacement of electrodes, individualized assessment of parameters, and a reliable method to discontinue monitoring.¹³ Because most patients in the PICU are critically ill, the reliance on monitor discontinuation as a main approach to decreasing alarms is not feasible in this setting. Instead, the use of targeted alarm parameter adjustments for low-acuity patients demonstrated a safe and feasible approach to decreasing alarms in PICU patients. The daily electrode change and age-based parameters were already in place at our institution.

There are a few limitations to this study. First, we focused only on low-acuity PICU patients. We believe that focusing on low-acuity patients allows for reduction in nonactionable alarms with limited potential for adverse events; however, this approach excludes many critically ill patients who might be at highest risk for harm from alarm fatigue if important alarms are ignored. Second, many of our patients were not present for the full 24 hours pre- and posthuddle due to their low acuity limiting our ability to follow alarm rates over time. Third, changes in alarm parameters were only monitored for a set period of 5 hours following the huddle to determine the effect of the recommended rounding script on changes to alarms. It is possible the changes to alarm parameters outside of the observed posthuddle period affected the alarm rates of both intervention and control patients. Lastly, the balancing metrics of unexpected changes in OptiLink status and code blue events are rare events, and therefore we may have been underpowered to find them. The effects of the huddle intervention on safety huddle length and rounding length were not measured.

 

CONCLUSION

Integrating a data-driven monitor alarm discussion into safety huddles was a safe and effective approach to reduce alarms in low-acuity, high-alarm PICU patients. Innovative approaches to make data-driven alarm decisions using informatics tools integrated into monitoring systems and electronic health records have the potential to facilitate cost-effective spread of this intervention.

Disclosure

This work was supported by a pilot grant from the Center for Pediatric Clinical Effectiveness, The Children’s Hospital of Philadelphia. Dr. Bonafide is supported by a Mentored Patient-Oriented Research Career Development Award from the National Heart, Lung, and Blood Institute of the National Institutes of Health under Award Number K23HL116427. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding organizations or employers. The funding organizations had no role in the design, preparation, review, or approval of this paper, nor the decision to submit for publication.

Central venous catheter (CVC) placement is commonly performed in emergency and critical care settings for parenteral access, central monitoring, and hemodialysis. Although potentially lifesaving CVC insertion is associated with immediate risks including injury to nerves, vessels, and lungs.^1-3 These “insertion-related complications” are of particular interest for several reasons. First, the frequency of such complications varies widely, with published rates between 1.4% and 33.2%.^2-7 Reasons for such variation include differences in study definitions of complications (eg, pneumothorax and tip position),^2,5 setting of CVC placement (eg, intensive care unit [ICU] vs emergency room), timing of placement (eg, elective vs emergent), differences in technique, and type of operator (eg, experienced vs learner). Thus, the precise incidence of such events in modern-day training settings with use of ultrasound guidance remains uncertain. Second, mechanical complications might be preventable with adequate training and supervision. Indeed, studies using simulation-based mastery techniques have demonstrated a reduction in rates of complications following intensive training.⁸ Finally, understanding risk factors associated with insertion complications might inform preventative strategies and improve patient safety.^9-11

Few studies to date have examined trainees’ perceptions on CVC training, experience, supervision, and ability to recognize and prevent mechanical complications. While research investigating effects of simulation training has accumulated, most focus on successful completion of the procedure or individual procedural steps with little emphasis on operator perceptions.^12-14 In addition, while multiple studies have shown that unsuccessful line attempts are a risk factor for CVC complications,^3,4,7,15 there is very little known about trainee behavior and perceptions regarding unsuccessful line placement. CVC simulation trainings often assume successful completion of the procedure and do not address the crucial postprocedure steps that should be undertaken if a procedure is unsuccessful. For these reasons, we developed a survey to specifically examine trainee experience with CVC placement, supervision, postprocedural behavior, and attitudes regarding unsuccessful line placement.

Therefore, we designed a study with 2 specific goals: The first is to perform a contemporary analysis of CVC mechanical complication rate at an academic teaching institution and identify potential risk factors associated with these complications. Second, we sought to determine trainee perceptions regarding CVC complication experience, prevention, procedural supervision, and perceptions surrounding unsuccessful line placement.

METHODS

Design and Setting

We conducted a single-center retrospective review of nontunneled acute CVC procedures between June 1, 2014, and May 1, 2015, at the University of Michigan Health System (UMHS). UMHS is a tertiary care referral center with over 900 inpatient beds, including 99 ICU beds.

All residents in internal medicine, surgery, anesthesia, and emergency medicine receive mandatory education in CVC placement that includes an online training module and simulation-based training with competency assessment. Use of real-time ultrasound guidance is considered the standard of care for CVC placement.

Data Collection

Inpatient procedure notes were electronically searched for terms indicating CVC placement. This was performed by using our hospital’s Data Office for Clinical and Translational Research using the Electronic Medical Record Search Engine tool. Please see the supplemental materials for the full list of search terms. We electronically extracted data, including date of procedure, gender, and most recent body mass index (BMI), within 1 year prior to note. Acute Physiology and Chronic Health Evaluation III (APACHE III) data are tracked for all patients on admission to ICU; this was collected when available. Charts were then manually reviewed to collect additional data, including international normalized ratio (INR), platelet count, lactate level on the day of CVC placement, anticoagulant use (actively prescribed coumadin, therapeutic enoxaparin, therapeutic unfractionated heparin, or direct oral anticoagulant), ventilator or noninvasive positive pressure ventilation (NIPPV) at time of CVC placement, and vasopressor requirement within 24 hours of CVC placement. The procedure note was reviewed to gather information about site of CVC placement, size and type of catheter, number of attempts, procedural success, training level of the operator, and attending presence. Small bore CVCs were defined as 7 French (Fr) or lower. Large bore CVCs were defined as >7 Fr; this includes dialysis catheters, Cordis catheters (Cordis, Fremont, CA), and cooling catheters. The times of the procedure note and postprocedure chest x-ray (CXR) were recorded, including whether the CVC was placed on a weekend (Friday 7 pm to Sunday at midnight) or weekday.

 

Primary Outcome

The primary outcome was the rate of severe mechanical complications related to CVC placement. Similar to prior studies,² we defined severe mechanical complications as arterial placement of dilator or catheter, hemothorax, pneumothorax, cerebral ischemia, patient death (related to procedure), significant hematoma, or vascular injury (defined as complication requiring expert consultation or blood product transfusion). We did not require a lower limit on blood transfusion. We considered pneumothorax a complication regardless of whether chest tube intervention was performed, as pneumothorax subjects the patient to additional tests (eg, serial CXRs) and sometimes symptoms (shortness of breath, pain, anxiety) regardless of whether or not a chest tube was required. Complications were confirmed by a direct review of procedure notes, progress notes, discharge summaries, and imaging studies.

Trainee Survey

A survey was electronically disseminated to all internal medicine and medicine-pediatric residents to inquire about CVC experiences, including time spent in the medical ICU, number of CVCs performed, postprocedure behavior for both failed and successful CVCs, and supervision experience and attitudes. Please see supplemental materials for full survey contents.

Statistical Methods

Descriptive statistics (percentage) were used to summarize data. Continuous and categorical variables were compared using Student t tests and chi-square tests, respectively. All analyses were performed using SAS 9.3 (SAS Institute, Cary, NC).

Ethical and Regulatory Oversight

The study was deemed exempt by the University of Michigan Institutional Review Board (HUM00100549) as data collection was part of a quality improvement effort.

RESULTS

Demographics and Characteristics of Device Insertion

Between June 1, 2014, and May 1, 2015, 730 CVC procedure notes were reviewed (Table 1). The mean age of the study population was 58.9 years, and 41.6% (n = 304) were female. BMI data were available in 400 patients without complications and 5 patients with complications; the average BMI was 31.5 kg/m². The APACHE III score was available for 442 patients without complications and 10 patients with complications; the average score was 86 (range 19-200). Most of the CVCs placed (n= 504, 69%) were small bore (<7 Fr). The majority of catheters were placed in the internal jugular (IJ) position (n = 525, 71.9%), followed by femoral (n = 144, 19.7%), subclavian (N = 57, 7.8%), and undocumented (n = 4, 0.6%). Ninety-six percent (n = 699) of CVCs were successfully placed. Seventy-six percent (n = 558) of procedure notes included documentation of the number of CVC attempts; of these, 85% documented 2 or fewer attempts. The majority of CVCs were placed by residents (n = 537, 73.9%), followed by fellows (N = 127, 17.5%) and attendings (n = 27, 3.7%). Attending supervision for all or key portions of CVC placement occurred 34.7% (n = 244) of the time overall and was lower for internal medicine trainees (n = 98/463, 21.2%) compared with surgical trainees (n = 73/127, 57.4%) or emergency medicine trainees (n = 62/96, 64.6%; P < 0.001). All successful IJ and subclavian CVCs except for 2 insertions (0.3%) had a postprocedure CXR. A minority of notes documented pressure transduction (4.5%) or blood gas analysis (0.2%) to confirm venous placement.

Lab data, information on utilization of anticoagulation, vasopressors, ventilation, and information on the use of transduction and blood gas data were collected for the first 410 uncomplicated patients and all patients that experienced complications (Table 2). The mean INR was 1.5 (range 0.9-9.7), mean platelets 180 K/uL (range 9-816 K/uL), and mean lactate 2.9 mmol/L (range 0.4-16 mmol/L). Twenty-one percent (n = 86) of patients were systemically anticoagulated at the time of CVC placement, 59% (n = 251) received vasopressors within 24 hours of CVC placement, and 63% (n = 265) were mechanically ventilated (Table 2).

Mechanical Complications

The mechanical complications identified included pneumothorax (n = 5), bleeding requiring transfusion (n = 3), vascular injury requiring expert consultation or intervention (n = 3), stroke (n = 1), and death (n = 2). Vascular injuries included 1 neck hematoma with superinfection requiring antibiotics, 1 neck hematoma requiring otolaryngology and vascular surgery consultation, and 1 venous dissection of IJ vein requiring vascular surgery consultation. None of these cases required operative intervention. The stroke was caused by inadvertent CVC placement into the carotid artery. One patient experienced tension pneumothorax and died due to this complication; this death occurred after 3 failed left subclavian CVC attempts and an ultimately successful CVC placement into left IJ vein. Another death occurred immediately following unsuccessful Cordis placement. As no autopsy was performed, it is impossible to know if the cause of death was the line placement. However, it would be prudent to consider this as a CVC complication given the temporal relationship to line placement. Thus, the total number of patients who experienced severe mechanical complications was 14 out of 730 (1.92%).

 

Risk Factors for Mechanical Complications

Certain patient factors were more commonly associated with complications. For example, BMI was significantly lower in the group that experienced complications vs those that did not (25.7 vs 31.0 kg/m², P = 0.001). No other associations between demographic factors, including age (61.4 years vs 58.9 years, P = 0.57) or sex (57.1% male vs 41.3% female, P = 0.24), or admission APACHE III score (96 vs 86, P = 0.397) were noted. The mean INR, platelets, and lactate did not differ between the 2 groups. There was no difference between the use of vasopressors. Ventilator use (including endotracheal tube or NIPPV) was found to be significantly higher in the group that experienced mechanical complications (78.5% vs 65.9%, P = 0.001). Anticoagulation use was also associated with mechanical complications (28.6% vs 20.6%, P = 0.05); 3 patients on anticoagulation experienced significant hematomas. Mechanical complications were more common with subclavian location (21.4% vs 7.8%, P = 0.001); in all 3 cases involving subclavian CVC placement, the complication experienced was pneumothorax. The number of attempts significantly differed between the 2 groups, with an average of 1.5 attempts in the group without complications and 2.2 attempts in the group that experienced complications (P = 0.02). Additionally, rates of successful placement were lower among patients who experienced complications (78.6% vs 95.7%, P = 0.001).

With respect to operator characteristics, no significant difference between the levels of training was noted among those who experienced complications vs those who did not. Attending supervision was more frequent for the group that experienced complications (61.5% vs 34.2%, P = 0.04). There was no significant difference in complication rate according to the first vs the second half of the academic year (0.4% vs 0.3% per month, P = 0.30) or CVC placement during the day vs night (1.9% vs 2.0%, P = 0.97). A trend toward more complications in CVCs placed over the weekend compared to a weekday was observed (2.80% vs 1.23%, P = 0.125).

Unsuccessful CVCs

There were 30 documented unsuccessful CVC procedures, representing 4.1% of all procedures. Of these, 3 procedures had complications; these included 2 pneumothoraxes (1 leading to death) and 1 unexplained death. Twenty-four of the unsuccessful CVC attempts were in either the subclavian or IJ location; of these, 5 (21%) did not have a postprocedure CXR obtained.

Survey Results

The survey was completed by 103 out of 166 internal medicine residents (62% response rate). Of these, 55% (n = 57) reported having performed 5 or more CVCs, and 14% (n = 14) had performed more than 15 CVCs.

All respondents who had performed at least 1 CVC (n = 80) were asked about their perceptions regarding attending supervision. Eighty-one percent (n = 65/80) responded that they have never been directly supervised by an attending during CVC placement, while 16% (n = 13/80) reported being supervised less than 25% of the time. Most (n = 53/75, 71%) did not feel that attending supervision affected their performance, while 21% (n = 16/75) felt it affected performance negatively, and only 8% (n = 6/75) stated it affected performance positively. Nineteen percent (n = 15/80) indicated that they prefer more supervision by attendings, while 35% (n = 28/80) did not wish for more attending supervision, and 46% (n = 37/80) were indifferent.

All respondents who had performed at least 1 CVC were asked about postprocedure protocols. The vast majority (n = 77/80, 95%) reported documenting a postprocedure note more than 75% of the time after a successful procedure; in contrast, only 38% (n = 30/80) of those who had failed a CVC placement reported documenting a procedure note more than 75% of the time (Figure 1). Only 35% (n = 21/60) of respondents reported routinely (100% of the time) ordering a CXR after a failed chest CVC attempt (Figure 2), and 47% (n = 28/60) only ordered a CXR if they were concerned there was a complication. Most (69%, n = 55/80) felt it was important to order a CXR after a failed chest CVC placement, while 6% (n = 5/80) did not feel it was important, and 25% (n = 20/80) did not know or were indifferent.

DISCUSSION

We performed a contemporary analysis of CVC placement at an academic tertiary care center and observed a rate of severe mechanical complications of 1.9%. This rate is within previously described acceptable thresholds.¹⁶ Our study adds to the literature by identifying several important risk factors for development of mechanical complications. We confirm many risk factors that have been noted historically, such as subclavian line location,^2,3 attending supervision,³ low BMI,⁴ number of CVC attempts, and unsuccessful CVC placement.^3,4,7,15 We identified several unique risk factors, including systemic anticoagulation as well as ventilator use. Lastly, we identified unexpected deficits in trainee knowledge surrounding management of failed CVCs and negative attitudes regarding attending supervision.

 

Most existing literature evaluated risk factors for CVC complication prior to routine ultrasound use;^3-5,7,15 surprisingly, it appears that severe mechanical complications do not differ dramatically in the real-time ultrasound era. Eisen et al.³ prospectively studied CVC placement at an academic medical center and found a severe mechanical complication rate (as defined in our paper) of 1.9% due to pneumothorax (1.3%), hemothorax (0.3%), and death (0.3%).We would expect the number of complications to decrease in the postultrasound era, and indeed it appears that pneumothoraces have decreased likely due to ultrasound guidance and decrease in subclavian location. However, in contrast, rates of significant hematomas and bleeding are higher in our study. Although we are unable to state why this may be the case, increasing use of anticoagulation in the general population might explain this finding.¹⁷ For instance, of the 6 patients who experienced hematomas or vascular injuries in our study, 3 were on anticoagulation at the time of CVC placement.

Interestingly, time of academic year of CVC placement and level of training were not correlated with an increased risk of complications, nor was time of day of CVC placement. In contrast, Merrer et al.showed that CVC insertion during nighttime was significantly associated with increased mechanical complications (odds ratio 2.06, 95% confidence interval, 1.04-4.08;,P = 0.03).⁵ This difference may be attributable to the fact that most of our ICUs now have a night float system rather than a more traditional 24-hour call model; therefore, trainees are less likely to be sleep deprived during CVC placement at night.

Severity of illness did not appear to significantly affect mechanical complication rates based on similar APACHE scores between the 2 groups. In addition, other indicators of illness severity (vasopressor use or lactate level) did not suggest that sicker patients may be more likely to experience mechanical complications than others. One could conjecture that perhaps sicker patients were more likely to have lines placed by more experienced trainees, although the present study design does not allow us to answer this question. Interestingly, ventilator use was associated with higher rates of complications. We cannot say definitively why this was the case; however, 1 contributing factor may be the physical constraints of placing the CVC around ventilator tubing.

Several unexpected findings surrounding attending supervision were noted: first, attending supervision appears to be significantly associated with increased complication rate, and second, trainees have negative perceptions regarding attending supervision. Eisen et al.showed a similar association between attending supervision and complication rate.³ It is possible that the increased complication rate is because sicker patients are more likely to have procedural supervision by attendings, attending physicians may be called to supervise when a CVC placement is not going as planned, or attendings may supervise more inexperienced operators. Reasons behind negative trainee attitudes surrounding supervision are unclear and literature on this topic is limited. This is an area that warrants further exploration in future studies.

Another unexpected finding is trainee practices regarding unsuccessful CVC placement; most trainees do not document failed procedures or order follow-up CXRs after unsuccessful CVC attempts. Given the higher risk of complications after unsuccessful CVCs, it is paramount that all physicians are trained to order postprocedure CXR to rule out pneumothorax or hemothorax. Furthermore, documentation of failed procedures is important for medical accuracy, transparency, and also hospital billing. It is unknown if these practices surrounding unsuccessful CVCs are institution-specific or more widespread. As far as we know, this is the first time that trainee practices regarding failed CVC placement have been published. Interestingly, while many current guidelines call attention to prevention, recognition, and management of central line-associated mechanical complications, specific recommendations about postprocedure behavior after failed CVC placement are not published.^9-11 We feel it is critical that institutions reflect on their own practices, especially given that unsuccessful CVCs are shown to be correlated with a significant increase in complication rate. At our own institution, we have initiated an educational component of central line training for medicine trainees specifically addressing failed central line attempts.

This study has several limitations, including a retrospective study design at a single institution. There was a low overall number of complications, which reduced our ability to detect risk factors for complications and did not allow us to perform multivariable adjustment. Other limitations are that only documented CVC attempts were recorded and only those that met our search criteria. Lastly, not all notes contain information such as the number of attempts or peer supervision. Furthermore, the definition of CVC “attempt” is left to the operator’s discretion.

In conclusion, we observed a modern CVC mechanical complication rate of 1.9%. While the complication rate is similar to previous studies, there appear to be lower rates of pneumothorax and higher rates of bleeding complications. We also identified a deficit in trainee education regarding unsuccessful CVC placement; this is a novel finding and requires further investigation at other centers.

 

Disclosure: The authors have no conflicts of interest to report.

Central venous catheter (CVC) placement is commonly performed in emergency and critical care settings for parenteral access, central monitoring, and hemodialysis. Although potentially lifesaving CVC insertion is associated with immediate risks including injury to nerves, vessels, and lungs.^1-3 These “insertion-related complications” are of particular interest for several reasons. First, the frequency of such complications varies widely, with published rates between 1.4% and 33.2%.^2-7 Reasons for such variation include differences in study definitions of complications (eg, pneumothorax and tip position),^2,5 setting of CVC placement (eg, intensive care unit [ICU] vs emergency room), timing of placement (eg, elective vs emergent), differences in technique, and type of operator (eg, experienced vs learner). Thus, the precise incidence of such events in modern-day training settings with use of ultrasound guidance remains uncertain. Second, mechanical complications might be preventable with adequate training and supervision. Indeed, studies using simulation-based mastery techniques have demonstrated a reduction in rates of complications following intensive training.⁸ Finally, understanding risk factors associated with insertion complications might inform preventative strategies and improve patient safety.^9-11

Few studies to date have examined trainees’ perceptions on CVC training, experience, supervision, and ability to recognize and prevent mechanical complications. While research investigating effects of simulation training has accumulated, most focus on successful completion of the procedure or individual procedural steps with little emphasis on operator perceptions.^12-14 In addition, while multiple studies have shown that unsuccessful line attempts are a risk factor for CVC complications,^3,4,7,15 there is very little known about trainee behavior and perceptions regarding unsuccessful line placement. CVC simulation trainings often assume successful completion of the procedure and do not address the crucial postprocedure steps that should be undertaken if a procedure is unsuccessful. For these reasons, we developed a survey to specifically examine trainee experience with CVC placement, supervision, postprocedural behavior, and attitudes regarding unsuccessful line placement.

Therefore, we designed a study with 2 specific goals: The first is to perform a contemporary analysis of CVC mechanical complication rate at an academic teaching institution and identify potential risk factors associated with these complications. Second, we sought to determine trainee perceptions regarding CVC complication experience, prevention, procedural supervision, and perceptions surrounding unsuccessful line placement.

METHODS

Design and Setting

We conducted a single-center retrospective review of nontunneled acute CVC procedures between June 1, 2014, and May 1, 2015, at the University of Michigan Health System (UMHS). UMHS is a tertiary care referral center with over 900 inpatient beds, including 99 ICU beds.

All residents in internal medicine, surgery, anesthesia, and emergency medicine receive mandatory education in CVC placement that includes an online training module and simulation-based training with competency assessment. Use of real-time ultrasound guidance is considered the standard of care for CVC placement.

Data Collection

Inpatient procedure notes were electronically searched for terms indicating CVC placement. This was performed by using our hospital’s Data Office for Clinical and Translational Research using the Electronic Medical Record Search Engine tool. Please see the supplemental materials for the full list of search terms. We electronically extracted data, including date of procedure, gender, and most recent body mass index (BMI), within 1 year prior to note. Acute Physiology and Chronic Health Evaluation III (APACHE III) data are tracked for all patients on admission to ICU; this was collected when available. Charts were then manually reviewed to collect additional data, including international normalized ratio (INR), platelet count, lactate level on the day of CVC placement, anticoagulant use (actively prescribed coumadin, therapeutic enoxaparin, therapeutic unfractionated heparin, or direct oral anticoagulant), ventilator or noninvasive positive pressure ventilation (NIPPV) at time of CVC placement, and vasopressor requirement within 24 hours of CVC placement. The procedure note was reviewed to gather information about site of CVC placement, size and type of catheter, number of attempts, procedural success, training level of the operator, and attending presence. Small bore CVCs were defined as 7 French (Fr) or lower. Large bore CVCs were defined as >7 Fr; this includes dialysis catheters, Cordis catheters (Cordis, Fremont, CA), and cooling catheters. The times of the procedure note and postprocedure chest x-ray (CXR) were recorded, including whether the CVC was placed on a weekend (Friday 7 pm to Sunday at midnight) or weekday.

 

Primary Outcome

The primary outcome was the rate of severe mechanical complications related to CVC placement. Similar to prior studies,² we defined severe mechanical complications as arterial placement of dilator or catheter, hemothorax, pneumothorax, cerebral ischemia, patient death (related to procedure), significant hematoma, or vascular injury (defined as complication requiring expert consultation or blood product transfusion). We did not require a lower limit on blood transfusion. We considered pneumothorax a complication regardless of whether chest tube intervention was performed, as pneumothorax subjects the patient to additional tests (eg, serial CXRs) and sometimes symptoms (shortness of breath, pain, anxiety) regardless of whether or not a chest tube was required. Complications were confirmed by a direct review of procedure notes, progress notes, discharge summaries, and imaging studies.

Trainee Survey

A survey was electronically disseminated to all internal medicine and medicine-pediatric residents to inquire about CVC experiences, including time spent in the medical ICU, number of CVCs performed, postprocedure behavior for both failed and successful CVCs, and supervision experience and attitudes. Please see supplemental materials for full survey contents.

Statistical Methods

Descriptive statistics (percentage) were used to summarize data. Continuous and categorical variables were compared using Student t tests and chi-square tests, respectively. All analyses were performed using SAS 9.3 (SAS Institute, Cary, NC).

Ethical and Regulatory Oversight

The study was deemed exempt by the University of Michigan Institutional Review Board (HUM00100549) as data collection was part of a quality improvement effort.

RESULTS

Demographics and Characteristics of Device Insertion

Between June 1, 2014, and May 1, 2015, 730 CVC procedure notes were reviewed (Table 1). The mean age of the study population was 58.9 years, and 41.6% (n = 304) were female. BMI data were available in 400 patients without complications and 5 patients with complications; the average BMI was 31.5 kg/m². The APACHE III score was available for 442 patients without complications and 10 patients with complications; the average score was 86 (range 19-200). Most of the CVCs placed (n= 504, 69%) were small bore (<7 Fr). The majority of catheters were placed in the internal jugular (IJ) position (n = 525, 71.9%), followed by femoral (n = 144, 19.7%), subclavian (N = 57, 7.8%), and undocumented (n = 4, 0.6%). Ninety-six percent (n = 699) of CVCs were successfully placed. Seventy-six percent (n = 558) of procedure notes included documentation of the number of CVC attempts; of these, 85% documented 2 or fewer attempts. The majority of CVCs were placed by residents (n = 537, 73.9%), followed by fellows (N = 127, 17.5%) and attendings (n = 27, 3.7%). Attending supervision for all or key portions of CVC placement occurred 34.7% (n = 244) of the time overall and was lower for internal medicine trainees (n = 98/463, 21.2%) compared with surgical trainees (n = 73/127, 57.4%) or emergency medicine trainees (n = 62/96, 64.6%; P < 0.001). All successful IJ and subclavian CVCs except for 2 insertions (0.3%) had a postprocedure CXR. A minority of notes documented pressure transduction (4.5%) or blood gas analysis (0.2%) to confirm venous placement.

Lab data, information on utilization of anticoagulation, vasopressors, ventilation, and information on the use of transduction and blood gas data were collected for the first 410 uncomplicated patients and all patients that experienced complications (Table 2). The mean INR was 1.5 (range 0.9-9.7), mean platelets 180 K/uL (range 9-816 K/uL), and mean lactate 2.9 mmol/L (range 0.4-16 mmol/L). Twenty-one percent (n = 86) of patients were systemically anticoagulated at the time of CVC placement, 59% (n = 251) received vasopressors within 24 hours of CVC placement, and 63% (n = 265) were mechanically ventilated (Table 2).

Mechanical Complications

The mechanical complications identified included pneumothorax (n = 5), bleeding requiring transfusion (n = 3), vascular injury requiring expert consultation or intervention (n = 3), stroke (n = 1), and death (n = 2). Vascular injuries included 1 neck hematoma with superinfection requiring antibiotics, 1 neck hematoma requiring otolaryngology and vascular surgery consultation, and 1 venous dissection of IJ vein requiring vascular surgery consultation. None of these cases required operative intervention. The stroke was caused by inadvertent CVC placement into the carotid artery. One patient experienced tension pneumothorax and died due to this complication; this death occurred after 3 failed left subclavian CVC attempts and an ultimately successful CVC placement into left IJ vein. Another death occurred immediately following unsuccessful Cordis placement. As no autopsy was performed, it is impossible to know if the cause of death was the line placement. However, it would be prudent to consider this as a CVC complication given the temporal relationship to line placement. Thus, the total number of patients who experienced severe mechanical complications was 14 out of 730 (1.92%).

 

Risk Factors for Mechanical Complications

Certain patient factors were more commonly associated with complications. For example, BMI was significantly lower in the group that experienced complications vs those that did not (25.7 vs 31.0 kg/m², P = 0.001). No other associations between demographic factors, including age (61.4 years vs 58.9 years, P = 0.57) or sex (57.1% male vs 41.3% female, P = 0.24), or admission APACHE III score (96 vs 86, P = 0.397) were noted. The mean INR, platelets, and lactate did not differ between the 2 groups. There was no difference between the use of vasopressors. Ventilator use (including endotracheal tube or NIPPV) was found to be significantly higher in the group that experienced mechanical complications (78.5% vs 65.9%, P = 0.001). Anticoagulation use was also associated with mechanical complications (28.6% vs 20.6%, P = 0.05); 3 patients on anticoagulation experienced significant hematomas. Mechanical complications were more common with subclavian location (21.4% vs 7.8%, P = 0.001); in all 3 cases involving subclavian CVC placement, the complication experienced was pneumothorax. The number of attempts significantly differed between the 2 groups, with an average of 1.5 attempts in the group without complications and 2.2 attempts in the group that experienced complications (P = 0.02). Additionally, rates of successful placement were lower among patients who experienced complications (78.6% vs 95.7%, P = 0.001).

With respect to operator characteristics, no significant difference between the levels of training was noted among those who experienced complications vs those who did not. Attending supervision was more frequent for the group that experienced complications (61.5% vs 34.2%, P = 0.04). There was no significant difference in complication rate according to the first vs the second half of the academic year (0.4% vs 0.3% per month, P = 0.30) or CVC placement during the day vs night (1.9% vs 2.0%, P = 0.97). A trend toward more complications in CVCs placed over the weekend compared to a weekday was observed (2.80% vs 1.23%, P = 0.125).

Unsuccessful CVCs

There were 30 documented unsuccessful CVC procedures, representing 4.1% of all procedures. Of these, 3 procedures had complications; these included 2 pneumothoraxes (1 leading to death) and 1 unexplained death. Twenty-four of the unsuccessful CVC attempts were in either the subclavian or IJ location; of these, 5 (21%) did not have a postprocedure CXR obtained.

Survey Results

The survey was completed by 103 out of 166 internal medicine residents (62% response rate). Of these, 55% (n = 57) reported having performed 5 or more CVCs, and 14% (n = 14) had performed more than 15 CVCs.

All respondents who had performed at least 1 CVC (n = 80) were asked about their perceptions regarding attending supervision. Eighty-one percent (n = 65/80) responded that they have never been directly supervised by an attending during CVC placement, while 16% (n = 13/80) reported being supervised less than 25% of the time. Most (n = 53/75, 71%) did not feel that attending supervision affected their performance, while 21% (n = 16/75) felt it affected performance negatively, and only 8% (n = 6/75) stated it affected performance positively. Nineteen percent (n = 15/80) indicated that they prefer more supervision by attendings, while 35% (n = 28/80) did not wish for more attending supervision, and 46% (n = 37/80) were indifferent.

All respondents who had performed at least 1 CVC were asked about postprocedure protocols. The vast majority (n = 77/80, 95%) reported documenting a postprocedure note more than 75% of the time after a successful procedure; in contrast, only 38% (n = 30/80) of those who had failed a CVC placement reported documenting a procedure note more than 75% of the time (Figure 1). Only 35% (n = 21/60) of respondents reported routinely (100% of the time) ordering a CXR after a failed chest CVC attempt (Figure 2), and 47% (n = 28/60) only ordered a CXR if they were concerned there was a complication. Most (69%, n = 55/80) felt it was important to order a CXR after a failed chest CVC placement, while 6% (n = 5/80) did not feel it was important, and 25% (n = 20/80) did not know or were indifferent.

DISCUSSION

We performed a contemporary analysis of CVC placement at an academic tertiary care center and observed a rate of severe mechanical complications of 1.9%. This rate is within previously described acceptable thresholds.¹⁶ Our study adds to the literature by identifying several important risk factors for development of mechanical complications. We confirm many risk factors that have been noted historically, such as subclavian line location,^2,3 attending supervision,³ low BMI,⁴ number of CVC attempts, and unsuccessful CVC placement.^3,4,7,15 We identified several unique risk factors, including systemic anticoagulation as well as ventilator use. Lastly, we identified unexpected deficits in trainee knowledge surrounding management of failed CVCs and negative attitudes regarding attending supervision.

 

Most existing literature evaluated risk factors for CVC complication prior to routine ultrasound use;^3-5,7,15 surprisingly, it appears that severe mechanical complications do not differ dramatically in the real-time ultrasound era. Eisen et al.³ prospectively studied CVC placement at an academic medical center and found a severe mechanical complication rate (as defined in our paper) of 1.9% due to pneumothorax (1.3%), hemothorax (0.3%), and death (0.3%).We would expect the number of complications to decrease in the postultrasound era, and indeed it appears that pneumothoraces have decreased likely due to ultrasound guidance and decrease in subclavian location. However, in contrast, rates of significant hematomas and bleeding are higher in our study. Although we are unable to state why this may be the case, increasing use of anticoagulation in the general population might explain this finding.¹⁷ For instance, of the 6 patients who experienced hematomas or vascular injuries in our study, 3 were on anticoagulation at the time of CVC placement.

Interestingly, time of academic year of CVC placement and level of training were not correlated with an increased risk of complications, nor was time of day of CVC placement. In contrast, Merrer et al.showed that CVC insertion during nighttime was significantly associated with increased mechanical complications (odds ratio 2.06, 95% confidence interval, 1.04-4.08;,P = 0.03).⁵ This difference may be attributable to the fact that most of our ICUs now have a night float system rather than a more traditional 24-hour call model; therefore, trainees are less likely to be sleep deprived during CVC placement at night.

Severity of illness did not appear to significantly affect mechanical complication rates based on similar APACHE scores between the 2 groups. In addition, other indicators of illness severity (vasopressor use or lactate level) did not suggest that sicker patients may be more likely to experience mechanical complications than others. One could conjecture that perhaps sicker patients were more likely to have lines placed by more experienced trainees, although the present study design does not allow us to answer this question. Interestingly, ventilator use was associated with higher rates of complications. We cannot say definitively why this was the case; however, 1 contributing factor may be the physical constraints of placing the CVC around ventilator tubing.

Several unexpected findings surrounding attending supervision were noted: first, attending supervision appears to be significantly associated with increased complication rate, and second, trainees have negative perceptions regarding attending supervision. Eisen et al.showed a similar association between attending supervision and complication rate.³ It is possible that the increased complication rate is because sicker patients are more likely to have procedural supervision by attendings, attending physicians may be called to supervise when a CVC placement is not going as planned, or attendings may supervise more inexperienced operators. Reasons behind negative trainee attitudes surrounding supervision are unclear and literature on this topic is limited. This is an area that warrants further exploration in future studies.

Another unexpected finding is trainee practices regarding unsuccessful CVC placement; most trainees do not document failed procedures or order follow-up CXRs after unsuccessful CVC attempts. Given the higher risk of complications after unsuccessful CVCs, it is paramount that all physicians are trained to order postprocedure CXR to rule out pneumothorax or hemothorax. Furthermore, documentation of failed procedures is important for medical accuracy, transparency, and also hospital billing. It is unknown if these practices surrounding unsuccessful CVCs are institution-specific or more widespread. As far as we know, this is the first time that trainee practices regarding failed CVC placement have been published. Interestingly, while many current guidelines call attention to prevention, recognition, and management of central line-associated mechanical complications, specific recommendations about postprocedure behavior after failed CVC placement are not published.^9-11 We feel it is critical that institutions reflect on their own practices, especially given that unsuccessful CVCs are shown to be correlated with a significant increase in complication rate. At our own institution, we have initiated an educational component of central line training for medicine trainees specifically addressing failed central line attempts.

This study has several limitations, including a retrospective study design at a single institution. There was a low overall number of complications, which reduced our ability to detect risk factors for complications and did not allow us to perform multivariable adjustment. Other limitations are that only documented CVC attempts were recorded and only those that met our search criteria. Lastly, not all notes contain information such as the number of attempts or peer supervision. Furthermore, the definition of CVC “attempt” is left to the operator’s discretion.

In conclusion, we observed a modern CVC mechanical complication rate of 1.9%. While the complication rate is similar to previous studies, there appear to be lower rates of pneumothorax and higher rates of bleeding complications. We also identified a deficit in trainee education regarding unsuccessful CVC placement; this is a novel finding and requires further investigation at other centers.

 

Disclosure: The authors have no conflicts of interest to report.

Central venous catheter (CVC) placement is commonly performed in emergency and critical care settings for parenteral access, central monitoring, and hemodialysis. Although potentially lifesaving CVC insertion is associated with immediate risks including injury to nerves, vessels, and lungs.^1-3 These “insertion-related complications” are of particular interest for several reasons. First, the frequency of such complications varies widely, with published rates between 1.4% and 33.2%.^2-7 Reasons for such variation include differences in study definitions of complications (eg, pneumothorax and tip position),^2,5 setting of CVC placement (eg, intensive care unit [ICU] vs emergency room), timing of placement (eg, elective vs emergent), differences in technique, and type of operator (eg, experienced vs learner). Thus, the precise incidence of such events in modern-day training settings with use of ultrasound guidance remains uncertain. Second, mechanical complications might be preventable with adequate training and supervision. Indeed, studies using simulation-based mastery techniques have demonstrated a reduction in rates of complications following intensive training.⁸ Finally, understanding risk factors associated with insertion complications might inform preventative strategies and improve patient safety.^9-11

Few studies to date have examined trainees’ perceptions on CVC training, experience, supervision, and ability to recognize and prevent mechanical complications. While research investigating effects of simulation training has accumulated, most focus on successful completion of the procedure or individual procedural steps with little emphasis on operator perceptions.^12-14 In addition, while multiple studies have shown that unsuccessful line attempts are a risk factor for CVC complications,^3,4,7,15 there is very little known about trainee behavior and perceptions regarding unsuccessful line placement. CVC simulation trainings often assume successful completion of the procedure and do not address the crucial postprocedure steps that should be undertaken if a procedure is unsuccessful. For these reasons, we developed a survey to specifically examine trainee experience with CVC placement, supervision, postprocedural behavior, and attitudes regarding unsuccessful line placement.

Therefore, we designed a study with 2 specific goals: The first is to perform a contemporary analysis of CVC mechanical complication rate at an academic teaching institution and identify potential risk factors associated with these complications. Second, we sought to determine trainee perceptions regarding CVC complication experience, prevention, procedural supervision, and perceptions surrounding unsuccessful line placement.

METHODS

Design and Setting

We conducted a single-center retrospective review of nontunneled acute CVC procedures between June 1, 2014, and May 1, 2015, at the University of Michigan Health System (UMHS). UMHS is a tertiary care referral center with over 900 inpatient beds, including 99 ICU beds.

All residents in internal medicine, surgery, anesthesia, and emergency medicine receive mandatory education in CVC placement that includes an online training module and simulation-based training with competency assessment. Use of real-time ultrasound guidance is considered the standard of care for CVC placement.

Data Collection

Inpatient procedure notes were electronically searched for terms indicating CVC placement. This was performed by using our hospital’s Data Office for Clinical and Translational Research using the Electronic Medical Record Search Engine tool. Please see the supplemental materials for the full list of search terms. We electronically extracted data, including date of procedure, gender, and most recent body mass index (BMI), within 1 year prior to note. Acute Physiology and Chronic Health Evaluation III (APACHE III) data are tracked for all patients on admission to ICU; this was collected when available. Charts were then manually reviewed to collect additional data, including international normalized ratio (INR), platelet count, lactate level on the day of CVC placement, anticoagulant use (actively prescribed coumadin, therapeutic enoxaparin, therapeutic unfractionated heparin, or direct oral anticoagulant), ventilator or noninvasive positive pressure ventilation (NIPPV) at time of CVC placement, and vasopressor requirement within 24 hours of CVC placement. The procedure note was reviewed to gather information about site of CVC placement, size and type of catheter, number of attempts, procedural success, training level of the operator, and attending presence. Small bore CVCs were defined as 7 French (Fr) or lower. Large bore CVCs were defined as >7 Fr; this includes dialysis catheters, Cordis catheters (Cordis, Fremont, CA), and cooling catheters. The times of the procedure note and postprocedure chest x-ray (CXR) were recorded, including whether the CVC was placed on a weekend (Friday 7 pm to Sunday at midnight) or weekday.

 

Primary Outcome

The primary outcome was the rate of severe mechanical complications related to CVC placement. Similar to prior studies,² we defined severe mechanical complications as arterial placement of dilator or catheter, hemothorax, pneumothorax, cerebral ischemia, patient death (related to procedure), significant hematoma, or vascular injury (defined as complication requiring expert consultation or blood product transfusion). We did not require a lower limit on blood transfusion. We considered pneumothorax a complication regardless of whether chest tube intervention was performed, as pneumothorax subjects the patient to additional tests (eg, serial CXRs) and sometimes symptoms (shortness of breath, pain, anxiety) regardless of whether or not a chest tube was required. Complications were confirmed by a direct review of procedure notes, progress notes, discharge summaries, and imaging studies.

Trainee Survey

A survey was electronically disseminated to all internal medicine and medicine-pediatric residents to inquire about CVC experiences, including time spent in the medical ICU, number of CVCs performed, postprocedure behavior for both failed and successful CVCs, and supervision experience and attitudes. Please see supplemental materials for full survey contents.

Statistical Methods

Descriptive statistics (percentage) were used to summarize data. Continuous and categorical variables were compared using Student t tests and chi-square tests, respectively. All analyses were performed using SAS 9.3 (SAS Institute, Cary, NC).

Ethical and Regulatory Oversight

The study was deemed exempt by the University of Michigan Institutional Review Board (HUM00100549) as data collection was part of a quality improvement effort.

RESULTS

Demographics and Characteristics of Device Insertion

Between June 1, 2014, and May 1, 2015, 730 CVC procedure notes were reviewed (Table 1). The mean age of the study population was 58.9 years, and 41.6% (n = 304) were female. BMI data were available in 400 patients without complications and 5 patients with complications; the average BMI was 31.5 kg/m². The APACHE III score was available for 442 patients without complications and 10 patients with complications; the average score was 86 (range 19-200). Most of the CVCs placed (n= 504, 69%) were small bore (<7 Fr). The majority of catheters were placed in the internal jugular (IJ) position (n = 525, 71.9%), followed by femoral (n = 144, 19.7%), subclavian (N = 57, 7.8%), and undocumented (n = 4, 0.6%). Ninety-six percent (n = 699) of CVCs were successfully placed. Seventy-six percent (n = 558) of procedure notes included documentation of the number of CVC attempts; of these, 85% documented 2 or fewer attempts. The majority of CVCs were placed by residents (n = 537, 73.9%), followed by fellows (N = 127, 17.5%) and attendings (n = 27, 3.7%). Attending supervision for all or key portions of CVC placement occurred 34.7% (n = 244) of the time overall and was lower for internal medicine trainees (n = 98/463, 21.2%) compared with surgical trainees (n = 73/127, 57.4%) or emergency medicine trainees (n = 62/96, 64.6%; P < 0.001). All successful IJ and subclavian CVCs except for 2 insertions (0.3%) had a postprocedure CXR. A minority of notes documented pressure transduction (4.5%) or blood gas analysis (0.2%) to confirm venous placement.

Lab data, information on utilization of anticoagulation, vasopressors, ventilation, and information on the use of transduction and blood gas data were collected for the first 410 uncomplicated patients and all patients that experienced complications (Table 2). The mean INR was 1.5 (range 0.9-9.7), mean platelets 180 K/uL (range 9-816 K/uL), and mean lactate 2.9 mmol/L (range 0.4-16 mmol/L). Twenty-one percent (n = 86) of patients were systemically anticoagulated at the time of CVC placement, 59% (n = 251) received vasopressors within 24 hours of CVC placement, and 63% (n = 265) were mechanically ventilated (Table 2).

Mechanical Complications

The mechanical complications identified included pneumothorax (n = 5), bleeding requiring transfusion (n = 3), vascular injury requiring expert consultation or intervention (n = 3), stroke (n = 1), and death (n = 2). Vascular injuries included 1 neck hematoma with superinfection requiring antibiotics, 1 neck hematoma requiring otolaryngology and vascular surgery consultation, and 1 venous dissection of IJ vein requiring vascular surgery consultation. None of these cases required operative intervention. The stroke was caused by inadvertent CVC placement into the carotid artery. One patient experienced tension pneumothorax and died due to this complication; this death occurred after 3 failed left subclavian CVC attempts and an ultimately successful CVC placement into left IJ vein. Another death occurred immediately following unsuccessful Cordis placement. As no autopsy was performed, it is impossible to know if the cause of death was the line placement. However, it would be prudent to consider this as a CVC complication given the temporal relationship to line placement. Thus, the total number of patients who experienced severe mechanical complications was 14 out of 730 (1.92%).

 

Risk Factors for Mechanical Complications

Certain patient factors were more commonly associated with complications. For example, BMI was significantly lower in the group that experienced complications vs those that did not (25.7 vs 31.0 kg/m², P = 0.001). No other associations between demographic factors, including age (61.4 years vs 58.9 years, P = 0.57) or sex (57.1% male vs 41.3% female, P = 0.24), or admission APACHE III score (96 vs 86, P = 0.397) were noted. The mean INR, platelets, and lactate did not differ between the 2 groups. There was no difference between the use of vasopressors. Ventilator use (including endotracheal tube or NIPPV) was found to be significantly higher in the group that experienced mechanical complications (78.5% vs 65.9%, P = 0.001). Anticoagulation use was also associated with mechanical complications (28.6% vs 20.6%, P = 0.05); 3 patients on anticoagulation experienced significant hematomas. Mechanical complications were more common with subclavian location (21.4% vs 7.8%, P = 0.001); in all 3 cases involving subclavian CVC placement, the complication experienced was pneumothorax. The number of attempts significantly differed between the 2 groups, with an average of 1.5 attempts in the group without complications and 2.2 attempts in the group that experienced complications (P = 0.02). Additionally, rates of successful placement were lower among patients who experienced complications (78.6% vs 95.7%, P = 0.001).

With respect to operator characteristics, no significant difference between the levels of training was noted among those who experienced complications vs those who did not. Attending supervision was more frequent for the group that experienced complications (61.5% vs 34.2%, P = 0.04). There was no significant difference in complication rate according to the first vs the second half of the academic year (0.4% vs 0.3% per month, P = 0.30) or CVC placement during the day vs night (1.9% vs 2.0%, P = 0.97). A trend toward more complications in CVCs placed over the weekend compared to a weekday was observed (2.80% vs 1.23%, P = 0.125).

Unsuccessful CVCs

There were 30 documented unsuccessful CVC procedures, representing 4.1% of all procedures. Of these, 3 procedures had complications; these included 2 pneumothoraxes (1 leading to death) and 1 unexplained death. Twenty-four of the unsuccessful CVC attempts were in either the subclavian or IJ location; of these, 5 (21%) did not have a postprocedure CXR obtained.

Survey Results

The survey was completed by 103 out of 166 internal medicine residents (62% response rate). Of these, 55% (n = 57) reported having performed 5 or more CVCs, and 14% (n = 14) had performed more than 15 CVCs.

All respondents who had performed at least 1 CVC (n = 80) were asked about their perceptions regarding attending supervision. Eighty-one percent (n = 65/80) responded that they have never been directly supervised by an attending during CVC placement, while 16% (n = 13/80) reported being supervised less than 25% of the time. Most (n = 53/75, 71%) did not feel that attending supervision affected their performance, while 21% (n = 16/75) felt it affected performance negatively, and only 8% (n = 6/75) stated it affected performance positively. Nineteen percent (n = 15/80) indicated that they prefer more supervision by attendings, while 35% (n = 28/80) did not wish for more attending supervision, and 46% (n = 37/80) were indifferent.

All respondents who had performed at least 1 CVC were asked about postprocedure protocols. The vast majority (n = 77/80, 95%) reported documenting a postprocedure note more than 75% of the time after a successful procedure; in contrast, only 38% (n = 30/80) of those who had failed a CVC placement reported documenting a procedure note more than 75% of the time (Figure 1). Only 35% (n = 21/60) of respondents reported routinely (100% of the time) ordering a CXR after a failed chest CVC attempt (Figure 2), and 47% (n = 28/60) only ordered a CXR if they were concerned there was a complication. Most (69%, n = 55/80) felt it was important to order a CXR after a failed chest CVC placement, while 6% (n = 5/80) did not feel it was important, and 25% (n = 20/80) did not know or were indifferent.

DISCUSSION

We performed a contemporary analysis of CVC placement at an academic tertiary care center and observed a rate of severe mechanical complications of 1.9%. This rate is within previously described acceptable thresholds.¹⁶ Our study adds to the literature by identifying several important risk factors for development of mechanical complications. We confirm many risk factors that have been noted historically, such as subclavian line location,^2,3 attending supervision,³ low BMI,⁴ number of CVC attempts, and unsuccessful CVC placement.^3,4,7,15 We identified several unique risk factors, including systemic anticoagulation as well as ventilator use. Lastly, we identified unexpected deficits in trainee knowledge surrounding management of failed CVCs and negative attitudes regarding attending supervision.

 

Most existing literature evaluated risk factors for CVC complication prior to routine ultrasound use;^3-5,7,15 surprisingly, it appears that severe mechanical complications do not differ dramatically in the real-time ultrasound era. Eisen et al.³ prospectively studied CVC placement at an academic medical center and found a severe mechanical complication rate (as defined in our paper) of 1.9% due to pneumothorax (1.3%), hemothorax (0.3%), and death (0.3%).We would expect the number of complications to decrease in the postultrasound era, and indeed it appears that pneumothoraces have decreased likely due to ultrasound guidance and decrease in subclavian location. However, in contrast, rates of significant hematomas and bleeding are higher in our study. Although we are unable to state why this may be the case, increasing use of anticoagulation in the general population might explain this finding.¹⁷ For instance, of the 6 patients who experienced hematomas or vascular injuries in our study, 3 were on anticoagulation at the time of CVC placement.

Interestingly, time of academic year of CVC placement and level of training were not correlated with an increased risk of complications, nor was time of day of CVC placement. In contrast, Merrer et al.showed that CVC insertion during nighttime was significantly associated with increased mechanical complications (odds ratio 2.06, 95% confidence interval, 1.04-4.08;,P = 0.03).⁵ This difference may be attributable to the fact that most of our ICUs now have a night float system rather than a more traditional 24-hour call model; therefore, trainees are less likely to be sleep deprived during CVC placement at night.

Severity of illness did not appear to significantly affect mechanical complication rates based on similar APACHE scores between the 2 groups. In addition, other indicators of illness severity (vasopressor use or lactate level) did not suggest that sicker patients may be more likely to experience mechanical complications than others. One could conjecture that perhaps sicker patients were more likely to have lines placed by more experienced trainees, although the present study design does not allow us to answer this question. Interestingly, ventilator use was associated with higher rates of complications. We cannot say definitively why this was the case; however, 1 contributing factor may be the physical constraints of placing the CVC around ventilator tubing.

Several unexpected findings surrounding attending supervision were noted: first, attending supervision appears to be significantly associated with increased complication rate, and second, trainees have negative perceptions regarding attending supervision. Eisen et al.showed a similar association between attending supervision and complication rate.³ It is possible that the increased complication rate is because sicker patients are more likely to have procedural supervision by attendings, attending physicians may be called to supervise when a CVC placement is not going as planned, or attendings may supervise more inexperienced operators. Reasons behind negative trainee attitudes surrounding supervision are unclear and literature on this topic is limited. This is an area that warrants further exploration in future studies.

Another unexpected finding is trainee practices regarding unsuccessful CVC placement; most trainees do not document failed procedures or order follow-up CXRs after unsuccessful CVC attempts. Given the higher risk of complications after unsuccessful CVCs, it is paramount that all physicians are trained to order postprocedure CXR to rule out pneumothorax or hemothorax. Furthermore, documentation of failed procedures is important for medical accuracy, transparency, and also hospital billing. It is unknown if these practices surrounding unsuccessful CVCs are institution-specific or more widespread. As far as we know, this is the first time that trainee practices regarding failed CVC placement have been published. Interestingly, while many current guidelines call attention to prevention, recognition, and management of central line-associated mechanical complications, specific recommendations about postprocedure behavior after failed CVC placement are not published.^9-11 We feel it is critical that institutions reflect on their own practices, especially given that unsuccessful CVCs are shown to be correlated with a significant increase in complication rate. At our own institution, we have initiated an educational component of central line training for medicine trainees specifically addressing failed central line attempts.

This study has several limitations, including a retrospective study design at a single institution. There was a low overall number of complications, which reduced our ability to detect risk factors for complications and did not allow us to perform multivariable adjustment. Other limitations are that only documented CVC attempts were recorded and only those that met our search criteria. Lastly, not all notes contain information such as the number of attempts or peer supervision. Furthermore, the definition of CVC “attempt” is left to the operator’s discretion.

In conclusion, we observed a modern CVC mechanical complication rate of 1.9%. While the complication rate is similar to previous studies, there appear to be lower rates of pneumothorax and higher rates of bleeding complications. We also identified a deficit in trainee education regarding unsuccessful CVC placement; this is a novel finding and requires further investigation at other centers.

 

Disclosure: The authors have no conflicts of interest to report.

Skilled nursing facilities (SNFs) play a crucial role in the hospital readmission process.Approximately 1 in 4 Medicare beneficiaries discharged from an acute care hospital is admitted to a SNF instead of returning directly home. Of these patients, 1 in 4 will be readmitted within 30 days,¹ a rate significantly higher than the readmission rate of the inpatient population as a whole.² The 2014 Protecting Access to Medicare Act created a value-based purchasing program that will use quality measures to steer funds to, or away from, individual SNFs. When the program takes effect in 2018, the Centers for Medicare & Medicaid Services will use SNFs’ 30-day all-cause readmission rate to determine which SNFs receive payments and which receive penalties.³ The Affordable Care Act, passed in 2010, has also established penalties for hospitals with higher than expected readmission rates for Medicare patients.⁴

Despite this intensifying regulatory focus, relatively little is known about the factors that drive readmissions from SNFs. A prospective review of data from SNFs in 4 states has shown that SNFs staffed by nurse practitioners or physician assistants and those equipped to provide intravenous therapy were less likely to transfer patients to the hospital for ambulatory care-sensitive diagnoses.⁵ Qualitative studies have provided useful insight into the causes of SNF-to-hospital transfers but have not focused on 30-day readmissions.^6,7 A single survey-based study has examined the causes of SNF-to-hospital readmissions.⁸ However, survey-based methodologies have limited ability to capture the complex perspectives of SNF clinicians, who play a critical role in determining which SNF patients require evaluation or treatment in an acute care setting.

To address this gap in knowledge about factors contributing to SNF readmissions, we conducted a qualitative study examining SNF clinicians’ perspectives on patients readmitted to the hospital within 30 days of discharge. We used a structured interview tool to explore the root causes of readmission with frontline SNF staff, with the goal of using this knowledge to inform future hospital quality improvement (QI) efforts.

METHODS

Case Identification

Hospital data-tracking software (Allscripts) was used to identify patients who experienced a 30-day, unplanned readmission from SNFs to an academic medical center. We restricted our search to patients whose index admission and readmission were to the medical center’s inpatient general medicine service. A study team member (BWC) monitored the dataset on a weekly basis and contacted SNF clinicians by e-mail and telephone to arrange interviews at times of mutual convenience. To mitigate against recall bias, interviews were conducted within 30 days of the readmission in question. A total of 32 cases were identified. No SNF clinicians refused a request for interview. For 8 of these cases, it was not possible to find a time of mutual convenience within the specified 30-day window. The remaining 24 cases involved patients from 15 SNFs across Connecticut. Interviews were conducted from August 2015 to November 2015.

The project was reviewed by our institution’s Human Investigation Committee and was exempted from Institutional Review Board review.

Study Participants

Interviews were conducted on-site at SNFs with groups of 1 to 4 SNF clinicians and administrators. SNF participants were informed of interviewer credentials and the study’s QI goals prior to participation. Participation was voluntary and did not affect the clinician’s relationship with the hospital or the SNF. Participants were not paid.

DATA COLLECTION

Interventions to Reduce Acute Care Transfers (INTERACT) is a QI program that includes training for clinicians, communication tools, and advance care planning tools.⁹ INTERACT is currently used in 138 Connecticut SNFs as part of a statewide QI effort funded by the Connecticut State Department of Public Health. In prospective QI studies,^10,11 implementation of INTERACT has been associated with decreased transfers from SNFs to acute care hospitals. The INTERACT Quality Improvement Tool, one part of the INTERACT bundle of interventions, is a 26-item questionnaire used to identify root causes of transfers from SNFs to acute care hospitals. It includes both checklists and open-ended questions about patient factors, SNF procedures, and SNF clinical decision-making.

We used the INTERACT QI Tool¹² to conduct structured interviews with nurses and administrators at SNFs. Interviewers used a hard copy of the tool to maintain field notes, and all parts of the questionnaire were completed in each interview. Although the questionnaire elicits baseline demographic and medical information, such as the patient’s age and vital signs prior to readmission, the majority of each interview was dedicated to discussion of the open-ended questions in Table 1. Upon completion of the INTERACT QI Tool, the interviewer asked 2 open-ended questions about reducing readmissions and 4 closed-ended questions regarding SNF admission procedures. (Table 1) The supplemental questions were added after preliminary interviews with SNF clinicians revealed concerns about the SNF referral process and about communication between the hospital, emergency department (ED) and SNFs—issues not included in the INTERACT questionnaire. Interviewers used phatic communication, probing questions, and follow-up questions to elicit detailed information from participants, and participant responses were not limited to topics in the questionnaire and the list of supplemental questions.

Interviews were conducted by a hospital clinical integration coordinator, social worker, and a physician (KB, MCB, BWC). All interviewers received formal training in qualitative research methods prior to the study.

All interviews were audio recorded, with permission from the participants, and were professionally transcribed. Field notes were maintained to ensure accuracy of INTERACT QI Tool data. Participant interviews covered no more than two cases per session and lasted from 18 to 71 minutes (mean duration, 38 minutes).

 

Analysis

Analysis of transcripts was inductive and informed by grounded theory methodology, in which data is reviewed for repeating ideas, which are then analyzed and grouped to develop a theoretical understanding of the phenomenon under investigation.^13,14

A preliminary codebook was developed using transcripts of the first 11 interviews. All statements relevant to the readmission process were extracted from the raw interview transcript and collected into a single list. This list was then reviewed for statements sharing a particular idea or concern. Such statements were grouped together under the heading of a repeating idea, and each repeating idea was assigned a code. Using this codebook, each transcript was independently reviewed and coded by three study team members with formal training in inductive qualitative analysis (KB, KTM, BWC). Reviewers assigned codes to sections of relevant text. Discrepancies in code assignment were discussed among the 3 analysts until consensus was reached. Using the method of constant comparison described in grounded theory,the codebook was updated continuously as the process of coding transcripts proceeded.¹² Changes to the codebook were discussed among the coding team until consensus was achieved. The process of data acquisition and coding continued until theoretical saturation was reached. Themes relating to underlying factors associated with readmissions were then identified based on shared properties among repeating ideas. ATLAS.ti (Scientific Software, Berlin, Germany, Version 7) was used to facilitate data organization and retrieval.

RESULTS

The SNFs in our study included 12 for-profit and 3 non-profit facilities. The number of licensed beds in each facility ranged from 73 to 360, with a mean of 148 beds. The SNFs had CMS Nursing Home Compare ratings ranging from 1 star, the lowest possible rating, to 5 stars, the highest possible,¹⁵ with a mean rating of 2.9 stars. Our analysis did not reveal differences in perceived contributions to readmissions from large vs. small or highly rated vs poorly rated SNFs.

Clinicians participating in the interviews came from diverse professional backgrounds. All participating administrators were licensed nurses and continued to provide 1 or more hours of direct patient care per week at the time of the interviews. (Table 2)

The patients in our analysis represented a highly comorbid and medically complex population (Table 3). Many had barriers to communication with clinical staff, including non–English-speaking status and underlying dementia.

Five main themes emerged from our analysis: (1) lack of coordination between EDs and SNFs; (2) incompletely addressed goals of care; (3) mismatch between patient clinical needs and SNF capabilities; (4) important clinical information not effectively communicated by hospital; and (5) challenges in SNF processes and culture.

Emergent transitions: Lack of coordination between ED and SNF

SNF clinicians frequently encountered situations in which a relatively stable patient was readmitted to the hospital after being transferred to the ED, despite the fact that SNF clinicians believed the patient should have returned to the SNF once a specific test was performed or service rendered at the ED. Commonly cited clinical scenarios that resulted in such readmissions included placement of urinary catheters and evaluation for cystitis. An assistant director of nursing reported that “the ER doesn’t want to hear my side of the story,” making it difficult for her to provide information that would prevent such readmissions. Other SNF clinicians reported similar difficulties in communicating with ED clinicians.

Code status: Incompletely addressed goals of care

The SNF clinicians in our study described cases in which patients with end-stage lung disease and disseminated cancer were readmitted to the hospital, despite SNF efforts to prevent readmission and provide palliative care within the SNF. For example, a SNF advanced practice nurse described a case in which a patient with widely metastatic cancer requested readmission to the hospital for treatment of deep vein thrombosis, despite longstanding recommendations from SNF staff that the patient forego hospitalization and enroll in hospice care. After discussion of code status and goals of care with hospital clinicians, the patient chose to enroll in hospice care and not to continue anticoagulation. SNF clinicians often perceived that, in the words of one administrator, “the palliative talks in the hospital outweigh our talks by a lot.” Numerous SNF clinicians believed that in-depth clarification of goals of care prior to discharge could prevent some readmissions.

Wrong patient, wrong place: Mismatch between clinical needs and SNF capabilities

One director of nursing stated that “[when] you read a referral, there’s a huge difference sometimes between what you read and what you see.” SNF clinicians reported that this discrepancy between clinical report and clinical reality often leads to patients being placed in SNFs that are unequipped to care for them. Many patients were perceived as being too ill for discharge from the acute-care setting in the first place. A nurse manager described this as a pattern of “pushing patients out of the hospital.” However, mismatches in clinical disposition were also seen as contributing to readmissions for medically stable patients, such as those with dementia, for whom SNFs frequently lack adequate staffing and physical safeguards.

 

Missing links: Important clinical information not effectively communicated by hospital

SNF clinicians described numerous challenges in formulating plans of care based on hospital discharge documentation. Discrepancies between discharge summaries and patient instructions were perceived as common and potential causes of readmissions. For patients discharged from the academic medical center in this study, medication instructions are included in both the discharge summary sent to the SNF and in a patient instruction packet. Several SNF clinicians said that it was common for a course of antibiotics to be listed on the discharge summary but not the patient instruction packet, or vice versa. SNF clinicians, who usually lack access to the hospital’s electronic medical record, have limited means for determining the correct document. Other important clinical data points, such as intermittent intravenous (IV) furosemide dosing and suppressive antibiotic regimens, were omitted from discharge paperwork altogether. SNF clinicians had difficulty reaching hospital clinicians who could clarify these clinical questions. “Good luck finding the person that took care of [the patient] three days before,” said one director of nursing.

Change starts at home: Challenges in SNF processes and culture

Many clinicians in our study reported that their facilities had recently added clinical capabilities in an effort to care for patients with complex medical problems. For example, to prevent transfers of patients with decompensated heart failure, several facilities in our study had recently obtained certification to give IV diuretics. However, as one director of nursing stated, these efforts require “buy-in” from doctors to decrease readmissions. That buy-in has not always been forthcoming. SNF clinicians also reported difficulty convincing patients and families that their facilities are capable of providing care that, in the past, might only have been available in acute-care settings.

These themes, along with associated sub-themes and representative quotations, are shown above (Table 4).

DISCUSSION

Our study suggests that the interaction between EDs and SNFs is an important and understudied domain in the spectrum of events leading to readmission. Prior studies have documented inadequacies in patient information provided by SNFs to EDs.^16,17 Efforts to improve SNF-to-ED information sharing have focused on making sure that ED clinicians have important baseline information about patients transferred from a SNF.^18,19 However, many of the clinicians in our study reported taking proactive steps to communicate with ED clinicians. These efforts encountered logistical and cultural barriers, with information that might have prevented readmission failing to reach ED providers. Many of the SNF clinicians in our study perceived this failure as a common cause of readmission, especially for relatively stable SNF patients.

Previous studies have pointed to a role for goals of care discussions in reducing hospital readmissions.²⁰ Our data underscore an important qualification to these findings: Location matters. The SNF clinicians in our study reported frequent and detailed goals of care discussions with their patients. However, they also reported that goals of care discussions held in the subacute setting carried less weight with patients and families than discussions held in the hospital. SNF clinicians described a number of cases in which patients were willing to adjust code status or goals of care only after being readmitted to the hospital.

Our study also points to the implications of existing research showing that patients are discharged from acute care hospitals “quicker and sicker” than they had been prior to the 1983 adoption of Medicare’s prospective payment system.²¹ Specifically, the SNF clinicians we interviewed perceived a strong link between patient acuity at the time of transfer and SNFs’ persistently high readmission rates. As SNFs have worked to expand their clinical capabilities, they struggle to win buy-in from physicians and families, many of whom view SNFs as incapable of managing acute illness. Many SNF clinicians also pointed to deficiencies in their own referral and admission processes as a recurring cause of readmissions. For example, several patients in our analysis suffered from dementia. Although these patients were stable enough to leave the acute care setting, the SNF clinicians responsible for their readmissions felt that their SNFs were not well-equipped to care for patients with dementia and that the patients should instead have been transferred to facilities with more robust resources for dementia care.

Finally, our findings highlight a fundamental tension between hospitals and SNFs: Which facility ought to shoulder the responsibility and cost for services that may prevent a readmission—the hospital or the SNF? For example, does responsibility for coordinating subspecialist evaluation of a patient’s chronic condition fall to the hospital or to the SNF? If such an evaluation is undertaken during a hospitalization, it prolongs the patient’s hospital stay and happens at the hospital’s expense. If the patient is discharged to a SNF and sees the subspecialist in clinic, then the SNF must pay for transportation to and from the clinic appointment. SNF clinicians expressed near unanimity that fragmented models of care and high barriers to communication made it difficult to design solutions to these dilemmas.

 

Strengths and limitations

To our knowledge, this is the first interview-based study examining SNF clinicians’ perspectives on unplanned, 30-day hospital readmissions. We gathered information from clinicians with a range of clinical experience, all of whom had cared directly for the patient who had been readmitted. Our data came from clinicians at 15 SNFs of varying sizes and quality ratings, allowing us to identify a broad range of factors contributing to readmissions.

Because this study relied on qualitative methods, it should be viewed as hypothesis-generating rather than hypothesis-confirming. Further research is needed to determine whether variables related to the themes above are causally linked to SNF readmissions. We identified cases for review using convenience sampling of a cohort of readmitted patients at a single tertiary-care hospital, and all participating SNFs were located in Connecticut. These factors may limit the generalizability of our findings. Although the clinicians we interviewed occupied diverse roles within their respective SNFs, our sample did not include direct-care staff without managerial responsibility, such as certified nursing assistants or licensed practical nurses. This prevented our study from identifying themes into which managers would have limited insight, especially those involving cultural and management practices leading to poor communication between them and their staff. Because our study examines cases in which discharge and readmission were to a general medicine service, it may not describe factors relevant to patients discharged from subspecialist or surgical services.

Implications for future QI efforts and research

Several clinicians we interviewed suggested that readmissions might be reduced by dedicating the services of a hospital professional, such as a nurse or case manager, to monitoring the clinical course of medically complex patients after discharge. A dedicated “transition coach” could clarify deficiencies in discharge paperwork, facilitate necessary follow-up appointments, liaise with staff at both the hospital and the SNF, or coordinate acquisition of necessary equipment. Prospective trials have demonstrated that such interventions can decrease readmission rates among hospitalized patients,^22,23 but formal studies have not been carried out among cohorts of SNF patients.

Prior efforts to improve SNF-ED information sharing have focused on making sure that ED clinicians have important baseline information about patients transferred from a SNF.^24,25 The experiences of SNF clinicians in our study suggest that important information also fails to make its way from ED providers to SNFs and that this failure results in unnecessary readmissions of relatively stable SNF patients. Thus, hospitals may be able to prevent SNF readmissions by creating lines of communication between EDs and SNFs and by ensuring that ED physicians and mid-level providers are familiar with the clinical capabilities of local SNFs.

Future research and QI work should also investigate approaches to care coordination that ensure that complex patients are placed in SNFs with resources adequate to address their comorbidities. Potential interventions might include increased use of SNF “liaisons,” who would evaluate patients in-person prior to approving transfer to a given SNF. As has been previously suggested,²⁶ hospitals might also reduce readmissions by narrowing the pool of facilities to which they transfer patients, thereby building more robust, interconnected relationships with a smaller number of SNFs.

CONCLUSION

SNF clinicians identified areas for improvement at almost every point in the chain of events spanning hospitalization, discharge, and transfer. Among the most frequently cited contributors to readmissions were clinical instability at the time of discharge and omission of clinically important information from discharge documentation. Improved communication between hospitals, ED clinicians, and SNFs, as well as more thoroughly defined goals of care at the time of discharge, were seen as promising ways of decreasing readmissions. Successful interventions for reducing readmissions from SNFs will likely require multifaceted approaches to these problems.

Disclosure: This research was supported by a grant (#P30HS023554-01) from the Agency for Healthcare Research and Quality (AHRQ) and received support from Yale New Haven Hospital and the Claude D. Pepper Older Americans Independence Center at Yale University School of Medicine (#P30AG021342 NIH/NIA).

Skilled nursing facilities (SNFs) play a crucial role in the hospital readmission process.Approximately 1 in 4 Medicare beneficiaries discharged from an acute care hospital is admitted to a SNF instead of returning directly home. Of these patients, 1 in 4 will be readmitted within 30 days,¹ a rate significantly higher than the readmission rate of the inpatient population as a whole.² The 2014 Protecting Access to Medicare Act created a value-based purchasing program that will use quality measures to steer funds to, or away from, individual SNFs. When the program takes effect in 2018, the Centers for Medicare & Medicaid Services will use SNFs’ 30-day all-cause readmission rate to determine which SNFs receive payments and which receive penalties.³ The Affordable Care Act, passed in 2010, has also established penalties for hospitals with higher than expected readmission rates for Medicare patients.⁴

Despite this intensifying regulatory focus, relatively little is known about the factors that drive readmissions from SNFs. A prospective review of data from SNFs in 4 states has shown that SNFs staffed by nurse practitioners or physician assistants and those equipped to provide intravenous therapy were less likely to transfer patients to the hospital for ambulatory care-sensitive diagnoses.⁵ Qualitative studies have provided useful insight into the causes of SNF-to-hospital transfers but have not focused on 30-day readmissions.^6,7 A single survey-based study has examined the causes of SNF-to-hospital readmissions.⁸ However, survey-based methodologies have limited ability to capture the complex perspectives of SNF clinicians, who play a critical role in determining which SNF patients require evaluation or treatment in an acute care setting.

To address this gap in knowledge about factors contributing to SNF readmissions, we conducted a qualitative study examining SNF clinicians’ perspectives on patients readmitted to the hospital within 30 days of discharge. We used a structured interview tool to explore the root causes of readmission with frontline SNF staff, with the goal of using this knowledge to inform future hospital quality improvement (QI) efforts.

METHODS

Case Identification

Hospital data-tracking software (Allscripts) was used to identify patients who experienced a 30-day, unplanned readmission from SNFs to an academic medical center. We restricted our search to patients whose index admission and readmission were to the medical center’s inpatient general medicine service. A study team member (BWC) monitored the dataset on a weekly basis and contacted SNF clinicians by e-mail and telephone to arrange interviews at times of mutual convenience. To mitigate against recall bias, interviews were conducted within 30 days of the readmission in question. A total of 32 cases were identified. No SNF clinicians refused a request for interview. For 8 of these cases, it was not possible to find a time of mutual convenience within the specified 30-day window. The remaining 24 cases involved patients from 15 SNFs across Connecticut. Interviews were conducted from August 2015 to November 2015.

The project was reviewed by our institution’s Human Investigation Committee and was exempted from Institutional Review Board review.

Study Participants

Interviews were conducted on-site at SNFs with groups of 1 to 4 SNF clinicians and administrators. SNF participants were informed of interviewer credentials and the study’s QI goals prior to participation. Participation was voluntary and did not affect the clinician’s relationship with the hospital or the SNF. Participants were not paid.

DATA COLLECTION

Interventions to Reduce Acute Care Transfers (INTERACT) is a QI program that includes training for clinicians, communication tools, and advance care planning tools.⁹ INTERACT is currently used in 138 Connecticut SNFs as part of a statewide QI effort funded by the Connecticut State Department of Public Health. In prospective QI studies,^10,11 implementation of INTERACT has been associated with decreased transfers from SNFs to acute care hospitals. The INTERACT Quality Improvement Tool, one part of the INTERACT bundle of interventions, is a 26-item questionnaire used to identify root causes of transfers from SNFs to acute care hospitals. It includes both checklists and open-ended questions about patient factors, SNF procedures, and SNF clinical decision-making.

We used the INTERACT QI Tool¹² to conduct structured interviews with nurses and administrators at SNFs. Interviewers used a hard copy of the tool to maintain field notes, and all parts of the questionnaire were completed in each interview. Although the questionnaire elicits baseline demographic and medical information, such as the patient’s age and vital signs prior to readmission, the majority of each interview was dedicated to discussion of the open-ended questions in Table 1. Upon completion of the INTERACT QI Tool, the interviewer asked 2 open-ended questions about reducing readmissions and 4 closed-ended questions regarding SNF admission procedures. (Table 1) The supplemental questions were added after preliminary interviews with SNF clinicians revealed concerns about the SNF referral process and about communication between the hospital, emergency department (ED) and SNFs—issues not included in the INTERACT questionnaire. Interviewers used phatic communication, probing questions, and follow-up questions to elicit detailed information from participants, and participant responses were not limited to topics in the questionnaire and the list of supplemental questions.

Interviews were conducted by a hospital clinical integration coordinator, social worker, and a physician (KB, MCB, BWC). All interviewers received formal training in qualitative research methods prior to the study.

All interviews were audio recorded, with permission from the participants, and were professionally transcribed. Field notes were maintained to ensure accuracy of INTERACT QI Tool data. Participant interviews covered no more than two cases per session and lasted from 18 to 71 minutes (mean duration, 38 minutes).

 

Analysis

Analysis of transcripts was inductive and informed by grounded theory methodology, in which data is reviewed for repeating ideas, which are then analyzed and grouped to develop a theoretical understanding of the phenomenon under investigation.^13,14

A preliminary codebook was developed using transcripts of the first 11 interviews. All statements relevant to the readmission process were extracted from the raw interview transcript and collected into a single list. This list was then reviewed for statements sharing a particular idea or concern. Such statements were grouped together under the heading of a repeating idea, and each repeating idea was assigned a code. Using this codebook, each transcript was independently reviewed and coded by three study team members with formal training in inductive qualitative analysis (KB, KTM, BWC). Reviewers assigned codes to sections of relevant text. Discrepancies in code assignment were discussed among the 3 analysts until consensus was reached. Using the method of constant comparison described in grounded theory,the codebook was updated continuously as the process of coding transcripts proceeded.¹² Changes to the codebook were discussed among the coding team until consensus was achieved. The process of data acquisition and coding continued until theoretical saturation was reached. Themes relating to underlying factors associated with readmissions were then identified based on shared properties among repeating ideas. ATLAS.ti (Scientific Software, Berlin, Germany, Version 7) was used to facilitate data organization and retrieval.

RESULTS

The SNFs in our study included 12 for-profit and 3 non-profit facilities. The number of licensed beds in each facility ranged from 73 to 360, with a mean of 148 beds. The SNFs had CMS Nursing Home Compare ratings ranging from 1 star, the lowest possible rating, to 5 stars, the highest possible,¹⁵ with a mean rating of 2.9 stars. Our analysis did not reveal differences in perceived contributions to readmissions from large vs. small or highly rated vs poorly rated SNFs.

Clinicians participating in the interviews came from diverse professional backgrounds. All participating administrators were licensed nurses and continued to provide 1 or more hours of direct patient care per week at the time of the interviews. (Table 2)

The patients in our analysis represented a highly comorbid and medically complex population (Table 3). Many had barriers to communication with clinical staff, including non–English-speaking status and underlying dementia.

Five main themes emerged from our analysis: (1) lack of coordination between EDs and SNFs; (2) incompletely addressed goals of care; (3) mismatch between patient clinical needs and SNF capabilities; (4) important clinical information not effectively communicated by hospital; and (5) challenges in SNF processes and culture.

Emergent transitions: Lack of coordination between ED and SNF

SNF clinicians frequently encountered situations in which a relatively stable patient was readmitted to the hospital after being transferred to the ED, despite the fact that SNF clinicians believed the patient should have returned to the SNF once a specific test was performed or service rendered at the ED. Commonly cited clinical scenarios that resulted in such readmissions included placement of urinary catheters and evaluation for cystitis. An assistant director of nursing reported that “the ER doesn’t want to hear my side of the story,” making it difficult for her to provide information that would prevent such readmissions. Other SNF clinicians reported similar difficulties in communicating with ED clinicians.

Code status: Incompletely addressed goals of care

The SNF clinicians in our study described cases in which patients with end-stage lung disease and disseminated cancer were readmitted to the hospital, despite SNF efforts to prevent readmission and provide palliative care within the SNF. For example, a SNF advanced practice nurse described a case in which a patient with widely metastatic cancer requested readmission to the hospital for treatment of deep vein thrombosis, despite longstanding recommendations from SNF staff that the patient forego hospitalization and enroll in hospice care. After discussion of code status and goals of care with hospital clinicians, the patient chose to enroll in hospice care and not to continue anticoagulation. SNF clinicians often perceived that, in the words of one administrator, “the palliative talks in the hospital outweigh our talks by a lot.” Numerous SNF clinicians believed that in-depth clarification of goals of care prior to discharge could prevent some readmissions.

Wrong patient, wrong place: Mismatch between clinical needs and SNF capabilities

One director of nursing stated that “[when] you read a referral, there’s a huge difference sometimes between what you read and what you see.” SNF clinicians reported that this discrepancy between clinical report and clinical reality often leads to patients being placed in SNFs that are unequipped to care for them. Many patients were perceived as being too ill for discharge from the acute-care setting in the first place. A nurse manager described this as a pattern of “pushing patients out of the hospital.” However, mismatches in clinical disposition were also seen as contributing to readmissions for medically stable patients, such as those with dementia, for whom SNFs frequently lack adequate staffing and physical safeguards.

 

Missing links: Important clinical information not effectively communicated by hospital

SNF clinicians described numerous challenges in formulating plans of care based on hospital discharge documentation. Discrepancies between discharge summaries and patient instructions were perceived as common and potential causes of readmissions. For patients discharged from the academic medical center in this study, medication instructions are included in both the discharge summary sent to the SNF and in a patient instruction packet. Several SNF clinicians said that it was common for a course of antibiotics to be listed on the discharge summary but not the patient instruction packet, or vice versa. SNF clinicians, who usually lack access to the hospital’s electronic medical record, have limited means for determining the correct document. Other important clinical data points, such as intermittent intravenous (IV) furosemide dosing and suppressive antibiotic regimens, were omitted from discharge paperwork altogether. SNF clinicians had difficulty reaching hospital clinicians who could clarify these clinical questions. “Good luck finding the person that took care of [the patient] three days before,” said one director of nursing.

Change starts at home: Challenges in SNF processes and culture

Many clinicians in our study reported that their facilities had recently added clinical capabilities in an effort to care for patients with complex medical problems. For example, to prevent transfers of patients with decompensated heart failure, several facilities in our study had recently obtained certification to give IV diuretics. However, as one director of nursing stated, these efforts require “buy-in” from doctors to decrease readmissions. That buy-in has not always been forthcoming. SNF clinicians also reported difficulty convincing patients and families that their facilities are capable of providing care that, in the past, might only have been available in acute-care settings.

These themes, along with associated sub-themes and representative quotations, are shown above (Table 4).

DISCUSSION

Our study suggests that the interaction between EDs and SNFs is an important and understudied domain in the spectrum of events leading to readmission. Prior studies have documented inadequacies in patient information provided by SNFs to EDs.^16,17 Efforts to improve SNF-to-ED information sharing have focused on making sure that ED clinicians have important baseline information about patients transferred from a SNF.^18,19 However, many of the clinicians in our study reported taking proactive steps to communicate with ED clinicians. These efforts encountered logistical and cultural barriers, with information that might have prevented readmission failing to reach ED providers. Many of the SNF clinicians in our study perceived this failure as a common cause of readmission, especially for relatively stable SNF patients.

Previous studies have pointed to a role for goals of care discussions in reducing hospital readmissions.²⁰ Our data underscore an important qualification to these findings: Location matters. The SNF clinicians in our study reported frequent and detailed goals of care discussions with their patients. However, they also reported that goals of care discussions held in the subacute setting carried less weight with patients and families than discussions held in the hospital. SNF clinicians described a number of cases in which patients were willing to adjust code status or goals of care only after being readmitted to the hospital.

Our study also points to the implications of existing research showing that patients are discharged from acute care hospitals “quicker and sicker” than they had been prior to the 1983 adoption of Medicare’s prospective payment system.²¹ Specifically, the SNF clinicians we interviewed perceived a strong link between patient acuity at the time of transfer and SNFs’ persistently high readmission rates. As SNFs have worked to expand their clinical capabilities, they struggle to win buy-in from physicians and families, many of whom view SNFs as incapable of managing acute illness. Many SNF clinicians also pointed to deficiencies in their own referral and admission processes as a recurring cause of readmissions. For example, several patients in our analysis suffered from dementia. Although these patients were stable enough to leave the acute care setting, the SNF clinicians responsible for their readmissions felt that their SNFs were not well-equipped to care for patients with dementia and that the patients should instead have been transferred to facilities with more robust resources for dementia care.

Finally, our findings highlight a fundamental tension between hospitals and SNFs: Which facility ought to shoulder the responsibility and cost for services that may prevent a readmission—the hospital or the SNF? For example, does responsibility for coordinating subspecialist evaluation of a patient’s chronic condition fall to the hospital or to the SNF? If such an evaluation is undertaken during a hospitalization, it prolongs the patient’s hospital stay and happens at the hospital’s expense. If the patient is discharged to a SNF and sees the subspecialist in clinic, then the SNF must pay for transportation to and from the clinic appointment. SNF clinicians expressed near unanimity that fragmented models of care and high barriers to communication made it difficult to design solutions to these dilemmas.

 

Strengths and limitations

To our knowledge, this is the first interview-based study examining SNF clinicians’ perspectives on unplanned, 30-day hospital readmissions. We gathered information from clinicians with a range of clinical experience, all of whom had cared directly for the patient who had been readmitted. Our data came from clinicians at 15 SNFs of varying sizes and quality ratings, allowing us to identify a broad range of factors contributing to readmissions.

Because this study relied on qualitative methods, it should be viewed as hypothesis-generating rather than hypothesis-confirming. Further research is needed to determine whether variables related to the themes above are causally linked to SNF readmissions. We identified cases for review using convenience sampling of a cohort of readmitted patients at a single tertiary-care hospital, and all participating SNFs were located in Connecticut. These factors may limit the generalizability of our findings. Although the clinicians we interviewed occupied diverse roles within their respective SNFs, our sample did not include direct-care staff without managerial responsibility, such as certified nursing assistants or licensed practical nurses. This prevented our study from identifying themes into which managers would have limited insight, especially those involving cultural and management practices leading to poor communication between them and their staff. Because our study examines cases in which discharge and readmission were to a general medicine service, it may not describe factors relevant to patients discharged from subspecialist or surgical services.

Implications for future QI efforts and research

Several clinicians we interviewed suggested that readmissions might be reduced by dedicating the services of a hospital professional, such as a nurse or case manager, to monitoring the clinical course of medically complex patients after discharge. A dedicated “transition coach” could clarify deficiencies in discharge paperwork, facilitate necessary follow-up appointments, liaise with staff at both the hospital and the SNF, or coordinate acquisition of necessary equipment. Prospective trials have demonstrated that such interventions can decrease readmission rates among hospitalized patients,^22,23 but formal studies have not been carried out among cohorts of SNF patients.

Prior efforts to improve SNF-ED information sharing have focused on making sure that ED clinicians have important baseline information about patients transferred from a SNF.^24,25 The experiences of SNF clinicians in our study suggest that important information also fails to make its way from ED providers to SNFs and that this failure results in unnecessary readmissions of relatively stable SNF patients. Thus, hospitals may be able to prevent SNF readmissions by creating lines of communication between EDs and SNFs and by ensuring that ED physicians and mid-level providers are familiar with the clinical capabilities of local SNFs.

Future research and QI work should also investigate approaches to care coordination that ensure that complex patients are placed in SNFs with resources adequate to address their comorbidities. Potential interventions might include increased use of SNF “liaisons,” who would evaluate patients in-person prior to approving transfer to a given SNF. As has been previously suggested,²⁶ hospitals might also reduce readmissions by narrowing the pool of facilities to which they transfer patients, thereby building more robust, interconnected relationships with a smaller number of SNFs.

CONCLUSION

SNF clinicians identified areas for improvement at almost every point in the chain of events spanning hospitalization, discharge, and transfer. Among the most frequently cited contributors to readmissions were clinical instability at the time of discharge and omission of clinically important information from discharge documentation. Improved communication between hospitals, ED clinicians, and SNFs, as well as more thoroughly defined goals of care at the time of discharge, were seen as promising ways of decreasing readmissions. Successful interventions for reducing readmissions from SNFs will likely require multifaceted approaches to these problems.

Disclosure: This research was supported by a grant (#P30HS023554-01) from the Agency for Healthcare Research and Quality (AHRQ) and received support from Yale New Haven Hospital and the Claude D. Pepper Older Americans Independence Center at Yale University School of Medicine (#P30AG021342 NIH/NIA).

Skilled nursing facilities (SNFs) play a crucial role in the hospital readmission process.Approximately 1 in 4 Medicare beneficiaries discharged from an acute care hospital is admitted to a SNF instead of returning directly home. Of these patients, 1 in 4 will be readmitted within 30 days,¹ a rate significantly higher than the readmission rate of the inpatient population as a whole.² The 2014 Protecting Access to Medicare Act created a value-based purchasing program that will use quality measures to steer funds to, or away from, individual SNFs. When the program takes effect in 2018, the Centers for Medicare & Medicaid Services will use SNFs’ 30-day all-cause readmission rate to determine which SNFs receive payments and which receive penalties.³ The Affordable Care Act, passed in 2010, has also established penalties for hospitals with higher than expected readmission rates for Medicare patients.⁴

Despite this intensifying regulatory focus, relatively little is known about the factors that drive readmissions from SNFs. A prospective review of data from SNFs in 4 states has shown that SNFs staffed by nurse practitioners or physician assistants and those equipped to provide intravenous therapy were less likely to transfer patients to the hospital for ambulatory care-sensitive diagnoses.⁵ Qualitative studies have provided useful insight into the causes of SNF-to-hospital transfers but have not focused on 30-day readmissions.^6,7 A single survey-based study has examined the causes of SNF-to-hospital readmissions.⁸ However, survey-based methodologies have limited ability to capture the complex perspectives of SNF clinicians, who play a critical role in determining which SNF patients require evaluation or treatment in an acute care setting.

To address this gap in knowledge about factors contributing to SNF readmissions, we conducted a qualitative study examining SNF clinicians’ perspectives on patients readmitted to the hospital within 30 days of discharge. We used a structured interview tool to explore the root causes of readmission with frontline SNF staff, with the goal of using this knowledge to inform future hospital quality improvement (QI) efforts.

METHODS

Case Identification

Hospital data-tracking software (Allscripts) was used to identify patients who experienced a 30-day, unplanned readmission from SNFs to an academic medical center. We restricted our search to patients whose index admission and readmission were to the medical center’s inpatient general medicine service. A study team member (BWC) monitored the dataset on a weekly basis and contacted SNF clinicians by e-mail and telephone to arrange interviews at times of mutual convenience. To mitigate against recall bias, interviews were conducted within 30 days of the readmission in question. A total of 32 cases were identified. No SNF clinicians refused a request for interview. For 8 of these cases, it was not possible to find a time of mutual convenience within the specified 30-day window. The remaining 24 cases involved patients from 15 SNFs across Connecticut. Interviews were conducted from August 2015 to November 2015.

The project was reviewed by our institution’s Human Investigation Committee and was exempted from Institutional Review Board review.

Study Participants

Interviews were conducted on-site at SNFs with groups of 1 to 4 SNF clinicians and administrators. SNF participants were informed of interviewer credentials and the study’s QI goals prior to participation. Participation was voluntary and did not affect the clinician’s relationship with the hospital or the SNF. Participants were not paid.

DATA COLLECTION

Interventions to Reduce Acute Care Transfers (INTERACT) is a QI program that includes training for clinicians, communication tools, and advance care planning tools.⁹ INTERACT is currently used in 138 Connecticut SNFs as part of a statewide QI effort funded by the Connecticut State Department of Public Health. In prospective QI studies,^10,11 implementation of INTERACT has been associated with decreased transfers from SNFs to acute care hospitals. The INTERACT Quality Improvement Tool, one part of the INTERACT bundle of interventions, is a 26-item questionnaire used to identify root causes of transfers from SNFs to acute care hospitals. It includes both checklists and open-ended questions about patient factors, SNF procedures, and SNF clinical decision-making.

We used the INTERACT QI Tool¹² to conduct structured interviews with nurses and administrators at SNFs. Interviewers used a hard copy of the tool to maintain field notes, and all parts of the questionnaire were completed in each interview. Although the questionnaire elicits baseline demographic and medical information, such as the patient’s age and vital signs prior to readmission, the majority of each interview was dedicated to discussion of the open-ended questions in Table 1. Upon completion of the INTERACT QI Tool, the interviewer asked 2 open-ended questions about reducing readmissions and 4 closed-ended questions regarding SNF admission procedures. (Table 1) The supplemental questions were added after preliminary interviews with SNF clinicians revealed concerns about the SNF referral process and about communication between the hospital, emergency department (ED) and SNFs—issues not included in the INTERACT questionnaire. Interviewers used phatic communication, probing questions, and follow-up questions to elicit detailed information from participants, and participant responses were not limited to topics in the questionnaire and the list of supplemental questions.

Interviews were conducted by a hospital clinical integration coordinator, social worker, and a physician (KB, MCB, BWC). All interviewers received formal training in qualitative research methods prior to the study.

All interviews were audio recorded, with permission from the participants, and were professionally transcribed. Field notes were maintained to ensure accuracy of INTERACT QI Tool data. Participant interviews covered no more than two cases per session and lasted from 18 to 71 minutes (mean duration, 38 minutes).

 

Analysis

Analysis of transcripts was inductive and informed by grounded theory methodology, in which data is reviewed for repeating ideas, which are then analyzed and grouped to develop a theoretical understanding of the phenomenon under investigation.^13,14

A preliminary codebook was developed using transcripts of the first 11 interviews. All statements relevant to the readmission process were extracted from the raw interview transcript and collected into a single list. This list was then reviewed for statements sharing a particular idea or concern. Such statements were grouped together under the heading of a repeating idea, and each repeating idea was assigned a code. Using this codebook, each transcript was independently reviewed and coded by three study team members with formal training in inductive qualitative analysis (KB, KTM, BWC). Reviewers assigned codes to sections of relevant text. Discrepancies in code assignment were discussed among the 3 analysts until consensus was reached. Using the method of constant comparison described in grounded theory,the codebook was updated continuously as the process of coding transcripts proceeded.¹² Changes to the codebook were discussed among the coding team until consensus was achieved. The process of data acquisition and coding continued until theoretical saturation was reached. Themes relating to underlying factors associated with readmissions were then identified based on shared properties among repeating ideas. ATLAS.ti (Scientific Software, Berlin, Germany, Version 7) was used to facilitate data organization and retrieval.

RESULTS

The SNFs in our study included 12 for-profit and 3 non-profit facilities. The number of licensed beds in each facility ranged from 73 to 360, with a mean of 148 beds. The SNFs had CMS Nursing Home Compare ratings ranging from 1 star, the lowest possible rating, to 5 stars, the highest possible,¹⁵ with a mean rating of 2.9 stars. Our analysis did not reveal differences in perceived contributions to readmissions from large vs. small or highly rated vs poorly rated SNFs.

Clinicians participating in the interviews came from diverse professional backgrounds. All participating administrators were licensed nurses and continued to provide 1 or more hours of direct patient care per week at the time of the interviews. (Table 2)

The patients in our analysis represented a highly comorbid and medically complex population (Table 3). Many had barriers to communication with clinical staff, including non–English-speaking status and underlying dementia.

Five main themes emerged from our analysis: (1) lack of coordination between EDs and SNFs; (2) incompletely addressed goals of care; (3) mismatch between patient clinical needs and SNF capabilities; (4) important clinical information not effectively communicated by hospital; and (5) challenges in SNF processes and culture.

Emergent transitions: Lack of coordination between ED and SNF

SNF clinicians frequently encountered situations in which a relatively stable patient was readmitted to the hospital after being transferred to the ED, despite the fact that SNF clinicians believed the patient should have returned to the SNF once a specific test was performed or service rendered at the ED. Commonly cited clinical scenarios that resulted in such readmissions included placement of urinary catheters and evaluation for cystitis. An assistant director of nursing reported that “the ER doesn’t want to hear my side of the story,” making it difficult for her to provide information that would prevent such readmissions. Other SNF clinicians reported similar difficulties in communicating with ED clinicians.

Code status: Incompletely addressed goals of care

The SNF clinicians in our study described cases in which patients with end-stage lung disease and disseminated cancer were readmitted to the hospital, despite SNF efforts to prevent readmission and provide palliative care within the SNF. For example, a SNF advanced practice nurse described a case in which a patient with widely metastatic cancer requested readmission to the hospital for treatment of deep vein thrombosis, despite longstanding recommendations from SNF staff that the patient forego hospitalization and enroll in hospice care. After discussion of code status and goals of care with hospital clinicians, the patient chose to enroll in hospice care and not to continue anticoagulation. SNF clinicians often perceived that, in the words of one administrator, “the palliative talks in the hospital outweigh our talks by a lot.” Numerous SNF clinicians believed that in-depth clarification of goals of care prior to discharge could prevent some readmissions.

Wrong patient, wrong place: Mismatch between clinical needs and SNF capabilities

One director of nursing stated that “[when] you read a referral, there’s a huge difference sometimes between what you read and what you see.” SNF clinicians reported that this discrepancy between clinical report and clinical reality often leads to patients being placed in SNFs that are unequipped to care for them. Many patients were perceived as being too ill for discharge from the acute-care setting in the first place. A nurse manager described this as a pattern of “pushing patients out of the hospital.” However, mismatches in clinical disposition were also seen as contributing to readmissions for medically stable patients, such as those with dementia, for whom SNFs frequently lack adequate staffing and physical safeguards.

 

Missing links: Important clinical information not effectively communicated by hospital

SNF clinicians described numerous challenges in formulating plans of care based on hospital discharge documentation. Discrepancies between discharge summaries and patient instructions were perceived as common and potential causes of readmissions. For patients discharged from the academic medical center in this study, medication instructions are included in both the discharge summary sent to the SNF and in a patient instruction packet. Several SNF clinicians said that it was common for a course of antibiotics to be listed on the discharge summary but not the patient instruction packet, or vice versa. SNF clinicians, who usually lack access to the hospital’s electronic medical record, have limited means for determining the correct document. Other important clinical data points, such as intermittent intravenous (IV) furosemide dosing and suppressive antibiotic regimens, were omitted from discharge paperwork altogether. SNF clinicians had difficulty reaching hospital clinicians who could clarify these clinical questions. “Good luck finding the person that took care of [the patient] three days before,” said one director of nursing.

Change starts at home: Challenges in SNF processes and culture

Many clinicians in our study reported that their facilities had recently added clinical capabilities in an effort to care for patients with complex medical problems. For example, to prevent transfers of patients with decompensated heart failure, several facilities in our study had recently obtained certification to give IV diuretics. However, as one director of nursing stated, these efforts require “buy-in” from doctors to decrease readmissions. That buy-in has not always been forthcoming. SNF clinicians also reported difficulty convincing patients and families that their facilities are capable of providing care that, in the past, might only have been available in acute-care settings.

These themes, along with associated sub-themes and representative quotations, are shown above (Table 4).

DISCUSSION

Our study suggests that the interaction between EDs and SNFs is an important and understudied domain in the spectrum of events leading to readmission. Prior studies have documented inadequacies in patient information provided by SNFs to EDs.^16,17 Efforts to improve SNF-to-ED information sharing have focused on making sure that ED clinicians have important baseline information about patients transferred from a SNF.^18,19 However, many of the clinicians in our study reported taking proactive steps to communicate with ED clinicians. These efforts encountered logistical and cultural barriers, with information that might have prevented readmission failing to reach ED providers. Many of the SNF clinicians in our study perceived this failure as a common cause of readmission, especially for relatively stable SNF patients.

Previous studies have pointed to a role for goals of care discussions in reducing hospital readmissions.²⁰ Our data underscore an important qualification to these findings: Location matters. The SNF clinicians in our study reported frequent and detailed goals of care discussions with their patients. However, they also reported that goals of care discussions held in the subacute setting carried less weight with patients and families than discussions held in the hospital. SNF clinicians described a number of cases in which patients were willing to adjust code status or goals of care only after being readmitted to the hospital.

Our study also points to the implications of existing research showing that patients are discharged from acute care hospitals “quicker and sicker” than they had been prior to the 1983 adoption of Medicare’s prospective payment system.²¹ Specifically, the SNF clinicians we interviewed perceived a strong link between patient acuity at the time of transfer and SNFs’ persistently high readmission rates. As SNFs have worked to expand their clinical capabilities, they struggle to win buy-in from physicians and families, many of whom view SNFs as incapable of managing acute illness. Many SNF clinicians also pointed to deficiencies in their own referral and admission processes as a recurring cause of readmissions. For example, several patients in our analysis suffered from dementia. Although these patients were stable enough to leave the acute care setting, the SNF clinicians responsible for their readmissions felt that their SNFs were not well-equipped to care for patients with dementia and that the patients should instead have been transferred to facilities with more robust resources for dementia care.

Finally, our findings highlight a fundamental tension between hospitals and SNFs: Which facility ought to shoulder the responsibility and cost for services that may prevent a readmission—the hospital or the SNF? For example, does responsibility for coordinating subspecialist evaluation of a patient’s chronic condition fall to the hospital or to the SNF? If such an evaluation is undertaken during a hospitalization, it prolongs the patient’s hospital stay and happens at the hospital’s expense. If the patient is discharged to a SNF and sees the subspecialist in clinic, then the SNF must pay for transportation to and from the clinic appointment. SNF clinicians expressed near unanimity that fragmented models of care and high barriers to communication made it difficult to design solutions to these dilemmas.

 

Strengths and limitations

To our knowledge, this is the first interview-based study examining SNF clinicians’ perspectives on unplanned, 30-day hospital readmissions. We gathered information from clinicians with a range of clinical experience, all of whom had cared directly for the patient who had been readmitted. Our data came from clinicians at 15 SNFs of varying sizes and quality ratings, allowing us to identify a broad range of factors contributing to readmissions.

Because this study relied on qualitative methods, it should be viewed as hypothesis-generating rather than hypothesis-confirming. Further research is needed to determine whether variables related to the themes above are causally linked to SNF readmissions. We identified cases for review using convenience sampling of a cohort of readmitted patients at a single tertiary-care hospital, and all participating SNFs were located in Connecticut. These factors may limit the generalizability of our findings. Although the clinicians we interviewed occupied diverse roles within their respective SNFs, our sample did not include direct-care staff without managerial responsibility, such as certified nursing assistants or licensed practical nurses. This prevented our study from identifying themes into which managers would have limited insight, especially those involving cultural and management practices leading to poor communication between them and their staff. Because our study examines cases in which discharge and readmission were to a general medicine service, it may not describe factors relevant to patients discharged from subspecialist or surgical services.

Implications for future QI efforts and research

Several clinicians we interviewed suggested that readmissions might be reduced by dedicating the services of a hospital professional, such as a nurse or case manager, to monitoring the clinical course of medically complex patients after discharge. A dedicated “transition coach” could clarify deficiencies in discharge paperwork, facilitate necessary follow-up appointments, liaise with staff at both the hospital and the SNF, or coordinate acquisition of necessary equipment. Prospective trials have demonstrated that such interventions can decrease readmission rates among hospitalized patients,^22,23 but formal studies have not been carried out among cohorts of SNF patients.

Prior efforts to improve SNF-ED information sharing have focused on making sure that ED clinicians have important baseline information about patients transferred from a SNF.^24,25 The experiences of SNF clinicians in our study suggest that important information also fails to make its way from ED providers to SNFs and that this failure results in unnecessary readmissions of relatively stable SNF patients. Thus, hospitals may be able to prevent SNF readmissions by creating lines of communication between EDs and SNFs and by ensuring that ED physicians and mid-level providers are familiar with the clinical capabilities of local SNFs.

Future research and QI work should also investigate approaches to care coordination that ensure that complex patients are placed in SNFs with resources adequate to address their comorbidities. Potential interventions might include increased use of SNF “liaisons,” who would evaluate patients in-person prior to approving transfer to a given SNF. As has been previously suggested,²⁶ hospitals might also reduce readmissions by narrowing the pool of facilities to which they transfer patients, thereby building more robust, interconnected relationships with a smaller number of SNFs.

CONCLUSION

SNF clinicians identified areas for improvement at almost every point in the chain of events spanning hospitalization, discharge, and transfer. Among the most frequently cited contributors to readmissions were clinical instability at the time of discharge and omission of clinically important information from discharge documentation. Improved communication between hospitals, ED clinicians, and SNFs, as well as more thoroughly defined goals of care at the time of discharge, were seen as promising ways of decreasing readmissions. Successful interventions for reducing readmissions from SNFs will likely require multifaceted approaches to these problems.

Disclosure: This research was supported by a grant (#P30HS023554-01) from the Agency for Healthcare Research and Quality (AHRQ) and received support from Yale New Haven Hospital and the Claude D. Pepper Older Americans Independence Center at Yale University School of Medicine (#P30AG021342 NIH/NIA).

Respiratory illness (RI) is one of the most common reasons for pediatric hospitalization.¹ Examples of RI include acute illness, such as bronchiolitis, bacterial pneumonia, and asthma, as well as chronic conditions, such as obstructive sleep apnea and chronic respiratory insufficiency. Hospital care for RI includes monitoring and treatment to optimize oxygenation, ventilation, hydration, and other body functions. Most previously healthy children hospitalized with RI stay in the hospital for a limited duration (eg, a few days) because the severity of their illness is short lived and they quickly return to their previous healthy status.² However, hospital care is increasing for children with fragile and tenuous health due to complex medical conditions.³ RI is a common reason for hospitalization among these children as well and recovery of respiratory health and function can be slow and protracted for some of them.⁴ Weeks, months, or longer periods of time may be necessary for the children to return to their previous respiratory baseline health and function after hospital discharge; other children may not return to their baseline.^5,6

Hospitalized older adults with high-severity RI are routinely streamlined for transfer to post-acute facility care (PAC) shortly (eg, a few days) after acute-care hospitalization. Nearly 70% of elderly Medicare beneficiaries use PAC following a brief length of stay (LOS) in the acute-care hospital.⁷ It is believed that PAC helps optimize the patients’ health and functional status and relieves the family caregiving burden that would have occurred at home.^8-10 PAC use also helps to shorten acute-care hospitalization for RI while avoiding readmission.^8-10 In contrast with adult patients, use of PAC for hospitalized children is not routine.¹¹ While PAC use in children is infrequent, RI is one of the most common reasons for acute admission among children who use it.¹²

For some children with RI, PAC might be positioned to offer a safe, therapeutic, and high-value setting for pulmonary rehabilitation, as well as related medical, nutritional, functional, and family cares.⁶ PAC, by design, could possibly help some of the children transition back into their homes and communities. As studies continue to emerge that assess the value of PAC in children, it is important to learn more about the use of PAC in children hospitalized with RI. The objectives were to (1) assess which children admitted with RI are the most likely to use PAC services for recovery and (2) estimate how many hospitalized children not using PAC had the same characteristics as those who did.

METHODS

Study Design, Setting, and Population

We conducted a retrospective cohort analysis of 609,800 hospitalizations for RI occurring from January 1, 2010 to December 31, 2015, in 43 freestanding children’s hospitals in the Pediatric Health Information Systems (PHIS) dataset. All hospitals participating in PHIS are members of the Children’s Hospital Association.¹³ The Boston Children’s Hospital Institutional Review Board approved this study with a waiver for informed consent.

RI was identified using the Agency for Healthcare Research and Quality (AHRQ) Clinical Classification System (CCS).¹⁴ Using diagnosis CCS category 8 (“Diseases of the Respiratory System”) and the procedure CCS category 6 (“Operations on the Respiratory System”), we identified all hospitalizations from the participating hospitals with a principal diagnosis or procedure International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) code for an RI.

Main Outcome Measure

Discharge disposition following the acute-care hospitalization for RI was the main outcome measure. We used PHIS uniform disposition coding to classify the discharge disposition as transfer to PAC (ie, rehabilitation facility, skilled nursing facility, etc.) vs all other dispositions (ie, routine to home, against medical advice, etc.).¹² The PAC disposition category was derived from the Centers for Medicare & Medicaid Services Patient Discharge Status Codes and Hospital Transfer Policies as informed by the National Uniform Billing Committee Official UB-04 Data Specifications Manual, 2008. PAC transfer included disposition to external PAC facilities, as well as to internal, embedded PAC units residing in a few of the acute-care children’s hospitals included in the cohort.

 

Demographic and Clinical Characteristics

We assessed patient demographic and clinical characteristics that might correlate with PAC use following acute-care hospitalization for RI. Demographic characteristics included gender, age at admission in years, payer (public, private, and other), and race/ethnicity (Hispanic, non-Hispanic black, non-Hispanic white, other).

Clinical characteristics included chronic conditions (type and number) and assistance with medical technology. Chronic condition and medical technology characteristics were assessed with ICD-9-CM diagnosis codes. PHIS contain up to 41 ICD-9-CM diagnosis codes per hospital discharge record. To identify the presence and number of chronic conditions, we used the AHRQ Chronic Condition Indicator system, which categorizes over 14,000 ICD-9-CM diagnosis codes into chronic vs non-chronic conditions.^14,15 Children hospitalized with a chronic condition were further classified as having a complex chronic condition (CCC) using Feudtner and colleagues’ ICD-9-CM diagnosis classification scheme.¹⁶ CCCs represent defined diagnosis groupings expected to last longer than 12 months and involving either a single organ system, severe enough to require specialty pediatric care and hospitalization, or multiple organ systems.^17,18 Hospitalized children who were assisted with medical technology were identified with ICD-9-CM codes indicating the use of a medical device to manage and treat a chronic illness (eg, ventricular shunt to treat hydrocephalus) or to maintain basic body functions necessary for sustaining life (eg, a tracheostomy tube for breathing).^19,20 We distinguished children undergoing tracheotomy during hospitalization using ICD-9-CM procedure codes 31.1 and 31.2.

Acute-Care Hospitalization Characteristics

We also assessed the relationship between acute-care hospitalization characteristics and use of PAC after discharge, including US census region, LOS, use of intensive care, number of medication classes administered, and use of enhanced respiratory support. Enhanced respiratory support was defined as use of continuous or bilevel positive airway pressure (CPAP or BiPAP) or mechanical ventilation during the acute-care hospitalization for RI. These respiratory supports were identified using billing data in PHIS.

Statistical Analysis

In bivariable analysis, we compared demographic, clinical, and hospitalization characteristics of hospitalized children with vs without discharge to PAC using Rao-Scott chi-square tests and Wilcoxon rank-sum tests as appropriate. In multivariable analysis, we derived a generalized linear mix effects model with fixed effects for demographic, clinical, and hospitalization characteristics that were associated with PAC at P < 0.1 in bivariable analysis (ie, age, gender, race/ethnicity, payer, medical technology, use of intensive care unit [ICU], use of positive pressure or mechanical ventilation, hospital region, LOS, new tracheostomy, existing tracheostomy, other technologies, number of medications, number of chronic conditions [of any complexity], and type of complex chronic conditions). We controlled for clustering of patients within hospitals by including a random intercept for each hospital. We also assessed combinations of patient characteristics on the likelihood of PAC use with classification and regression tree (CART) modeling. Using CART, we determined which characteristic combinations were associated with the highest and lowest use of PAC using binary split and post-pruning, goodness of fit rules.²¹ All statistical analyses were performed using SAS v.9.4 (SAS Institute, Cary, NC), and R v.3.2 (R Foundation for Statistical Computing, Vienna, Austria) using the “party” package. The threshold for statistical significance was set at P < 0.05.

RESULTS

Of the 609,800 hospitalizations for RI, PAC use after discharge occurred for 2660 (0.4%). RI discharges to PAC accounted for 2.1% (n = 67,405) of hospital days and 2.7% ($280 million) of hospital cost of all RI hospitalizations. For discharges to PAC, the most common RI were pneumonia (29.1% [n = 773]), respiratory failure or insufficiency (unspecified reason; 22.0% [n = 584]), and upper respiratory infection (12.2% [n = 323]).

Demographic Characteristics

Median age at acute-care admission was higher for PAC vs non-PAC discharges (6 years [interquartile range {IQR} 1-15] vs 2 years [0-7], P < 0.001; Table 1). Hispanic patients accounted for a smaller percentage of RI discharges to PAC vs non-PAC (14.1% vs 21.8%, P < 0.001) and a higher percentage to PAC were for patients with public insurance (75.9% vs 62.5, P < 0.001; Table 1).

Clinical Characteristics

A greater percentage of RI hospitalizations discharged to PAC vs not-PAC had ≥1 CCC (94.9% vs 33.5%), including a neuromuscular CCC (57.5% vs 8.9%) or respiratory CCC (62.5% vs 12.0%), P < 0.001 for all (Table 2). A greater percentage discharged to PAC was assisted with medical technology (83.2% vs 15.1%), including respiratory technology (eg, tracheostomy; 53.8% vs 5.4%) and gastrointestinal technology (eg, gastrostomy; 71.9% vs 11.8%), P < 0.001 for all. Of the children with respiratory technology, 14.8% (n = 394) underwent tracheotomy during the acute-care hospitalization. Children discharged to PAC had a higher percentage of multiple chronic conditions. For example, the percentages of children discharged to PAC vs not with ≥7 conditions were 54.5% vs 7.0% (P < 0.001; Table 2). The most common chronic conditions experienced by children discharged to PAC included epilepsy (41.2%), gastroesophageal reflux (36.6%), cerebral palsy (28.2%), and asthma (18.2%).

 

Hospitalization Characteristics

Acute-care RI hospitalization median LOS was longer for discharges to PAC vs non-PAC (10 days [IQR 4-27] vs 2 days [IQR 1-4], P < 0.001; Table 1). A greater percentage of discharges to PAC were administered medications from multiple classes during the acute-care RI admission (eg, 54.8% vs 13.4% used medications from ≥7 classes, P < 0.001). A greater percentage of discharges to PAC used intensive care services during the acute-care admission (65.6% vs 22.4%, P < 0.001). A greater percentage of discharges to PAC received CPAP (10.6 vs 5.0%), BiPAP (19.8% vs 11.4%), or mechanical ventilation (52.7% vs 9.1%) during the acute-care RI hospitalization (P < 0.001 for all; Table 1).

Multivariable Analysis of the Likelihood of Post-Acute Care Use Following Discharge

In multivariable analysis, the patient characteristics associated with the highest likelihood of discharge to PAC included ≥11 vs no chronic conditions (odds ratio [OR] 11.8 , 95% CI, 8.0-17.2), ≥9 classes vs no classes of medications administered during the acute-care hospitalization (OR 4.8 , 95% CI, 1.8-13.0), and existing tracheostomy (OR 3.0, [95% CI, 2.6-3.5; Figure 2 and eTable). Patient characteristics associated with a more modest likelihood of discharge to PAC included public vs private insurance (OR 1.8, 95% CI, 1.6-2.0), neuromuscular complex chronic condition (OR 1.6, 95% CI, 1.5-1.8), new tracheostomy (OR 1.9, 95% CI, 1.7-2.2), and use of any enhanced respiratory support (ie, CPAP/BiPAP/mechanical ventilation) during the acute-care hospitalization (OR 1.4, 95% CI, 1.3-1.6; Figure 2 and Supplementary Table).

Classification and Regression Tree Analysis

In the CART analysis, the highest percentage (6.3%) of children hospitalized with RI who were discharged to PAC had the following combination of characteristics: ≥6 chronic conditions, ≥7 classes of medications administered, and respiratory technology. Median (IQR) length of acute-care LOS for children with these attributes who were transferred to PAC was 19 (IQR 8-56; range 1-1005) days; LOS remained long (median 13 days [IQR 6-41, range 1-1413]) for children with the same attributes not transferred to PAC (n = 9448). Between these children transferred vs not to PAC, 79.3% vs 65.9% received ICU services; 74.4% vs 73.5% received CPAP, BiPAP, or mechanical ventilation; and 31.0% vs 22.7% underwent tracheotomy during the acute-care hospitalization. Of these children who were not transferred to PAC, 18.9% were discharged to home nursing services.

DISCUSSION

The findings from the present study suggest that patients with RI hospitalization in children’s hospitals who use PAC are medically complex, with high rates of multiple chronic conditions—including cerebral palsy, asthma, chronic respiratory insufficiency, dysphagia, epilepsy, and gastroesophageal reflux—and high rates of technology assistance including enterostomy and tracheostomy. The characteristics of patients most likely to use PAC include long LOS, a large number of chronic conditions, many types of medications administered during the acute-care hospitalization, respiratory technology use, and an underlying neuromuscular condition. Specifically, the highest percentage of children hospitalized with RI who were discharged to PAC had ≥6 chronic conditions, ≥7 classes of medications administered, and respiratory technology. Our analysis suggests that there may be a large population of children with these same characteristics who experienced a prolonged LOS but were not transferred to PAC.

Physiologically, it makes sense that children hospitalized with RI who have a large number of chronic conditions rely more often on PAC for recovery of their health than other children. In our clinical experience, the most prevalent conditions experienced by these children impede their recovery from RI. For example, children’s length of hospitalization for pneumonia can be prolonged with epilepsy because the RI lowers their seizure threshold; with gastroesophageal reflux because impaired digestive motility precludes their hydration and caloric intake abilities; and with cerebral palsy (and other neuromuscular complex chronic conditions) because impaired innervation of the respiratory tract and musculature can limit the depth of respiration, airway protection, and mucus clearance.²² Addressing the cumulative effects of these comorbidities is typically a measured rather than a rapid process. This may help explain why these children had a lengthy acute-care LOS regardless of whether they were transferred to PAC. Further investigation is needed to assess whether earlier transfer to PAC—like that typically experienced by adult patients (eg, within a few days of hospital admission)—might be suited for some of these children.

There are several reasons to explain why children hospitalized with RI who rely on medical technology, such as existing tracheostomy, are more likely to use PAC. Tracheostomy often indicates the presence of life-limiting impairment in oxygenation or ventilation, thereby representing a high degree of medical fragility. Tracheostomy, in some cases, offers enhanced ability to assist with RI treatment, including establishment of airway clearance of secretions (ie, suctioning and chest physiotherapy), administration of antimicrobials (eg, nebulized antibiotics), and optimization of ventilation (eg, non-invasive positive airway pressure). However, not all acute-care inpatient clinicians have experience and clinical proficiencies in the care of children with pediatric tracheostomy.²³ As a result, a more cautious approach, with prolonged LOS and gradual arrival to hospital discharge, is often taken in the acute-care hospital setting for children with tracheostomy. Tracheostomy care delivered during recovery from RI by trained and experienced teams of providers in the PAC setting may be best positioned to help optimize respiratory health and ensure proper family education and readiness to continue care at home.⁶

Further investigation is needed of the long LOS in children not transferred to PAC who had similar characteristics to those who were transferred. In hospitalized adult patients with RI, PAC is routinely introduced early in the admission process, with anticipated transfer within a few days into the hospitalization. In the current study, LOS was nearly 2 weeks or longer in many children not transferred to PAC who had similar characteristics to those who were transferred. Perhaps some of the children not transferred experienced long LOS in the acute-care hospital because of a limited number of pediatric PAC beds in their local area. Some families of these children may have been offered but declined use of PAC. PAC may not have been offered to some because illness acuity was too high or there was lack of PAC awareness as a possible setting for recovery.

There are several limitations to this study. PHIS does not contain non-freestanding children’s hospitals; therefore, the study results may generalize best to children’s hospitals. PHIS does not contain information on the amount (eg, number of days used), cost, or treatments provided in PAC. Therefore, we were unable to determine the true reasons why children used PAC services following RI hospitalization (eg, for respiratory rehabilitation vs other reasons, such as epilepsy or nutrition/hydration management). Moreover, we could not assess which children truly used PAC for short-term recovery vs longer-term care because they were unable to reside at home (eg, they were too medically complex). We were unable to assess PAC availability (eg, number of beds) in the surrounding areas of the acute-care hospitals in the PHIS database. Although we assessed use of medical technology, PHIS does not contain data on functional status or activities of daily living, which correlate with the use of PAC in adults. We could not distinguish whether children receiving BiPAP, CPAP, or mechanical ventilation during hospitalization were using it chronically. Although higher PAC use was associated with public insurance, due to absent information on the children’s home, family, and social environment, we were unable to assess whether PAC use was influenced by limited caregiving support or resources.

Data on the type and number of chronic conditions are limited by the ICD-9-CM codes available to distinguish them. Although several patient demographic and clinical characteristics were significantly associated with the use of PAC, significance may have occurred because of the large sample size and consequent robust statistical power. This is why we elected to highlight and discuss the characteristics with the strongest and most clinically meaningful associations (eg, multiple chronic conditions). There may be additional characteristics, including social, familial, and community resources, that are not available to assess in PHIS that could have affected PAC use.

Despite these limitations, the current study suggests that the characteristics of children hospitalized with RI who use PAC for recovery are evident and that there is a large population of children with these characteristics who experienced a prolonged LOS that did not result in transfer to PAC. These findings could be used in subsequent studies to help create the base of a matched cohort of children with similar clinical, demographic, and hospitalization characteristics who used vs didn’t use PAC. Comparison of the functional status, health trajectory, and family and/or social attributes of these 2 groups of children, as well as their post-discharge outcomes and utilization (eg, length of PAC stay, emergency department revisits, and acute-care hospital readmissions), could occur with chart review, clinician and parent interview, and other methods. This body of work might ultimately lead to an assessment of value in PAC and potentially help us understand the need for PAC capacity in various communities. In the meantime, clinicians may find it useful to consider the results of the current study when contemplating PAC use in their hospitalized children with RI, including exploration of health system opportunities of clinical collaboration between acute-care children’s hospitals and PAC facilities. Ultimately, all of this work will generate meaningful knowledge regarding the most appropriate, safe, and cost-effective settings for hospitalized children with RI to regain their health.

 

Acknowledgments

Dr. Berry was supported by the Agency for Healthcare Research and Quality (R21 HS023092-01), the Lucile Packard Foundation for Children’s Health, and Franciscan Hospital for Children. The funders were not involved in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclosure: The authors have no financial relationships relevant to this article to disclose.

Respiratory illness (RI) is one of the most common reasons for pediatric hospitalization.¹ Examples of RI include acute illness, such as bronchiolitis, bacterial pneumonia, and asthma, as well as chronic conditions, such as obstructive sleep apnea and chronic respiratory insufficiency. Hospital care for RI includes monitoring and treatment to optimize oxygenation, ventilation, hydration, and other body functions. Most previously healthy children hospitalized with RI stay in the hospital for a limited duration (eg, a few days) because the severity of their illness is short lived and they quickly return to their previous healthy status.² However, hospital care is increasing for children with fragile and tenuous health due to complex medical conditions.³ RI is a common reason for hospitalization among these children as well and recovery of respiratory health and function can be slow and protracted for some of them.⁴ Weeks, months, or longer periods of time may be necessary for the children to return to their previous respiratory baseline health and function after hospital discharge; other children may not return to their baseline.^5,6

Hospitalized older adults with high-severity RI are routinely streamlined for transfer to post-acute facility care (PAC) shortly (eg, a few days) after acute-care hospitalization. Nearly 70% of elderly Medicare beneficiaries use PAC following a brief length of stay (LOS) in the acute-care hospital.⁷ It is believed that PAC helps optimize the patients’ health and functional status and relieves the family caregiving burden that would have occurred at home.^8-10 PAC use also helps to shorten acute-care hospitalization for RI while avoiding readmission.^8-10 In contrast with adult patients, use of PAC for hospitalized children is not routine.¹¹ While PAC use in children is infrequent, RI is one of the most common reasons for acute admission among children who use it.¹²

For some children with RI, PAC might be positioned to offer a safe, therapeutic, and high-value setting for pulmonary rehabilitation, as well as related medical, nutritional, functional, and family cares.⁶ PAC, by design, could possibly help some of the children transition back into their homes and communities. As studies continue to emerge that assess the value of PAC in children, it is important to learn more about the use of PAC in children hospitalized with RI. The objectives were to (1) assess which children admitted with RI are the most likely to use PAC services for recovery and (2) estimate how many hospitalized children not using PAC had the same characteristics as those who did.

METHODS

Study Design, Setting, and Population

We conducted a retrospective cohort analysis of 609,800 hospitalizations for RI occurring from January 1, 2010 to December 31, 2015, in 43 freestanding children’s hospitals in the Pediatric Health Information Systems (PHIS) dataset. All hospitals participating in PHIS are members of the Children’s Hospital Association.¹³ The Boston Children’s Hospital Institutional Review Board approved this study with a waiver for informed consent.

RI was identified using the Agency for Healthcare Research and Quality (AHRQ) Clinical Classification System (CCS).¹⁴ Using diagnosis CCS category 8 (“Diseases of the Respiratory System”) and the procedure CCS category 6 (“Operations on the Respiratory System”), we identified all hospitalizations from the participating hospitals with a principal diagnosis or procedure International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) code for an RI.

Main Outcome Measure

Discharge disposition following the acute-care hospitalization for RI was the main outcome measure. We used PHIS uniform disposition coding to classify the discharge disposition as transfer to PAC (ie, rehabilitation facility, skilled nursing facility, etc.) vs all other dispositions (ie, routine to home, against medical advice, etc.).¹² The PAC disposition category was derived from the Centers for Medicare & Medicaid Services Patient Discharge Status Codes and Hospital Transfer Policies as informed by the National Uniform Billing Committee Official UB-04 Data Specifications Manual, 2008. PAC transfer included disposition to external PAC facilities, as well as to internal, embedded PAC units residing in a few of the acute-care children’s hospitals included in the cohort.

 

Demographic and Clinical Characteristics

We assessed patient demographic and clinical characteristics that might correlate with PAC use following acute-care hospitalization for RI. Demographic characteristics included gender, age at admission in years, payer (public, private, and other), and race/ethnicity (Hispanic, non-Hispanic black, non-Hispanic white, other).

Clinical characteristics included chronic conditions (type and number) and assistance with medical technology. Chronic condition and medical technology characteristics were assessed with ICD-9-CM diagnosis codes. PHIS contain up to 41 ICD-9-CM diagnosis codes per hospital discharge record. To identify the presence and number of chronic conditions, we used the AHRQ Chronic Condition Indicator system, which categorizes over 14,000 ICD-9-CM diagnosis codes into chronic vs non-chronic conditions.^14,15 Children hospitalized with a chronic condition were further classified as having a complex chronic condition (CCC) using Feudtner and colleagues’ ICD-9-CM diagnosis classification scheme.¹⁶ CCCs represent defined diagnosis groupings expected to last longer than 12 months and involving either a single organ system, severe enough to require specialty pediatric care and hospitalization, or multiple organ systems.^17,18 Hospitalized children who were assisted with medical technology were identified with ICD-9-CM codes indicating the use of a medical device to manage and treat a chronic illness (eg, ventricular shunt to treat hydrocephalus) or to maintain basic body functions necessary for sustaining life (eg, a tracheostomy tube for breathing).^19,20 We distinguished children undergoing tracheotomy during hospitalization using ICD-9-CM procedure codes 31.1 and 31.2.

Acute-Care Hospitalization Characteristics

We also assessed the relationship between acute-care hospitalization characteristics and use of PAC after discharge, including US census region, LOS, use of intensive care, number of medication classes administered, and use of enhanced respiratory support. Enhanced respiratory support was defined as use of continuous or bilevel positive airway pressure (CPAP or BiPAP) or mechanical ventilation during the acute-care hospitalization for RI. These respiratory supports were identified using billing data in PHIS.

Statistical Analysis

In bivariable analysis, we compared demographic, clinical, and hospitalization characteristics of hospitalized children with vs without discharge to PAC using Rao-Scott chi-square tests and Wilcoxon rank-sum tests as appropriate. In multivariable analysis, we derived a generalized linear mix effects model with fixed effects for demographic, clinical, and hospitalization characteristics that were associated with PAC at P < 0.1 in bivariable analysis (ie, age, gender, race/ethnicity, payer, medical technology, use of intensive care unit [ICU], use of positive pressure or mechanical ventilation, hospital region, LOS, new tracheostomy, existing tracheostomy, other technologies, number of medications, number of chronic conditions [of any complexity], and type of complex chronic conditions). We controlled for clustering of patients within hospitals by including a random intercept for each hospital. We also assessed combinations of patient characteristics on the likelihood of PAC use with classification and regression tree (CART) modeling. Using CART, we determined which characteristic combinations were associated with the highest and lowest use of PAC using binary split and post-pruning, goodness of fit rules.²¹ All statistical analyses were performed using SAS v.9.4 (SAS Institute, Cary, NC), and R v.3.2 (R Foundation for Statistical Computing, Vienna, Austria) using the “party” package. The threshold for statistical significance was set at P < 0.05.

RESULTS

Of the 609,800 hospitalizations for RI, PAC use after discharge occurred for 2660 (0.4%). RI discharges to PAC accounted for 2.1% (n = 67,405) of hospital days and 2.7% ($280 million) of hospital cost of all RI hospitalizations. For discharges to PAC, the most common RI were pneumonia (29.1% [n = 773]), respiratory failure or insufficiency (unspecified reason; 22.0% [n = 584]), and upper respiratory infection (12.2% [n = 323]).

Demographic Characteristics

Median age at acute-care admission was higher for PAC vs non-PAC discharges (6 years [interquartile range {IQR} 1-15] vs 2 years [0-7], P < 0.001; Table 1). Hispanic patients accounted for a smaller percentage of RI discharges to PAC vs non-PAC (14.1% vs 21.8%, P < 0.001) and a higher percentage to PAC were for patients with public insurance (75.9% vs 62.5, P < 0.001; Table 1).

Clinical Characteristics

A greater percentage of RI hospitalizations discharged to PAC vs not-PAC had ≥1 CCC (94.9% vs 33.5%), including a neuromuscular CCC (57.5% vs 8.9%) or respiratory CCC (62.5% vs 12.0%), P < 0.001 for all (Table 2). A greater percentage discharged to PAC was assisted with medical technology (83.2% vs 15.1%), including respiratory technology (eg, tracheostomy; 53.8% vs 5.4%) and gastrointestinal technology (eg, gastrostomy; 71.9% vs 11.8%), P < 0.001 for all. Of the children with respiratory technology, 14.8% (n = 394) underwent tracheotomy during the acute-care hospitalization. Children discharged to PAC had a higher percentage of multiple chronic conditions. For example, the percentages of children discharged to PAC vs not with ≥7 conditions were 54.5% vs 7.0% (P < 0.001; Table 2). The most common chronic conditions experienced by children discharged to PAC included epilepsy (41.2%), gastroesophageal reflux (36.6%), cerebral palsy (28.2%), and asthma (18.2%).

 

Hospitalization Characteristics

Acute-care RI hospitalization median LOS was longer for discharges to PAC vs non-PAC (10 days [IQR 4-27] vs 2 days [IQR 1-4], P < 0.001; Table 1). A greater percentage of discharges to PAC were administered medications from multiple classes during the acute-care RI admission (eg, 54.8% vs 13.4% used medications from ≥7 classes, P < 0.001). A greater percentage of discharges to PAC used intensive care services during the acute-care admission (65.6% vs 22.4%, P < 0.001). A greater percentage of discharges to PAC received CPAP (10.6 vs 5.0%), BiPAP (19.8% vs 11.4%), or mechanical ventilation (52.7% vs 9.1%) during the acute-care RI hospitalization (P < 0.001 for all; Table 1).

Multivariable Analysis of the Likelihood of Post-Acute Care Use Following Discharge

In multivariable analysis, the patient characteristics associated with the highest likelihood of discharge to PAC included ≥11 vs no chronic conditions (odds ratio [OR] 11.8 , 95% CI, 8.0-17.2), ≥9 classes vs no classes of medications administered during the acute-care hospitalization (OR 4.8 , 95% CI, 1.8-13.0), and existing tracheostomy (OR 3.0, [95% CI, 2.6-3.5; Figure 2 and eTable). Patient characteristics associated with a more modest likelihood of discharge to PAC included public vs private insurance (OR 1.8, 95% CI, 1.6-2.0), neuromuscular complex chronic condition (OR 1.6, 95% CI, 1.5-1.8), new tracheostomy (OR 1.9, 95% CI, 1.7-2.2), and use of any enhanced respiratory support (ie, CPAP/BiPAP/mechanical ventilation) during the acute-care hospitalization (OR 1.4, 95% CI, 1.3-1.6; Figure 2 and Supplementary Table).

Classification and Regression Tree Analysis

In the CART analysis, the highest percentage (6.3%) of children hospitalized with RI who were discharged to PAC had the following combination of characteristics: ≥6 chronic conditions, ≥7 classes of medications administered, and respiratory technology. Median (IQR) length of acute-care LOS for children with these attributes who were transferred to PAC was 19 (IQR 8-56; range 1-1005) days; LOS remained long (median 13 days [IQR 6-41, range 1-1413]) for children with the same attributes not transferred to PAC (n = 9448). Between these children transferred vs not to PAC, 79.3% vs 65.9% received ICU services; 74.4% vs 73.5% received CPAP, BiPAP, or mechanical ventilation; and 31.0% vs 22.7% underwent tracheotomy during the acute-care hospitalization. Of these children who were not transferred to PAC, 18.9% were discharged to home nursing services.

DISCUSSION

The findings from the present study suggest that patients with RI hospitalization in children’s hospitals who use PAC are medically complex, with high rates of multiple chronic conditions—including cerebral palsy, asthma, chronic respiratory insufficiency, dysphagia, epilepsy, and gastroesophageal reflux—and high rates of technology assistance including enterostomy and tracheostomy. The characteristics of patients most likely to use PAC include long LOS, a large number of chronic conditions, many types of medications administered during the acute-care hospitalization, respiratory technology use, and an underlying neuromuscular condition. Specifically, the highest percentage of children hospitalized with RI who were discharged to PAC had ≥6 chronic conditions, ≥7 classes of medications administered, and respiratory technology. Our analysis suggests that there may be a large population of children with these same characteristics who experienced a prolonged LOS but were not transferred to PAC.

Physiologically, it makes sense that children hospitalized with RI who have a large number of chronic conditions rely more often on PAC for recovery of their health than other children. In our clinical experience, the most prevalent conditions experienced by these children impede their recovery from RI. For example, children’s length of hospitalization for pneumonia can be prolonged with epilepsy because the RI lowers their seizure threshold; with gastroesophageal reflux because impaired digestive motility precludes their hydration and caloric intake abilities; and with cerebral palsy (and other neuromuscular complex chronic conditions) because impaired innervation of the respiratory tract and musculature can limit the depth of respiration, airway protection, and mucus clearance.²² Addressing the cumulative effects of these comorbidities is typically a measured rather than a rapid process. This may help explain why these children had a lengthy acute-care LOS regardless of whether they were transferred to PAC. Further investigation is needed to assess whether earlier transfer to PAC—like that typically experienced by adult patients (eg, within a few days of hospital admission)—might be suited for some of these children.

There are several reasons to explain why children hospitalized with RI who rely on medical technology, such as existing tracheostomy, are more likely to use PAC. Tracheostomy often indicates the presence of life-limiting impairment in oxygenation or ventilation, thereby representing a high degree of medical fragility. Tracheostomy, in some cases, offers enhanced ability to assist with RI treatment, including establishment of airway clearance of secretions (ie, suctioning and chest physiotherapy), administration of antimicrobials (eg, nebulized antibiotics), and optimization of ventilation (eg, non-invasive positive airway pressure). However, not all acute-care inpatient clinicians have experience and clinical proficiencies in the care of children with pediatric tracheostomy.²³ As a result, a more cautious approach, with prolonged LOS and gradual arrival to hospital discharge, is often taken in the acute-care hospital setting for children with tracheostomy. Tracheostomy care delivered during recovery from RI by trained and experienced teams of providers in the PAC setting may be best positioned to help optimize respiratory health and ensure proper family education and readiness to continue care at home.⁶

Further investigation is needed of the long LOS in children not transferred to PAC who had similar characteristics to those who were transferred. In hospitalized adult patients with RI, PAC is routinely introduced early in the admission process, with anticipated transfer within a few days into the hospitalization. In the current study, LOS was nearly 2 weeks or longer in many children not transferred to PAC who had similar characteristics to those who were transferred. Perhaps some of the children not transferred experienced long LOS in the acute-care hospital because of a limited number of pediatric PAC beds in their local area. Some families of these children may have been offered but declined use of PAC. PAC may not have been offered to some because illness acuity was too high or there was lack of PAC awareness as a possible setting for recovery.

There are several limitations to this study. PHIS does not contain non-freestanding children’s hospitals; therefore, the study results may generalize best to children’s hospitals. PHIS does not contain information on the amount (eg, number of days used), cost, or treatments provided in PAC. Therefore, we were unable to determine the true reasons why children used PAC services following RI hospitalization (eg, for respiratory rehabilitation vs other reasons, such as epilepsy or nutrition/hydration management). Moreover, we could not assess which children truly used PAC for short-term recovery vs longer-term care because they were unable to reside at home (eg, they were too medically complex). We were unable to assess PAC availability (eg, number of beds) in the surrounding areas of the acute-care hospitals in the PHIS database. Although we assessed use of medical technology, PHIS does not contain data on functional status or activities of daily living, which correlate with the use of PAC in adults. We could not distinguish whether children receiving BiPAP, CPAP, or mechanical ventilation during hospitalization were using it chronically. Although higher PAC use was associated with public insurance, due to absent information on the children’s home, family, and social environment, we were unable to assess whether PAC use was influenced by limited caregiving support or resources.

Data on the type and number of chronic conditions are limited by the ICD-9-CM codes available to distinguish them. Although several patient demographic and clinical characteristics were significantly associated with the use of PAC, significance may have occurred because of the large sample size and consequent robust statistical power. This is why we elected to highlight and discuss the characteristics with the strongest and most clinically meaningful associations (eg, multiple chronic conditions). There may be additional characteristics, including social, familial, and community resources, that are not available to assess in PHIS that could have affected PAC use.

Despite these limitations, the current study suggests that the characteristics of children hospitalized with RI who use PAC for recovery are evident and that there is a large population of children with these characteristics who experienced a prolonged LOS that did not result in transfer to PAC. These findings could be used in subsequent studies to help create the base of a matched cohort of children with similar clinical, demographic, and hospitalization characteristics who used vs didn’t use PAC. Comparison of the functional status, health trajectory, and family and/or social attributes of these 2 groups of children, as well as their post-discharge outcomes and utilization (eg, length of PAC stay, emergency department revisits, and acute-care hospital readmissions), could occur with chart review, clinician and parent interview, and other methods. This body of work might ultimately lead to an assessment of value in PAC and potentially help us understand the need for PAC capacity in various communities. In the meantime, clinicians may find it useful to consider the results of the current study when contemplating PAC use in their hospitalized children with RI, including exploration of health system opportunities of clinical collaboration between acute-care children’s hospitals and PAC facilities. Ultimately, all of this work will generate meaningful knowledge regarding the most appropriate, safe, and cost-effective settings for hospitalized children with RI to regain their health.

 

Acknowledgments

Dr. Berry was supported by the Agency for Healthcare Research and Quality (R21 HS023092-01), the Lucile Packard Foundation for Children’s Health, and Franciscan Hospital for Children. The funders were not involved in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclosure: The authors have no financial relationships relevant to this article to disclose.

Respiratory illness (RI) is one of the most common reasons for pediatric hospitalization.¹ Examples of RI include acute illness, such as bronchiolitis, bacterial pneumonia, and asthma, as well as chronic conditions, such as obstructive sleep apnea and chronic respiratory insufficiency. Hospital care for RI includes monitoring and treatment to optimize oxygenation, ventilation, hydration, and other body functions. Most previously healthy children hospitalized with RI stay in the hospital for a limited duration (eg, a few days) because the severity of their illness is short lived and they quickly return to their previous healthy status.² However, hospital care is increasing for children with fragile and tenuous health due to complex medical conditions.³ RI is a common reason for hospitalization among these children as well and recovery of respiratory health and function can be slow and protracted for some of them.⁴ Weeks, months, or longer periods of time may be necessary for the children to return to their previous respiratory baseline health and function after hospital discharge; other children may not return to their baseline.^5,6

Hospitalized older adults with high-severity RI are routinely streamlined for transfer to post-acute facility care (PAC) shortly (eg, a few days) after acute-care hospitalization. Nearly 70% of elderly Medicare beneficiaries use PAC following a brief length of stay (LOS) in the acute-care hospital.⁷ It is believed that PAC helps optimize the patients’ health and functional status and relieves the family caregiving burden that would have occurred at home.^8-10 PAC use also helps to shorten acute-care hospitalization for RI while avoiding readmission.^8-10 In contrast with adult patients, use of PAC for hospitalized children is not routine.¹¹ While PAC use in children is infrequent, RI is one of the most common reasons for acute admission among children who use it.¹²

For some children with RI, PAC might be positioned to offer a safe, therapeutic, and high-value setting for pulmonary rehabilitation, as well as related medical, nutritional, functional, and family cares.⁶ PAC, by design, could possibly help some of the children transition back into their homes and communities. As studies continue to emerge that assess the value of PAC in children, it is important to learn more about the use of PAC in children hospitalized with RI. The objectives were to (1) assess which children admitted with RI are the most likely to use PAC services for recovery and (2) estimate how many hospitalized children not using PAC had the same characteristics as those who did.

METHODS

Study Design, Setting, and Population

We conducted a retrospective cohort analysis of 609,800 hospitalizations for RI occurring from January 1, 2010 to December 31, 2015, in 43 freestanding children’s hospitals in the Pediatric Health Information Systems (PHIS) dataset. All hospitals participating in PHIS are members of the Children’s Hospital Association.¹³ The Boston Children’s Hospital Institutional Review Board approved this study with a waiver for informed consent.

RI was identified using the Agency for Healthcare Research and Quality (AHRQ) Clinical Classification System (CCS).¹⁴ Using diagnosis CCS category 8 (“Diseases of the Respiratory System”) and the procedure CCS category 6 (“Operations on the Respiratory System”), we identified all hospitalizations from the participating hospitals with a principal diagnosis or procedure International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) code for an RI.

Main Outcome Measure

Discharge disposition following the acute-care hospitalization for RI was the main outcome measure. We used PHIS uniform disposition coding to classify the discharge disposition as transfer to PAC (ie, rehabilitation facility, skilled nursing facility, etc.) vs all other dispositions (ie, routine to home, against medical advice, etc.).¹² The PAC disposition category was derived from the Centers for Medicare & Medicaid Services Patient Discharge Status Codes and Hospital Transfer Policies as informed by the National Uniform Billing Committee Official UB-04 Data Specifications Manual, 2008. PAC transfer included disposition to external PAC facilities, as well as to internal, embedded PAC units residing in a few of the acute-care children’s hospitals included in the cohort.

 

Demographic and Clinical Characteristics

We assessed patient demographic and clinical characteristics that might correlate with PAC use following acute-care hospitalization for RI. Demographic characteristics included gender, age at admission in years, payer (public, private, and other), and race/ethnicity (Hispanic, non-Hispanic black, non-Hispanic white, other).

Clinical characteristics included chronic conditions (type and number) and assistance with medical technology. Chronic condition and medical technology characteristics were assessed with ICD-9-CM diagnosis codes. PHIS contain up to 41 ICD-9-CM diagnosis codes per hospital discharge record. To identify the presence and number of chronic conditions, we used the AHRQ Chronic Condition Indicator system, which categorizes over 14,000 ICD-9-CM diagnosis codes into chronic vs non-chronic conditions.^14,15 Children hospitalized with a chronic condition were further classified as having a complex chronic condition (CCC) using Feudtner and colleagues’ ICD-9-CM diagnosis classification scheme.¹⁶ CCCs represent defined diagnosis groupings expected to last longer than 12 months and involving either a single organ system, severe enough to require specialty pediatric care and hospitalization, or multiple organ systems.^17,18 Hospitalized children who were assisted with medical technology were identified with ICD-9-CM codes indicating the use of a medical device to manage and treat a chronic illness (eg, ventricular shunt to treat hydrocephalus) or to maintain basic body functions necessary for sustaining life (eg, a tracheostomy tube for breathing).^19,20 We distinguished children undergoing tracheotomy during hospitalization using ICD-9-CM procedure codes 31.1 and 31.2.

Acute-Care Hospitalization Characteristics

We also assessed the relationship between acute-care hospitalization characteristics and use of PAC after discharge, including US census region, LOS, use of intensive care, number of medication classes administered, and use of enhanced respiratory support. Enhanced respiratory support was defined as use of continuous or bilevel positive airway pressure (CPAP or BiPAP) or mechanical ventilation during the acute-care hospitalization for RI. These respiratory supports were identified using billing data in PHIS.

Statistical Analysis

In bivariable analysis, we compared demographic, clinical, and hospitalization characteristics of hospitalized children with vs without discharge to PAC using Rao-Scott chi-square tests and Wilcoxon rank-sum tests as appropriate. In multivariable analysis, we derived a generalized linear mix effects model with fixed effects for demographic, clinical, and hospitalization characteristics that were associated with PAC at P < 0.1 in bivariable analysis (ie, age, gender, race/ethnicity, payer, medical technology, use of intensive care unit [ICU], use of positive pressure or mechanical ventilation, hospital region, LOS, new tracheostomy, existing tracheostomy, other technologies, number of medications, number of chronic conditions [of any complexity], and type of complex chronic conditions). We controlled for clustering of patients within hospitals by including a random intercept for each hospital. We also assessed combinations of patient characteristics on the likelihood of PAC use with classification and regression tree (CART) modeling. Using CART, we determined which characteristic combinations were associated with the highest and lowest use of PAC using binary split and post-pruning, goodness of fit rules.²¹ All statistical analyses were performed using SAS v.9.4 (SAS Institute, Cary, NC), and R v.3.2 (R Foundation for Statistical Computing, Vienna, Austria) using the “party” package. The threshold for statistical significance was set at P < 0.05.

RESULTS

Of the 609,800 hospitalizations for RI, PAC use after discharge occurred for 2660 (0.4%). RI discharges to PAC accounted for 2.1% (n = 67,405) of hospital days and 2.7% ($280 million) of hospital cost of all RI hospitalizations. For discharges to PAC, the most common RI were pneumonia (29.1% [n = 773]), respiratory failure or insufficiency (unspecified reason; 22.0% [n = 584]), and upper respiratory infection (12.2% [n = 323]).

Demographic Characteristics

Median age at acute-care admission was higher for PAC vs non-PAC discharges (6 years [interquartile range {IQR} 1-15] vs 2 years [0-7], P < 0.001; Table 1). Hispanic patients accounted for a smaller percentage of RI discharges to PAC vs non-PAC (14.1% vs 21.8%, P < 0.001) and a higher percentage to PAC were for patients with public insurance (75.9% vs 62.5, P < 0.001; Table 1).

Clinical Characteristics

A greater percentage of RI hospitalizations discharged to PAC vs not-PAC had ≥1 CCC (94.9% vs 33.5%), including a neuromuscular CCC (57.5% vs 8.9%) or respiratory CCC (62.5% vs 12.0%), P < 0.001 for all (Table 2). A greater percentage discharged to PAC was assisted with medical technology (83.2% vs 15.1%), including respiratory technology (eg, tracheostomy; 53.8% vs 5.4%) and gastrointestinal technology (eg, gastrostomy; 71.9% vs 11.8%), P < 0.001 for all. Of the children with respiratory technology, 14.8% (n = 394) underwent tracheotomy during the acute-care hospitalization. Children discharged to PAC had a higher percentage of multiple chronic conditions. For example, the percentages of children discharged to PAC vs not with ≥7 conditions were 54.5% vs 7.0% (P < 0.001; Table 2). The most common chronic conditions experienced by children discharged to PAC included epilepsy (41.2%), gastroesophageal reflux (36.6%), cerebral palsy (28.2%), and asthma (18.2%).

 

Hospitalization Characteristics

Acute-care RI hospitalization median LOS was longer for discharges to PAC vs non-PAC (10 days [IQR 4-27] vs 2 days [IQR 1-4], P < 0.001; Table 1). A greater percentage of discharges to PAC were administered medications from multiple classes during the acute-care RI admission (eg, 54.8% vs 13.4% used medications from ≥7 classes, P < 0.001). A greater percentage of discharges to PAC used intensive care services during the acute-care admission (65.6% vs 22.4%, P < 0.001). A greater percentage of discharges to PAC received CPAP (10.6 vs 5.0%), BiPAP (19.8% vs 11.4%), or mechanical ventilation (52.7% vs 9.1%) during the acute-care RI hospitalization (P < 0.001 for all; Table 1).

Multivariable Analysis of the Likelihood of Post-Acute Care Use Following Discharge

In multivariable analysis, the patient characteristics associated with the highest likelihood of discharge to PAC included ≥11 vs no chronic conditions (odds ratio [OR] 11.8 , 95% CI, 8.0-17.2), ≥9 classes vs no classes of medications administered during the acute-care hospitalization (OR 4.8 , 95% CI, 1.8-13.0), and existing tracheostomy (OR 3.0, [95% CI, 2.6-3.5; Figure 2 and eTable). Patient characteristics associated with a more modest likelihood of discharge to PAC included public vs private insurance (OR 1.8, 95% CI, 1.6-2.0), neuromuscular complex chronic condition (OR 1.6, 95% CI, 1.5-1.8), new tracheostomy (OR 1.9, 95% CI, 1.7-2.2), and use of any enhanced respiratory support (ie, CPAP/BiPAP/mechanical ventilation) during the acute-care hospitalization (OR 1.4, 95% CI, 1.3-1.6; Figure 2 and Supplementary Table).

Classification and Regression Tree Analysis

In the CART analysis, the highest percentage (6.3%) of children hospitalized with RI who were discharged to PAC had the following combination of characteristics: ≥6 chronic conditions, ≥7 classes of medications administered, and respiratory technology. Median (IQR) length of acute-care LOS for children with these attributes who were transferred to PAC was 19 (IQR 8-56; range 1-1005) days; LOS remained long (median 13 days [IQR 6-41, range 1-1413]) for children with the same attributes not transferred to PAC (n = 9448). Between these children transferred vs not to PAC, 79.3% vs 65.9% received ICU services; 74.4% vs 73.5% received CPAP, BiPAP, or mechanical ventilation; and 31.0% vs 22.7% underwent tracheotomy during the acute-care hospitalization. Of these children who were not transferred to PAC, 18.9% were discharged to home nursing services.

DISCUSSION

The findings from the present study suggest that patients with RI hospitalization in children’s hospitals who use PAC are medically complex, with high rates of multiple chronic conditions—including cerebral palsy, asthma, chronic respiratory insufficiency, dysphagia, epilepsy, and gastroesophageal reflux—and high rates of technology assistance including enterostomy and tracheostomy. The characteristics of patients most likely to use PAC include long LOS, a large number of chronic conditions, many types of medications administered during the acute-care hospitalization, respiratory technology use, and an underlying neuromuscular condition. Specifically, the highest percentage of children hospitalized with RI who were discharged to PAC had ≥6 chronic conditions, ≥7 classes of medications administered, and respiratory technology. Our analysis suggests that there may be a large population of children with these same characteristics who experienced a prolonged LOS but were not transferred to PAC.

Physiologically, it makes sense that children hospitalized with RI who have a large number of chronic conditions rely more often on PAC for recovery of their health than other children. In our clinical experience, the most prevalent conditions experienced by these children impede their recovery from RI. For example, children’s length of hospitalization for pneumonia can be prolonged with epilepsy because the RI lowers their seizure threshold; with gastroesophageal reflux because impaired digestive motility precludes their hydration and caloric intake abilities; and with cerebral palsy (and other neuromuscular complex chronic conditions) because impaired innervation of the respiratory tract and musculature can limit the depth of respiration, airway protection, and mucus clearance.²² Addressing the cumulative effects of these comorbidities is typically a measured rather than a rapid process. This may help explain why these children had a lengthy acute-care LOS regardless of whether they were transferred to PAC. Further investigation is needed to assess whether earlier transfer to PAC—like that typically experienced by adult patients (eg, within a few days of hospital admission)—might be suited for some of these children.

There are several reasons to explain why children hospitalized with RI who rely on medical technology, such as existing tracheostomy, are more likely to use PAC. Tracheostomy often indicates the presence of life-limiting impairment in oxygenation or ventilation, thereby representing a high degree of medical fragility. Tracheostomy, in some cases, offers enhanced ability to assist with RI treatment, including establishment of airway clearance of secretions (ie, suctioning and chest physiotherapy), administration of antimicrobials (eg, nebulized antibiotics), and optimization of ventilation (eg, non-invasive positive airway pressure). However, not all acute-care inpatient clinicians have experience and clinical proficiencies in the care of children with pediatric tracheostomy.²³ As a result, a more cautious approach, with prolonged LOS and gradual arrival to hospital discharge, is often taken in the acute-care hospital setting for children with tracheostomy. Tracheostomy care delivered during recovery from RI by trained and experienced teams of providers in the PAC setting may be best positioned to help optimize respiratory health and ensure proper family education and readiness to continue care at home.⁶

Further investigation is needed of the long LOS in children not transferred to PAC who had similar characteristics to those who were transferred. In hospitalized adult patients with RI, PAC is routinely introduced early in the admission process, with anticipated transfer within a few days into the hospitalization. In the current study, LOS was nearly 2 weeks or longer in many children not transferred to PAC who had similar characteristics to those who were transferred. Perhaps some of the children not transferred experienced long LOS in the acute-care hospital because of a limited number of pediatric PAC beds in their local area. Some families of these children may have been offered but declined use of PAC. PAC may not have been offered to some because illness acuity was too high or there was lack of PAC awareness as a possible setting for recovery.

There are several limitations to this study. PHIS does not contain non-freestanding children’s hospitals; therefore, the study results may generalize best to children’s hospitals. PHIS does not contain information on the amount (eg, number of days used), cost, or treatments provided in PAC. Therefore, we were unable to determine the true reasons why children used PAC services following RI hospitalization (eg, for respiratory rehabilitation vs other reasons, such as epilepsy or nutrition/hydration management). Moreover, we could not assess which children truly used PAC for short-term recovery vs longer-term care because they were unable to reside at home (eg, they were too medically complex). We were unable to assess PAC availability (eg, number of beds) in the surrounding areas of the acute-care hospitals in the PHIS database. Although we assessed use of medical technology, PHIS does not contain data on functional status or activities of daily living, which correlate with the use of PAC in adults. We could not distinguish whether children receiving BiPAP, CPAP, or mechanical ventilation during hospitalization were using it chronically. Although higher PAC use was associated with public insurance, due to absent information on the children’s home, family, and social environment, we were unable to assess whether PAC use was influenced by limited caregiving support or resources.

Data on the type and number of chronic conditions are limited by the ICD-9-CM codes available to distinguish them. Although several patient demographic and clinical characteristics were significantly associated with the use of PAC, significance may have occurred because of the large sample size and consequent robust statistical power. This is why we elected to highlight and discuss the characteristics with the strongest and most clinically meaningful associations (eg, multiple chronic conditions). There may be additional characteristics, including social, familial, and community resources, that are not available to assess in PHIS that could have affected PAC use.

Despite these limitations, the current study suggests that the characteristics of children hospitalized with RI who use PAC for recovery are evident and that there is a large population of children with these characteristics who experienced a prolonged LOS that did not result in transfer to PAC. These findings could be used in subsequent studies to help create the base of a matched cohort of children with similar clinical, demographic, and hospitalization characteristics who used vs didn’t use PAC. Comparison of the functional status, health trajectory, and family and/or social attributes of these 2 groups of children, as well as their post-discharge outcomes and utilization (eg, length of PAC stay, emergency department revisits, and acute-care hospital readmissions), could occur with chart review, clinician and parent interview, and other methods. This body of work might ultimately lead to an assessment of value in PAC and potentially help us understand the need for PAC capacity in various communities. In the meantime, clinicians may find it useful to consider the results of the current study when contemplating PAC use in their hospitalized children with RI, including exploration of health system opportunities of clinical collaboration between acute-care children’s hospitals and PAC facilities. Ultimately, all of this work will generate meaningful knowledge regarding the most appropriate, safe, and cost-effective settings for hospitalized children with RI to regain their health.

 

Acknowledgments

Dr. Berry was supported by the Agency for Healthcare Research and Quality (R21 HS023092-01), the Lucile Packard Foundation for Children’s Health, and Franciscan Hospital for Children. The funders were not involved in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclosure: The authors have no financial relationships relevant to this article to disclose.

Given growing incentives to reduce readmission rates, predischarge checklists and bundles have recommended that inpatient providers schedule postdischarge follow-up visits (PDFVs) for their hospitalized patients.^1-4 PDFVs have been linked to lower readmission rates in patients with chronic conditions, including congestive heart failure, psychiatric illnesses, and chronic obstructive pulmonary disease.^5-8 In contrast, the impact of PDFVs on readmissions in hospitalized general medicine populations has been mixed.^9-12 Beyond the presence or absence of PDFVs, it may be a patient’s inability to keep scheduled PDFVs that contributes more strongly to preventable readmissions.¹¹

This challenge, dealing with the 12% to 37% of patients who miss their visits (“no-shows”), is not new.^13-17 In high-risk patient populations, such as those with substance abuse, diabetes, or human immunodeficiency virus, no-shows (NSs) have been linked to poorer short-term and long-term clinical outcomes.^16,18-20 Additionally, NSs pose a challenge for outpatient clinics and the healthcare system at large. The financial cost of NSs ranges from approximately $200 per patient in 2 analyses to $7 million in cumulative lost revenue per year at 1 large academic health system.^13,17,21 As such, increasing attendance at PDFVs is a potential target for improving both patient outcomes and clinic productivity.

Most prior PDFV research has focused on readmission risk rather than PDFV attendance as the primary outcome.^5-12 However, given the patient-oriented benefits of attending PDFVs and the clinic-oriented benefits of avoiding vacant time slots, NS PDFVs represent an important missed opportunity for our healthcare delivery system. To our knowledge, risk factors for PDFV nonattendance have not yet been systematically studied. The aim of our study was to analyze PDFV nonattendance, particularly NSs and same-day cancellations (SDCs), for hospitalizations and clinics within our healthcare system.

METHODS

Study Design

We conducted an observational cohort study of adult patients from 10 medical units at the Hospital of the University of Pennsylvania (a 789-bed quaternary-care hospital within an urban, academic medical system) who were scheduled with at least 1 PDFV. Specifically, the patients included in our analysis were hospitalized on general internal medicine services or medical subspecialty services with discharge dates between April 1, 2014, and March 31, 2015. Hospitalizations included in our study had at least 1 PDFV scheduled with an outpatient provider affiliated with the University of Pennsylvania Health System (UPHS). PDFVs scheduled with unaffiliated providers were not examined.

Each PDFV was requested by a patient’s inpatient care team. Once the care team had determined that a PDFV was clinically warranted, a member of the team (generally a resident, advanced practice provider, medical student, or designee) either called the UPHS clinic to schedule an appointment time or e-mailed the outpatient UPHS provider directly to facilitate a more urgent PDFV appointment time. Once a PDFV time was confirmed, PDFV details (ie, date, time, location, and phone number) were electronically entered into the patient’s discharge instructions by the inpatient care team. At the time of discharge, nurses reviewed these instructions with their patients. All patients left the hospital with a physical copy of these instructions. As part of routine care at our institution, patients then received automated telephone reminders from their UPHS-affiliated outpatient clinic 48 hours prior to each PDFV.

Data Collection

Our study was determined to meet criteria for quality improvement by the University of Pennsylvania’s Institutional Review Board. We used our healthcare system’s integrated electronic medical record system to track the dates of initial PDFV requests, the dates of hospitalization, and actual PDFV dates. PDFVs were included if the appointment request was made while a patient was hospitalized, including the day of discharge. Our study methodology only allowed us to investigate PDFVs scheduled with UPHS outpatient providers. We did not review discharge instructions or survey non-UPHS clinics to quantify visits scheduled with other providers, for example, community health centers or external private practices.

Exclusion criteria included the following: (1) office visits with nonproviders, for example, scheduled diagnostic procedures or pharmacist appointments for warfarin dosing; (2) visits cancelled by inpatient providers prior to discharge; (3) visits for patients not otherwise eligible for UPHS outpatient care because of insurance reasons; and (4) visits scheduled for dates after a patient’s death. Our motivation for the third exclusion criterion was the infrequent and irregular process by which PDFVs were authorized for these patients. These patients and their characteristics are described in Supplementary Table 1 in more detail.

For each PDFV, we recorded age, gender, race, insurance status, driving distance, length of stay for index hospitalization, discharging service (general internal medicine vs subspecialty), postdischarge disposition (home, home with home care services such as nursing or physical therapy, or facility), the number of PDFVs scheduled per index hospitalization, PDFV specialty type (oncologic subspecialty, nononcologic medical subspecialty, nononcologic surgical subspecialty, primary care, or other specialty), PDFV season, and PDFV lead time (the number of days between the discharge date and PDFV). We consolidated oncologic specialties into 1 group given the integrated nature of our healthcare system’s comprehensive cancer center. “Other” PDFV specialty subtypes are described in Supplementary Table 2. Driving distances between patient postal codes and our hospital were calculated using Excel VBA Master (Salt Lake City, Utah) and were subsequently categorized into patient-level quartiles for further analysis. For cancelled PDFVs, we collected dates of cancellation relative to the date of the appointment itself.

 

Study Outcomes

The primary study outcome was PDFV attendance. Each PDFV’s status was categorized by outpatient clinic staff as attended, cancelled, or NS. For cancelled appointments, cancellation dates and reasons (if entered by clinic representatives) were collected. In keeping with prior studies investigating outpatient nonattendance,we calculated collective NS/SDC rates for the variables listed above.^17,22-25 We additionally calculated NS/SDC and attendance-as-scheduled rates stratified by the number of PDFVs per patient to assess for a “high-utilizer” effect with regard to PDFV attendance.

Statistical Analysis

We used multivariable mixed-effects regression with a logit link to assess associations between age, gender, race, insurance, driving distance quartile, length of stay, discharging service, postdischarge disposition, the number of PDFVs per hospitalization, PDFV specialty type, PDFV season, PDFV lead time, and our NS/SDC outcome. The mixed-effects approach was used to account for correlation structures induced by patients who had multiple visits and for patients with multiple hospitalizations. Specifically, our model specified 2 levels of nesting (PDFVs nested within each hospitalization, which were nested within each patient) to obtain appropriate standard error estimates for our adjusted odds ratios (ORs). Correlation matrices and multivariable variance inflation factors were used to assess collinearity among the predictor variables. These assessments did not indicate strong collinearity; hence, all predictors were included in the model. Only driving distance had a small amount of missing data (0.18% of driving distances were unavailable), so multiple imputation was not undertaken. Analyses were performed using R version 3.3.1 (R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

Baseline Characteristics

During the 1-year study period, there were 11,829 discrete hospitalizations in medical units at our hospital. Of these hospitalizations, 6136 (52%) had at least 1 UPHS-affiliated PDFV meeting our inclusion and exclusion criteria, as detailed in the Figure. Across these hospitalizations, 9258 PDFVs were scheduled on behalf of 4653 patients. Demographic characteristics for these patients, hospitalizations, and visits are detailed in Table 1. The median age of patients in our cohort was 61 years old (interquartile range [IQR] 49-70, range 18-101). The median driving distance was 17 miles (IQR 4.3-38.8, range 0-2891). For hospitalizations, the median length of stay was 5 days (IQR 3-10, range 0-97). The median PDFV lead time, which is defined as the number of days between discharge and PDFV, was 12 days (IQR 6-23, range 1-60). Overall, 41% of patients (n = 1927) attended all of their PDFVs as scheduled; Supplementary Figure 1 lists patient-level PDFV attendance-as-scheduled percentages in more detail.

Incidence of NSs and SDCs

Twenty-five percent of PDFVs (n = 2303) were ultimately NS/SDCs; this included 1658 NSs (18% of all appointments) and 645 SDCs (7% of all appointments). Fifty-two percent of PDFVs (n = 4847) were kept as scheduled, while 23% (n = 2108) were cancelled before the day of the visit. Of the 2558 cancellations with valid cancellation dates, 49% (n = 1252) were cancelled 2 or fewer days beforehand, as shown in Supplementary Figure 2.

In Table 2, we show unadjusted NS/SDC rates and adjusted NS/SDC ORs based on patient and hospitalization characteristics. NS/SDC appointments were more likely to occur in patients who were black (adjusted OR 1.94, 95% confidence interval [CI], 1.63-2.32) or Medicaid insured (OR 1.41, 95% CI, 1.19-1.67). In contrast, NS/SDC appointments were less likely in elderly patients (age ≥65 years: OR 0.39, 95% CI, 0.31-0.49) and patients who lived further away (furthest quartile of driving distance: OR 0.65, 95% CI, 0.52-–0.81). Longer hospitalizations were associated with higher NS/SDC rates (length of stay ≥15 days: OR 1.51, 95% CI, 1.22-1.88). In contrast, discharges from subspecialty services (OR 0.79, 95% CI, 0.68-0.93) had independently lower NS/SDC rates. Compared to discharges to home without services, NS/SDC rates were higher with discharges to home with services (OR 1.32, 95% CI, 1.01-1.36) and with discharges to facilities (OR 2.10, 95% CI, 1.70-2.60).

The presence of exactly 2 PDFVs per hospitalization was also associated with higher NS/SDC rates (OR 1.17, 95% CI, 1.01-1.36), compared to a single PDFV per hospitalization; however, the presence of more than 2 PDFVs per hospitalization was associated with lower NS/SDC rates (OR 0.82, 95% CI, 0.69-0.98). A separate analysis (data not shown) of potential high utilizers revealed a 15% NS/SDC rate for the top 0.5% of patients (median: 18 PDFVs each) and an 18% NS/SDC rate for the top 1% of patients (median: 14 PDFVs each) with regard to the numbers of PDFVs scheduled, compared to the 25% overall NS/SDC rate for all patients.

NS/SDC rates and adjusted ORs with regard to individual PDFV characteristics are displayed in Table 3. Nononcologic visits had higher NS/SDC rates than oncologic visits; for example, the NS/SDC rate for primary care visits was 39% (OR 2.62, 95% CI, 2.03-3.38), compared to 12% for oncologic visits. Appointments in the “other” specialty category also had high nonattendance rates, as further described in Supplementary Table B. Summertime appointments were more likely to be attended (OR 0.81, 95% CI, 0.68-0.97) compared to those in the spring. PDFV lead time (the time interval between the discharge date and appointment date) was not associated with changes in visit attendance.

 

DISCUSSION

PDFVs were scheduled on patients’ behalf for more than half of all medical hospitalizations at our institution, a rate that is consistent with previous studies.^10,11,26 However, 1 in 4 of these PDFVs resulted in a NS/SDC. This figure contrasts sharply with our institution’s 10% overall NS/SDC rate for all outpatient visits (S. Schlegel, written communication, July 2016). In our study, patients who were younger, black, or Medicaid insured were more likely to miss their follow-up visits. Patients who lived farther from the study hospital had lower NS/SDC rates, which is consistent with another study of a low-income, urban patient population.²⁷ In contrast, patients with longer lengths of stay, discharges with home care services, or discharges to another facility were more likely to miss their PDFVs. Reasons for this are likely multifactorial, including readmission to a hospital or feeling too unwell to leave home to attend PDFVs. Insurance policies regarding ambulance reimbursement and outpatient billing can cause confusion and may have contributed to higher NS/SDC rates for facility-bound patients.^28,29

When comparing PDFV characteristics themselves, oncologic visits had the lowest NS/SDC incidence of any group analyzed in our study. This may be related to the inherent life-altering nature of a cancer diagnosis or our cancer center’s use of patient navigators.^23,30 In contrast, primary care clinics suffered from NS/SDC rates approaching 40%, which is a concerning finding given the importance of primary care coordination in the posthospitalization period.^9,31 Why are primary care appointments so commonly missed? Some studies suggest that forgetting about a primary care appointment is a leading reason.^15,32,33 For PDFVs, this phenomenon may be augmented because the visits are not scheduled by patients themselves. Additionally, patients may paradoxically undervalue the benefit of an all-encompassing primary care visit, compared to a PDFV focused on a specific problem, (eg, a cardiology follow-up appointment for a patient with congestive heart failure). In particular, patients with limited health literacy may potentially undervalue the capabilities of their primary care clinics.^34,35

The low absolute number of primary care PDFVs (only 8% of all visits) scheduled for patients at our hospital was an unexpected finding. This low percentage is likely a function of the patient population hospitalized at our large, urban quaternary-care facility. First, an unknown number of patients may have had PDFVs manually scheduled with primary care providers external to our health system; these PDFVs were not captured within our study. Second, 71% of the hospitalizations in our study occurred in subspecialty services, for which specific primary care follow-up may not be as urgent. Supporting this fact, further analysis of the 6136 hospitalizations in our study (data not shown) revealed that 28% of the hospitalizations in general internal medicine were scheduled with at least 1 primary care PDFV as opposed to only 5% of subspecialty-service hospitalizations.

In contrast to several previous studies of outpatient nonattendance,we did not find that visits scheduled for time points further in the future were more likely to be missed.^{14,24,25,36,37} Unlike other appointments, it may be that PDFV lead time does not affect attendance because of the unique manner in which PDFV times are scheduled and conveyed to patients. Unlike other appointments, patients do not schedule PDFVs themselves but instead learn about their PDFV dates as part of a large set of discharge instructions. This practice may result in poor recall of PDFV dates in recently hospitalized patients³⁸, regardless of the lead time between discharge and the visit itself.

Supplementary Table 1 details a 51% NS/SDC rate for the small number of PDFVs (n = 65) that were excluded a priori from our analysis because of general ineligibility for UPHS outpatient care. We specifically chose to exclude this population because of the infrequent and irregular process by which these PDFVs were authorized on a case-by-case basis, typically via active engagement by our hospital’s social work department. We did not study this population further but postulate that the 51% NS/SDC rate may reflect other social determinants of health that contribute to appointment nonadherence in a predominantly uninsured population.

Beyond their effect on patient outcomes, improving PDFV-related processes has the potential to boost both inpatient and outpatient provider satisfaction. From the standpoint of frontline inpatient providers (often resident physicians), calling outpatient clinics to request PDFVs is viewed as 1 of the top 5 administrative tasks that interfere with house staff education.³⁹ Future interventions that involve patients in the PDFV scheduling process may improve inpatient workflow while simultaneously engaging patients in their own care. For example, asking clinic representatives to directly schedule PDFVs with hospitalized patients, either by phone or in person, has been shown in pilot studies to improve PDFV attendance and decrease readmissions.^40-42 Conversely, NS/SDC visits harm outpatient provider productivity and decrease provider availability for other patients.^13,17,43 Strategies to mitigate the impact of unfilled appointment slots (eg, deliberately overbooking time slots in advance) carry their own risks, including provider burnout.⁴⁴ As such, preventing NSs may be superior to curing their adverse impacts. Many such strategies exist in the ambulatory setting,^13,43,45 for example, better communication with patients through texting or goal-directed, personalized phone reminders.^46-48Our study methodology has several limitations. Most importantly, we were unable to measure PDFVs made with providers unaffiliated with UPHS. As previously noted, our low proportion of primary care PDFVs may specifically reflect patients with primary care providers outside of our health system. Similarly, our low percentage of Medicaid patients receiving PDFVs may be related to follow-up visits with nonaffiliated community health centers. We were unable to measure patient acuity and health literacy as potential predictors of NS/SDC rates. Driving distances were calculated from patient postal codes to our hospital, not to individual outpatient clinics. However, the majority of our hospital-affiliated clinics are located adjacent to our hospital; additionally, we grouped driving distances into quartiles for our analysis. We had initially attempted to differentiate between clinic-initiated and patient-initiated cancellations, but unfortunately, we found that the data were too unreliable to be used for further analysis (outlined in Supplementary Table 3). Lastly, because we studied patients in medical units at a single large, urban, academic center, our results are not generalizable to other settings (eg, community hospitals, hospitals with smaller networks of outpatient providers, or patients being discharged from surgical services or observation units).

 

CONCLUSION

Given national efforts to enhance postdischarge transitions of care, we aimed to analyze attendance at provider-scheduled PDFV appointments. Our finding that 25% of PDFVs resulted in NS/SDCs raises both questions and opportunities for inpatient and outpatient providers. Further research is needed to understand why so many patients miss their PDFVs, and we should work as a field to develop creative solutions to improve PDFV scheduling and attendance.

Acknowledgments

The authors acknowledge Marie Synnestvedt, PhD, and Manik Chhabra, MD, for their assistance with data gathering and statistical analysis. They also acknowledge Allison DeKosky, MD, Michael Serpa, BS, Michael McFall, and Scott Schlegel, MBA, for their assistance with researching this topic. They did not receive external compensation for their assistance outside of their usual salary support.

DISCLOSURE

Nothing to report.

Given growing incentives to reduce readmission rates, predischarge checklists and bundles have recommended that inpatient providers schedule postdischarge follow-up visits (PDFVs) for their hospitalized patients.^1-4 PDFVs have been linked to lower readmission rates in patients with chronic conditions, including congestive heart failure, psychiatric illnesses, and chronic obstructive pulmonary disease.^5-8 In contrast, the impact of PDFVs on readmissions in hospitalized general medicine populations has been mixed.^9-12 Beyond the presence or absence of PDFVs, it may be a patient’s inability to keep scheduled PDFVs that contributes more strongly to preventable readmissions.¹¹

This challenge, dealing with the 12% to 37% of patients who miss their visits (“no-shows”), is not new.^13-17 In high-risk patient populations, such as those with substance abuse, diabetes, or human immunodeficiency virus, no-shows (NSs) have been linked to poorer short-term and long-term clinical outcomes.^16,18-20 Additionally, NSs pose a challenge for outpatient clinics and the healthcare system at large. The financial cost of NSs ranges from approximately $200 per patient in 2 analyses to $7 million in cumulative lost revenue per year at 1 large academic health system.^13,17,21 As such, increasing attendance at PDFVs is a potential target for improving both patient outcomes and clinic productivity.

Most prior PDFV research has focused on readmission risk rather than PDFV attendance as the primary outcome.^5-12 However, given the patient-oriented benefits of attending PDFVs and the clinic-oriented benefits of avoiding vacant time slots, NS PDFVs represent an important missed opportunity for our healthcare delivery system. To our knowledge, risk factors for PDFV nonattendance have not yet been systematically studied. The aim of our study was to analyze PDFV nonattendance, particularly NSs and same-day cancellations (SDCs), for hospitalizations and clinics within our healthcare system.

METHODS

Study Design

We conducted an observational cohort study of adult patients from 10 medical units at the Hospital of the University of Pennsylvania (a 789-bed quaternary-care hospital within an urban, academic medical system) who were scheduled with at least 1 PDFV. Specifically, the patients included in our analysis were hospitalized on general internal medicine services or medical subspecialty services with discharge dates between April 1, 2014, and March 31, 2015. Hospitalizations included in our study had at least 1 PDFV scheduled with an outpatient provider affiliated with the University of Pennsylvania Health System (UPHS). PDFVs scheduled with unaffiliated providers were not examined.

Each PDFV was requested by a patient’s inpatient care team. Once the care team had determined that a PDFV was clinically warranted, a member of the team (generally a resident, advanced practice provider, medical student, or designee) either called the UPHS clinic to schedule an appointment time or e-mailed the outpatient UPHS provider directly to facilitate a more urgent PDFV appointment time. Once a PDFV time was confirmed, PDFV details (ie, date, time, location, and phone number) were electronically entered into the patient’s discharge instructions by the inpatient care team. At the time of discharge, nurses reviewed these instructions with their patients. All patients left the hospital with a physical copy of these instructions. As part of routine care at our institution, patients then received automated telephone reminders from their UPHS-affiliated outpatient clinic 48 hours prior to each PDFV.

Data Collection

Our study was determined to meet criteria for quality improvement by the University of Pennsylvania’s Institutional Review Board. We used our healthcare system’s integrated electronic medical record system to track the dates of initial PDFV requests, the dates of hospitalization, and actual PDFV dates. PDFVs were included if the appointment request was made while a patient was hospitalized, including the day of discharge. Our study methodology only allowed us to investigate PDFVs scheduled with UPHS outpatient providers. We did not review discharge instructions or survey non-UPHS clinics to quantify visits scheduled with other providers, for example, community health centers or external private practices.

Exclusion criteria included the following: (1) office visits with nonproviders, for example, scheduled diagnostic procedures or pharmacist appointments for warfarin dosing; (2) visits cancelled by inpatient providers prior to discharge; (3) visits for patients not otherwise eligible for UPHS outpatient care because of insurance reasons; and (4) visits scheduled for dates after a patient’s death. Our motivation for the third exclusion criterion was the infrequent and irregular process by which PDFVs were authorized for these patients. These patients and their characteristics are described in Supplementary Table 1 in more detail.

For each PDFV, we recorded age, gender, race, insurance status, driving distance, length of stay for index hospitalization, discharging service (general internal medicine vs subspecialty), postdischarge disposition (home, home with home care services such as nursing or physical therapy, or facility), the number of PDFVs scheduled per index hospitalization, PDFV specialty type (oncologic subspecialty, nononcologic medical subspecialty, nononcologic surgical subspecialty, primary care, or other specialty), PDFV season, and PDFV lead time (the number of days between the discharge date and PDFV). We consolidated oncologic specialties into 1 group given the integrated nature of our healthcare system’s comprehensive cancer center. “Other” PDFV specialty subtypes are described in Supplementary Table 2. Driving distances between patient postal codes and our hospital were calculated using Excel VBA Master (Salt Lake City, Utah) and were subsequently categorized into patient-level quartiles for further analysis. For cancelled PDFVs, we collected dates of cancellation relative to the date of the appointment itself.

 

Study Outcomes

The primary study outcome was PDFV attendance. Each PDFV’s status was categorized by outpatient clinic staff as attended, cancelled, or NS. For cancelled appointments, cancellation dates and reasons (if entered by clinic representatives) were collected. In keeping with prior studies investigating outpatient nonattendance,we calculated collective NS/SDC rates for the variables listed above.^17,22-25 We additionally calculated NS/SDC and attendance-as-scheduled rates stratified by the number of PDFVs per patient to assess for a “high-utilizer” effect with regard to PDFV attendance.

Statistical Analysis

We used multivariable mixed-effects regression with a logit link to assess associations between age, gender, race, insurance, driving distance quartile, length of stay, discharging service, postdischarge disposition, the number of PDFVs per hospitalization, PDFV specialty type, PDFV season, PDFV lead time, and our NS/SDC outcome. The mixed-effects approach was used to account for correlation structures induced by patients who had multiple visits and for patients with multiple hospitalizations. Specifically, our model specified 2 levels of nesting (PDFVs nested within each hospitalization, which were nested within each patient) to obtain appropriate standard error estimates for our adjusted odds ratios (ORs). Correlation matrices and multivariable variance inflation factors were used to assess collinearity among the predictor variables. These assessments did not indicate strong collinearity; hence, all predictors were included in the model. Only driving distance had a small amount of missing data (0.18% of driving distances were unavailable), so multiple imputation was not undertaken. Analyses were performed using R version 3.3.1 (R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

Baseline Characteristics

During the 1-year study period, there were 11,829 discrete hospitalizations in medical units at our hospital. Of these hospitalizations, 6136 (52%) had at least 1 UPHS-affiliated PDFV meeting our inclusion and exclusion criteria, as detailed in the Figure. Across these hospitalizations, 9258 PDFVs were scheduled on behalf of 4653 patients. Demographic characteristics for these patients, hospitalizations, and visits are detailed in Table 1. The median age of patients in our cohort was 61 years old (interquartile range [IQR] 49-70, range 18-101). The median driving distance was 17 miles (IQR 4.3-38.8, range 0-2891). For hospitalizations, the median length of stay was 5 days (IQR 3-10, range 0-97). The median PDFV lead time, which is defined as the number of days between discharge and PDFV, was 12 days (IQR 6-23, range 1-60). Overall, 41% of patients (n = 1927) attended all of their PDFVs as scheduled; Supplementary Figure 1 lists patient-level PDFV attendance-as-scheduled percentages in more detail.

Incidence of NSs and SDCs

Twenty-five percent of PDFVs (n = 2303) were ultimately NS/SDCs; this included 1658 NSs (18% of all appointments) and 645 SDCs (7% of all appointments). Fifty-two percent of PDFVs (n = 4847) were kept as scheduled, while 23% (n = 2108) were cancelled before the day of the visit. Of the 2558 cancellations with valid cancellation dates, 49% (n = 1252) were cancelled 2 or fewer days beforehand, as shown in Supplementary Figure 2.

In Table 2, we show unadjusted NS/SDC rates and adjusted NS/SDC ORs based on patient and hospitalization characteristics. NS/SDC appointments were more likely to occur in patients who were black (adjusted OR 1.94, 95% confidence interval [CI], 1.63-2.32) or Medicaid insured (OR 1.41, 95% CI, 1.19-1.67). In contrast, NS/SDC appointments were less likely in elderly patients (age ≥65 years: OR 0.39, 95% CI, 0.31-0.49) and patients who lived further away (furthest quartile of driving distance: OR 0.65, 95% CI, 0.52-–0.81). Longer hospitalizations were associated with higher NS/SDC rates (length of stay ≥15 days: OR 1.51, 95% CI, 1.22-1.88). In contrast, discharges from subspecialty services (OR 0.79, 95% CI, 0.68-0.93) had independently lower NS/SDC rates. Compared to discharges to home without services, NS/SDC rates were higher with discharges to home with services (OR 1.32, 95% CI, 1.01-1.36) and with discharges to facilities (OR 2.10, 95% CI, 1.70-2.60).

The presence of exactly 2 PDFVs per hospitalization was also associated with higher NS/SDC rates (OR 1.17, 95% CI, 1.01-1.36), compared to a single PDFV per hospitalization; however, the presence of more than 2 PDFVs per hospitalization was associated with lower NS/SDC rates (OR 0.82, 95% CI, 0.69-0.98). A separate analysis (data not shown) of potential high utilizers revealed a 15% NS/SDC rate for the top 0.5% of patients (median: 18 PDFVs each) and an 18% NS/SDC rate for the top 1% of patients (median: 14 PDFVs each) with regard to the numbers of PDFVs scheduled, compared to the 25% overall NS/SDC rate for all patients.

NS/SDC rates and adjusted ORs with regard to individual PDFV characteristics are displayed in Table 3. Nononcologic visits had higher NS/SDC rates than oncologic visits; for example, the NS/SDC rate for primary care visits was 39% (OR 2.62, 95% CI, 2.03-3.38), compared to 12% for oncologic visits. Appointments in the “other” specialty category also had high nonattendance rates, as further described in Supplementary Table B. Summertime appointments were more likely to be attended (OR 0.81, 95% CI, 0.68-0.97) compared to those in the spring. PDFV lead time (the time interval between the discharge date and appointment date) was not associated with changes in visit attendance.

 

DISCUSSION

PDFVs were scheduled on patients’ behalf for more than half of all medical hospitalizations at our institution, a rate that is consistent with previous studies.^10,11,26 However, 1 in 4 of these PDFVs resulted in a NS/SDC. This figure contrasts sharply with our institution’s 10% overall NS/SDC rate for all outpatient visits (S. Schlegel, written communication, July 2016). In our study, patients who were younger, black, or Medicaid insured were more likely to miss their follow-up visits. Patients who lived farther from the study hospital had lower NS/SDC rates, which is consistent with another study of a low-income, urban patient population.²⁷ In contrast, patients with longer lengths of stay, discharges with home care services, or discharges to another facility were more likely to miss their PDFVs. Reasons for this are likely multifactorial, including readmission to a hospital or feeling too unwell to leave home to attend PDFVs. Insurance policies regarding ambulance reimbursement and outpatient billing can cause confusion and may have contributed to higher NS/SDC rates for facility-bound patients.^28,29

When comparing PDFV characteristics themselves, oncologic visits had the lowest NS/SDC incidence of any group analyzed in our study. This may be related to the inherent life-altering nature of a cancer diagnosis or our cancer center’s use of patient navigators.^23,30 In contrast, primary care clinics suffered from NS/SDC rates approaching 40%, which is a concerning finding given the importance of primary care coordination in the posthospitalization period.^9,31 Why are primary care appointments so commonly missed? Some studies suggest that forgetting about a primary care appointment is a leading reason.^15,32,33 For PDFVs, this phenomenon may be augmented because the visits are not scheduled by patients themselves. Additionally, patients may paradoxically undervalue the benefit of an all-encompassing primary care visit, compared to a PDFV focused on a specific problem, (eg, a cardiology follow-up appointment for a patient with congestive heart failure). In particular, patients with limited health literacy may potentially undervalue the capabilities of their primary care clinics.^34,35

The low absolute number of primary care PDFVs (only 8% of all visits) scheduled for patients at our hospital was an unexpected finding. This low percentage is likely a function of the patient population hospitalized at our large, urban quaternary-care facility. First, an unknown number of patients may have had PDFVs manually scheduled with primary care providers external to our health system; these PDFVs were not captured within our study. Second, 71% of the hospitalizations in our study occurred in subspecialty services, for which specific primary care follow-up may not be as urgent. Supporting this fact, further analysis of the 6136 hospitalizations in our study (data not shown) revealed that 28% of the hospitalizations in general internal medicine were scheduled with at least 1 primary care PDFV as opposed to only 5% of subspecialty-service hospitalizations.

In contrast to several previous studies of outpatient nonattendance,we did not find that visits scheduled for time points further in the future were more likely to be missed.^{14,24,25,36,37} Unlike other appointments, it may be that PDFV lead time does not affect attendance because of the unique manner in which PDFV times are scheduled and conveyed to patients. Unlike other appointments, patients do not schedule PDFVs themselves but instead learn about their PDFV dates as part of a large set of discharge instructions. This practice may result in poor recall of PDFV dates in recently hospitalized patients³⁸, regardless of the lead time between discharge and the visit itself.

Supplementary Table 1 details a 51% NS/SDC rate for the small number of PDFVs (n = 65) that were excluded a priori from our analysis because of general ineligibility for UPHS outpatient care. We specifically chose to exclude this population because of the infrequent and irregular process by which these PDFVs were authorized on a case-by-case basis, typically via active engagement by our hospital’s social work department. We did not study this population further but postulate that the 51% NS/SDC rate may reflect other social determinants of health that contribute to appointment nonadherence in a predominantly uninsured population.

Beyond their effect on patient outcomes, improving PDFV-related processes has the potential to boost both inpatient and outpatient provider satisfaction. From the standpoint of frontline inpatient providers (often resident physicians), calling outpatient clinics to request PDFVs is viewed as 1 of the top 5 administrative tasks that interfere with house staff education.³⁹ Future interventions that involve patients in the PDFV scheduling process may improve inpatient workflow while simultaneously engaging patients in their own care. For example, asking clinic representatives to directly schedule PDFVs with hospitalized patients, either by phone or in person, has been shown in pilot studies to improve PDFV attendance and decrease readmissions.^40-42 Conversely, NS/SDC visits harm outpatient provider productivity and decrease provider availability for other patients.^13,17,43 Strategies to mitigate the impact of unfilled appointment slots (eg, deliberately overbooking time slots in advance) carry their own risks, including provider burnout.⁴⁴ As such, preventing NSs may be superior to curing their adverse impacts. Many such strategies exist in the ambulatory setting,^13,43,45 for example, better communication with patients through texting or goal-directed, personalized phone reminders.^46-48Our study methodology has several limitations. Most importantly, we were unable to measure PDFVs made with providers unaffiliated with UPHS. As previously noted, our low proportion of primary care PDFVs may specifically reflect patients with primary care providers outside of our health system. Similarly, our low percentage of Medicaid patients receiving PDFVs may be related to follow-up visits with nonaffiliated community health centers. We were unable to measure patient acuity and health literacy as potential predictors of NS/SDC rates. Driving distances were calculated from patient postal codes to our hospital, not to individual outpatient clinics. However, the majority of our hospital-affiliated clinics are located adjacent to our hospital; additionally, we grouped driving distances into quartiles for our analysis. We had initially attempted to differentiate between clinic-initiated and patient-initiated cancellations, but unfortunately, we found that the data were too unreliable to be used for further analysis (outlined in Supplementary Table 3). Lastly, because we studied patients in medical units at a single large, urban, academic center, our results are not generalizable to other settings (eg, community hospitals, hospitals with smaller networks of outpatient providers, or patients being discharged from surgical services or observation units).

 

CONCLUSION

Given national efforts to enhance postdischarge transitions of care, we aimed to analyze attendance at provider-scheduled PDFV appointments. Our finding that 25% of PDFVs resulted in NS/SDCs raises both questions and opportunities for inpatient and outpatient providers. Further research is needed to understand why so many patients miss their PDFVs, and we should work as a field to develop creative solutions to improve PDFV scheduling and attendance.

Acknowledgments

The authors acknowledge Marie Synnestvedt, PhD, and Manik Chhabra, MD, for their assistance with data gathering and statistical analysis. They also acknowledge Allison DeKosky, MD, Michael Serpa, BS, Michael McFall, and Scott Schlegel, MBA, for their assistance with researching this topic. They did not receive external compensation for their assistance outside of their usual salary support.

DISCLOSURE

Nothing to report.

Given growing incentives to reduce readmission rates, predischarge checklists and bundles have recommended that inpatient providers schedule postdischarge follow-up visits (PDFVs) for their hospitalized patients.^1-4 PDFVs have been linked to lower readmission rates in patients with chronic conditions, including congestive heart failure, psychiatric illnesses, and chronic obstructive pulmonary disease.^5-8 In contrast, the impact of PDFVs on readmissions in hospitalized general medicine populations has been mixed.^9-12 Beyond the presence or absence of PDFVs, it may be a patient’s inability to keep scheduled PDFVs that contributes more strongly to preventable readmissions.¹¹

This challenge, dealing with the 12% to 37% of patients who miss their visits (“no-shows”), is not new.^13-17 In high-risk patient populations, such as those with substance abuse, diabetes, or human immunodeficiency virus, no-shows (NSs) have been linked to poorer short-term and long-term clinical outcomes.^16,18-20 Additionally, NSs pose a challenge for outpatient clinics and the healthcare system at large. The financial cost of NSs ranges from approximately $200 per patient in 2 analyses to $7 million in cumulative lost revenue per year at 1 large academic health system.^13,17,21 As such, increasing attendance at PDFVs is a potential target for improving both patient outcomes and clinic productivity.

Most prior PDFV research has focused on readmission risk rather than PDFV attendance as the primary outcome.^5-12 However, given the patient-oriented benefits of attending PDFVs and the clinic-oriented benefits of avoiding vacant time slots, NS PDFVs represent an important missed opportunity for our healthcare delivery system. To our knowledge, risk factors for PDFV nonattendance have not yet been systematically studied. The aim of our study was to analyze PDFV nonattendance, particularly NSs and same-day cancellations (SDCs), for hospitalizations and clinics within our healthcare system.

METHODS

Study Design

We conducted an observational cohort study of adult patients from 10 medical units at the Hospital of the University of Pennsylvania (a 789-bed quaternary-care hospital within an urban, academic medical system) who were scheduled with at least 1 PDFV. Specifically, the patients included in our analysis were hospitalized on general internal medicine services or medical subspecialty services with discharge dates between April 1, 2014, and March 31, 2015. Hospitalizations included in our study had at least 1 PDFV scheduled with an outpatient provider affiliated with the University of Pennsylvania Health System (UPHS). PDFVs scheduled with unaffiliated providers were not examined.

Each PDFV was requested by a patient’s inpatient care team. Once the care team had determined that a PDFV was clinically warranted, a member of the team (generally a resident, advanced practice provider, medical student, or designee) either called the UPHS clinic to schedule an appointment time or e-mailed the outpatient UPHS provider directly to facilitate a more urgent PDFV appointment time. Once a PDFV time was confirmed, PDFV details (ie, date, time, location, and phone number) were electronically entered into the patient’s discharge instructions by the inpatient care team. At the time of discharge, nurses reviewed these instructions with their patients. All patients left the hospital with a physical copy of these instructions. As part of routine care at our institution, patients then received automated telephone reminders from their UPHS-affiliated outpatient clinic 48 hours prior to each PDFV.

Data Collection

Our study was determined to meet criteria for quality improvement by the University of Pennsylvania’s Institutional Review Board. We used our healthcare system’s integrated electronic medical record system to track the dates of initial PDFV requests, the dates of hospitalization, and actual PDFV dates. PDFVs were included if the appointment request was made while a patient was hospitalized, including the day of discharge. Our study methodology only allowed us to investigate PDFVs scheduled with UPHS outpatient providers. We did not review discharge instructions or survey non-UPHS clinics to quantify visits scheduled with other providers, for example, community health centers or external private practices.

Exclusion criteria included the following: (1) office visits with nonproviders, for example, scheduled diagnostic procedures or pharmacist appointments for warfarin dosing; (2) visits cancelled by inpatient providers prior to discharge; (3) visits for patients not otherwise eligible for UPHS outpatient care because of insurance reasons; and (4) visits scheduled for dates after a patient’s death. Our motivation for the third exclusion criterion was the infrequent and irregular process by which PDFVs were authorized for these patients. These patients and their characteristics are described in Supplementary Table 1 in more detail.

For each PDFV, we recorded age, gender, race, insurance status, driving distance, length of stay for index hospitalization, discharging service (general internal medicine vs subspecialty), postdischarge disposition (home, home with home care services such as nursing or physical therapy, or facility), the number of PDFVs scheduled per index hospitalization, PDFV specialty type (oncologic subspecialty, nononcologic medical subspecialty, nononcologic surgical subspecialty, primary care, or other specialty), PDFV season, and PDFV lead time (the number of days between the discharge date and PDFV). We consolidated oncologic specialties into 1 group given the integrated nature of our healthcare system’s comprehensive cancer center. “Other” PDFV specialty subtypes are described in Supplementary Table 2. Driving distances between patient postal codes and our hospital were calculated using Excel VBA Master (Salt Lake City, Utah) and were subsequently categorized into patient-level quartiles for further analysis. For cancelled PDFVs, we collected dates of cancellation relative to the date of the appointment itself.

 

Study Outcomes

The primary study outcome was PDFV attendance. Each PDFV’s status was categorized by outpatient clinic staff as attended, cancelled, or NS. For cancelled appointments, cancellation dates and reasons (if entered by clinic representatives) were collected. In keeping with prior studies investigating outpatient nonattendance,we calculated collective NS/SDC rates for the variables listed above.^17,22-25 We additionally calculated NS/SDC and attendance-as-scheduled rates stratified by the number of PDFVs per patient to assess for a “high-utilizer” effect with regard to PDFV attendance.

Statistical Analysis

We used multivariable mixed-effects regression with a logit link to assess associations between age, gender, race, insurance, driving distance quartile, length of stay, discharging service, postdischarge disposition, the number of PDFVs per hospitalization, PDFV specialty type, PDFV season, PDFV lead time, and our NS/SDC outcome. The mixed-effects approach was used to account for correlation structures induced by patients who had multiple visits and for patients with multiple hospitalizations. Specifically, our model specified 2 levels of nesting (PDFVs nested within each hospitalization, which were nested within each patient) to obtain appropriate standard error estimates for our adjusted odds ratios (ORs). Correlation matrices and multivariable variance inflation factors were used to assess collinearity among the predictor variables. These assessments did not indicate strong collinearity; hence, all predictors were included in the model. Only driving distance had a small amount of missing data (0.18% of driving distances were unavailable), so multiple imputation was not undertaken. Analyses were performed using R version 3.3.1 (R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

Baseline Characteristics

During the 1-year study period, there were 11,829 discrete hospitalizations in medical units at our hospital. Of these hospitalizations, 6136 (52%) had at least 1 UPHS-affiliated PDFV meeting our inclusion and exclusion criteria, as detailed in the Figure. Across these hospitalizations, 9258 PDFVs were scheduled on behalf of 4653 patients. Demographic characteristics for these patients, hospitalizations, and visits are detailed in Table 1. The median age of patients in our cohort was 61 years old (interquartile range [IQR] 49-70, range 18-101). The median driving distance was 17 miles (IQR 4.3-38.8, range 0-2891). For hospitalizations, the median length of stay was 5 days (IQR 3-10, range 0-97). The median PDFV lead time, which is defined as the number of days between discharge and PDFV, was 12 days (IQR 6-23, range 1-60). Overall, 41% of patients (n = 1927) attended all of their PDFVs as scheduled; Supplementary Figure 1 lists patient-level PDFV attendance-as-scheduled percentages in more detail.

Incidence of NSs and SDCs

Twenty-five percent of PDFVs (n = 2303) were ultimately NS/SDCs; this included 1658 NSs (18% of all appointments) and 645 SDCs (7% of all appointments). Fifty-two percent of PDFVs (n = 4847) were kept as scheduled, while 23% (n = 2108) were cancelled before the day of the visit. Of the 2558 cancellations with valid cancellation dates, 49% (n = 1252) were cancelled 2 or fewer days beforehand, as shown in Supplementary Figure 2.

In Table 2, we show unadjusted NS/SDC rates and adjusted NS/SDC ORs based on patient and hospitalization characteristics. NS/SDC appointments were more likely to occur in patients who were black (adjusted OR 1.94, 95% confidence interval [CI], 1.63-2.32) or Medicaid insured (OR 1.41, 95% CI, 1.19-1.67). In contrast, NS/SDC appointments were less likely in elderly patients (age ≥65 years: OR 0.39, 95% CI, 0.31-0.49) and patients who lived further away (furthest quartile of driving distance: OR 0.65, 95% CI, 0.52-–0.81). Longer hospitalizations were associated with higher NS/SDC rates (length of stay ≥15 days: OR 1.51, 95% CI, 1.22-1.88). In contrast, discharges from subspecialty services (OR 0.79, 95% CI, 0.68-0.93) had independently lower NS/SDC rates. Compared to discharges to home without services, NS/SDC rates were higher with discharges to home with services (OR 1.32, 95% CI, 1.01-1.36) and with discharges to facilities (OR 2.10, 95% CI, 1.70-2.60).

The presence of exactly 2 PDFVs per hospitalization was also associated with higher NS/SDC rates (OR 1.17, 95% CI, 1.01-1.36), compared to a single PDFV per hospitalization; however, the presence of more than 2 PDFVs per hospitalization was associated with lower NS/SDC rates (OR 0.82, 95% CI, 0.69-0.98). A separate analysis (data not shown) of potential high utilizers revealed a 15% NS/SDC rate for the top 0.5% of patients (median: 18 PDFVs each) and an 18% NS/SDC rate for the top 1% of patients (median: 14 PDFVs each) with regard to the numbers of PDFVs scheduled, compared to the 25% overall NS/SDC rate for all patients.

NS/SDC rates and adjusted ORs with regard to individual PDFV characteristics are displayed in Table 3. Nononcologic visits had higher NS/SDC rates than oncologic visits; for example, the NS/SDC rate for primary care visits was 39% (OR 2.62, 95% CI, 2.03-3.38), compared to 12% for oncologic visits. Appointments in the “other” specialty category also had high nonattendance rates, as further described in Supplementary Table B. Summertime appointments were more likely to be attended (OR 0.81, 95% CI, 0.68-0.97) compared to those in the spring. PDFV lead time (the time interval between the discharge date and appointment date) was not associated with changes in visit attendance.

 

DISCUSSION

PDFVs were scheduled on patients’ behalf for more than half of all medical hospitalizations at our institution, a rate that is consistent with previous studies.^10,11,26 However, 1 in 4 of these PDFVs resulted in a NS/SDC. This figure contrasts sharply with our institution’s 10% overall NS/SDC rate for all outpatient visits (S. Schlegel, written communication, July 2016). In our study, patients who were younger, black, or Medicaid insured were more likely to miss their follow-up visits. Patients who lived farther from the study hospital had lower NS/SDC rates, which is consistent with another study of a low-income, urban patient population.²⁷ In contrast, patients with longer lengths of stay, discharges with home care services, or discharges to another facility were more likely to miss their PDFVs. Reasons for this are likely multifactorial, including readmission to a hospital or feeling too unwell to leave home to attend PDFVs. Insurance policies regarding ambulance reimbursement and outpatient billing can cause confusion and may have contributed to higher NS/SDC rates for facility-bound patients.^28,29

When comparing PDFV characteristics themselves, oncologic visits had the lowest NS/SDC incidence of any group analyzed in our study. This may be related to the inherent life-altering nature of a cancer diagnosis or our cancer center’s use of patient navigators.^23,30 In contrast, primary care clinics suffered from NS/SDC rates approaching 40%, which is a concerning finding given the importance of primary care coordination in the posthospitalization period.^9,31 Why are primary care appointments so commonly missed? Some studies suggest that forgetting about a primary care appointment is a leading reason.^15,32,33 For PDFVs, this phenomenon may be augmented because the visits are not scheduled by patients themselves. Additionally, patients may paradoxically undervalue the benefit of an all-encompassing primary care visit, compared to a PDFV focused on a specific problem, (eg, a cardiology follow-up appointment for a patient with congestive heart failure). In particular, patients with limited health literacy may potentially undervalue the capabilities of their primary care clinics.^34,35

The low absolute number of primary care PDFVs (only 8% of all visits) scheduled for patients at our hospital was an unexpected finding. This low percentage is likely a function of the patient population hospitalized at our large, urban quaternary-care facility. First, an unknown number of patients may have had PDFVs manually scheduled with primary care providers external to our health system; these PDFVs were not captured within our study. Second, 71% of the hospitalizations in our study occurred in subspecialty services, for which specific primary care follow-up may not be as urgent. Supporting this fact, further analysis of the 6136 hospitalizations in our study (data not shown) revealed that 28% of the hospitalizations in general internal medicine were scheduled with at least 1 primary care PDFV as opposed to only 5% of subspecialty-service hospitalizations.

In contrast to several previous studies of outpatient nonattendance,we did not find that visits scheduled for time points further in the future were more likely to be missed.^{14,24,25,36,37} Unlike other appointments, it may be that PDFV lead time does not affect attendance because of the unique manner in which PDFV times are scheduled and conveyed to patients. Unlike other appointments, patients do not schedule PDFVs themselves but instead learn about their PDFV dates as part of a large set of discharge instructions. This practice may result in poor recall of PDFV dates in recently hospitalized patients³⁸, regardless of the lead time between discharge and the visit itself.

Supplementary Table 1 details a 51% NS/SDC rate for the small number of PDFVs (n = 65) that were excluded a priori from our analysis because of general ineligibility for UPHS outpatient care. We specifically chose to exclude this population because of the infrequent and irregular process by which these PDFVs were authorized on a case-by-case basis, typically via active engagement by our hospital’s social work department. We did not study this population further but postulate that the 51% NS/SDC rate may reflect other social determinants of health that contribute to appointment nonadherence in a predominantly uninsured population.

Beyond their effect on patient outcomes, improving PDFV-related processes has the potential to boost both inpatient and outpatient provider satisfaction. From the standpoint of frontline inpatient providers (often resident physicians), calling outpatient clinics to request PDFVs is viewed as 1 of the top 5 administrative tasks that interfere with house staff education.³⁹ Future interventions that involve patients in the PDFV scheduling process may improve inpatient workflow while simultaneously engaging patients in their own care. For example, asking clinic representatives to directly schedule PDFVs with hospitalized patients, either by phone or in person, has been shown in pilot studies to improve PDFV attendance and decrease readmissions.^40-42 Conversely, NS/SDC visits harm outpatient provider productivity and decrease provider availability for other patients.^13,17,43 Strategies to mitigate the impact of unfilled appointment slots (eg, deliberately overbooking time slots in advance) carry their own risks, including provider burnout.⁴⁴ As such, preventing NSs may be superior to curing their adverse impacts. Many such strategies exist in the ambulatory setting,^13,43,45 for example, better communication with patients through texting or goal-directed, personalized phone reminders.^46-48Our study methodology has several limitations. Most importantly, we were unable to measure PDFVs made with providers unaffiliated with UPHS. As previously noted, our low proportion of primary care PDFVs may specifically reflect patients with primary care providers outside of our health system. Similarly, our low percentage of Medicaid patients receiving PDFVs may be related to follow-up visits with nonaffiliated community health centers. We were unable to measure patient acuity and health literacy as potential predictors of NS/SDC rates. Driving distances were calculated from patient postal codes to our hospital, not to individual outpatient clinics. However, the majority of our hospital-affiliated clinics are located adjacent to our hospital; additionally, we grouped driving distances into quartiles for our analysis. We had initially attempted to differentiate between clinic-initiated and patient-initiated cancellations, but unfortunately, we found that the data were too unreliable to be used for further analysis (outlined in Supplementary Table 3). Lastly, because we studied patients in medical units at a single large, urban, academic center, our results are not generalizable to other settings (eg, community hospitals, hospitals with smaller networks of outpatient providers, or patients being discharged from surgical services or observation units).

 

CONCLUSION

Given national efforts to enhance postdischarge transitions of care, we aimed to analyze attendance at provider-scheduled PDFV appointments. Our finding that 25% of PDFVs resulted in NS/SDCs raises both questions and opportunities for inpatient and outpatient providers. Further research is needed to understand why so many patients miss their PDFVs, and we should work as a field to develop creative solutions to improve PDFV scheduling and attendance.

Acknowledgments

The authors acknowledge Marie Synnestvedt, PhD, and Manik Chhabra, MD, for their assistance with data gathering and statistical analysis. They also acknowledge Allison DeKosky, MD, Michael Serpa, BS, Michael McFall, and Scott Schlegel, MBA, for their assistance with researching this topic. They did not receive external compensation for their assistance outside of their usual salary support.

DISCLOSURE

Nothing to report.

INTRODUCTION

According to Centers for Medicare & Medicaid Services (CMS), approximately 1 in 5 patients discharged from a hospital will be readmitted within 30 days.¹ The Hospital Readmission Reduction Program (HRRP) is designed to reduce readmission by withholding up to 3% of all Medicare reimbursement from hospitals with “excess” readmissions; however, absent from the HRRP is adjustment for socioeconomic status (SES), which CMS holds may undermine incentives to reduce health disparities and institutionalize lower standards for hospitals serving disadvantaged populations.²

Lack of SES adjustment has been criticized by those who point to evidence highlighting postdischarge environment and patient SES as drivers of readmission and suggest hospitals that serve low SES individuals will bear a disproportionate share of penalties.^3-6 Single-center,^3,7,8 regional,^9,10 and nationwide^6,11 studies highlight census tract level socioeconomic variables as predictive of readmission. Single-center studies, robust in controlling for confounders, including staffing, training, electronic medical record utilization, and transitional care processes, do not allow comparisons between hospitals, limiting utility in HRRP evaluation. Multicenter cohorts, on the other hand, allow for comparisons between high and low penalty hospitals, pioneered by Joynt et al¹² after the first round of HRRP penalties; yet this technique may not account for confounding caused by extensive demographic, socioeconomic, and hospital characteristic heterogeneity inherent in a national cohort. Analysis of the 2015 HRRP penalty data by Sjoding et al.⁶ revealed higher chronic obstructive pulmonary disease (COPD) readmission rates in the Mid-Atlantic, Midwest, and South relative to other regions; however, the magnitude of small-area variation and its relationship to population SES have yet to be characterized.

Therefore, we conducted a matched case-control design, whereby each maximum penalty hospital was matched to a nonpenalty hospital using key hospital characteristics. We then used geographic matching to isolate SES factors predictive of readmission within specific geographies in an effort to control for regional population differences. We hypothesized that, among both matched and localized hospital pairs, the disparities in population SES are the most significant predictors of a maximum penalty. Now in the 3rd year of the HRRP with approximately 75% of eligible hospitals to receive penalties worth an estimated $428 million in the 2015 fiscal year,¹³ we offer a small-area analysis of bipolar extremes to inform debate surrounding the HRRP with evidence regarding the causes and implications of readmission penalties.

METHODS

Study Design and Sample

This study relies on a case-control design. The cases were defined as US hospitals to receive the maximum 3% HRRP penalty in fiscal year 2015. Controls were drawn from the cohort of hospitals potentially subject to HRRP penalties that received no readmission penalty in the 2015 fiscal year with at least 1 admission for any of the following conditions: heart failure (HF), acute myocardial infarction (AMI), pneumonia (PN), total knee arthroscopy or total hip arthroscopy (THA/TKA), or chronic obstructive pulmonary disease (COPD).

Data Sources

Penalty data were drawn from the 2015 master penalty file,¹⁴ which were accessed via CMS.gov. County-level demographic and socioeconomic data were gathered from the 2015 American Community Survey (ACS), a subsidiary of the US Census. Data on hospital characteristics, capacity, and regional healthcare utilization were drawn from 2012 Dartmouth Atlas,¹⁵ 2012 Medicare Cost Report,¹⁶ 2012 American Hospital Association Hospital Statistics Database, and 2014 Hospital Care Downloadable Database.

Hospital-level CMS data were linked to the master 2015 penalty file. Dartmouth Atlas data were subsequently linked to the file using the Dartmouth Atlas “Hospital to HSA/HRR Crosswalk” file (accessed via DartmouthAtlas.org.) Each hospital was assigned the profile of the hospital service area (HSA) and hospital referral region (HRR) in which it is located. An HSA is a geographic region defined by hospital admissions; the majority, but not entirety, of residents of a given HSA utilize the corresponding hospital. Similarly, an HRR is a geographic region defined by referrals for major cardiovascular and neurosurgery procedures. County-level socioeconomic data were linked to the dataset by county name; thus, hospital socioeconomic profiles are based on the county in which they are located.

 

Case-Control Matching

In the primary analysis, coarsened exact matching (CEM) matched controls to cases by potential confounding hospital characteristics, including the following: ownership, number of beds, case mix index (measure of acuity), ambulatory care visit rates within 14 days of discharge, and total number of penalty-eligible cases, including HF, AMI, COPD, PN, and THA/TKA.

In the secondary analysis, hospitals were geocoded by zip code. Geographic Information Systems mapping software (ESRI ArcGIS, Redlands, CA) relied upon Euclidean allocation distance spatial analysis^17,18 to match each maximum-penalty hospital to the nearest nonpenalty hospital. Each case was matched to a distinct control; duplicate controls were replaced with the nearest unmatched no-penalty hospital.

Statistical Analysis

Univariate analyses utilized unpaired Student t tests (primary analysis) and paired Student t tests (secondary analysis). The CEM algorithm matches by strata rather than pairs, precluding paired Student t tests in the primary analysis. Statistical analyses were conducted using STATA (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, TX).

RESULTS

Maximum Penalty and Nonpenalty Hospital Matching

Of 3383 hospitals eligible for the HRRP, 39 received the maximum penalty and 770 received no penalty. Thirty-eight control hospitals were identified using CEM algorithm; 1 maximum-penalty hospital could not be matched and was excluded from primary analy

sis.

Hospital Characteristics

Case and control profiles are presented in Table 1. Cases and controls were matched by characteristics which may impact readmission rates (Table 1). CEM yielded cohorts similar across a spectrum of metrics, and identical in terms of matching criteria including ownership, beds (quartile), case mix index (above median), ambulatory care visit within 14 days of discharge (above median), and total number of penalty-eligible cases (above median). Relative to no-penalty hospitals, maximum-penalty hospitals were more likely rural (n = 9 vs n = 2, P = 0.022) and have a less profitable operating margin (0.1% vs 6.9%), and location within HSAs with higher age, sex, and race adjusted hospital-wide mortality rate (5.3% vs 4.9%, P = 0.009) and higher rates of discharge for ambulatory care sensitive conditions (108 vs 63 discharges per 1000 Medicare enrollees).

Demographic and Socioeconomic Characteristics

As presented in Table 2, cases a

nd controls are in counties with similar age, sex, and ethnicity profiles. Per capita income was similar between cohorts. However, relative to non-penalty hospitals, maximum-penalty hospitals are in counties with a larger percentage of individuals below the poverty line (19.1% vs 15.5%, P = 0.015), a larger percentage of individuals qualifying for food stamp benefits (16.8% vs 12.7%, P = 0.005), lower rates of labor force participation (57.0% vs 63.6%), and lower rates of high school graduation (82.2% vs 87.5%, P = 0.0011).

Secondary Analysis: Geographical Matching

Secondary analysis matched each maximum-penalty hospital to the nearest no-penalty hospital using a global information system vector analysis algorithm. As shown in the Figure, median distance between the case and the control was 42.5 miles (interquartile range: 25th percentile, 15.4 miles; 75th percentile, 98.4 miles). Seventeen pairs (44%) were in the same HRR, 6 of which were in the same HSA. Seven pairs (18%) were within the same county

.

Secondary Analysis: Economic and Demographic Profiles of Geographically Matched Pairs

Demographic and socioeconomic profiles are presented in Table 3. The cases and controls are in counties with similar age, sex, and ethnicity distributions. Relative to no-penalty hospitals, maximum-penalty hospitals are in counties with lower socioeconomic profiles, including increased rates of poverty (15.6% vs 19.2%, P = 0.007) and lower rates of high school (86.4% vs 82.1%, P = 0.005) or college graduation (22.3% vs 28.1%, P = 0.002). Seven pairs were in the same county; a sensitivity analysis excluding these hospitals revealed similarly lower SES profile in cases relative to controls (Supplementary Table 1).

DISCUSSION

Our analysis reveals that county-level socioeconomic profiles are predictors of maximum HRRP penalties. Specifically, after matching cases and controls on 5 hospital characteristics that may influence readmission, maximum-penalty hospitals were more likely to be in rural counties with higher rates of poverty and lower rates of education relative to no-penalty hospitals. We observed no difference between cases and controls with respect to age, sex, or ethnicity.

Our study complement

s that of Joynt et al.,¹² whose analysis of the first year of the HRRP revealed safety net hospitals (top quartile in disproportionate share index) had nearly double the odds to receive a high penalty (highest 50% of penalties). We add to current literature with evidence that national and regional variation in readmission penalties is associated with income and education but not race and ethnicity. Others have shown racial and ethnic disparities in readmission rates even after adjusting for income and disease severity,^19,20 leading the American Hospital Association to call for race and ethnicity adjustments of HRRP penalties.²¹ In contrast, we offer evidence that maximum penalties are not a function of race or ethnicity.

 

Maximum Penalties as a Function of Population Health

The Dartmouth Atlas of Healthcare measures health outcomes, which are regionally aggregated among local hospitals by either HSA or HRR; see Methods. Such small-area aggregation does not precisely reflect outcomes from a specific hospital, but rather it describes the health status of localities. Disparities in health outcomes exist between maximum-penalty and no-penalty HSAs. Complication rates were slightly higher in maximum penalty HSAs, consistent with studies highlighting complications as drivers of surgical readmissions.^22,23 Moreover, hospital-wide mortality rates were higher in maximum-penalty areas relative to nonpenalty HSAs (5.3 vs 4.9, P = 0.009).

Using national data, Krumholz et al. found no correlation between rates of readmission and mortality for HF, AMI, and PN²⁴, which is a phenomenon acknowledged by the Medicare Payment Advisory Commission (MedPac) in a 2013 report titled, “Refining the hospital readmission reduction program.”²⁵ In large national studies, others have shown low SES to be associated with elevated readmission but not mortality.^10,11 In contrast, we restricted our analysis to matched cohorts and are, to our knowledge, the first to present evidence of an association between readmission and hospital-wide mortality adjusted for age, sex, and ethnicity.

Our results suggest maximum readmission penalties are a function of population health and public health capacity. The rates of ambulatory care sensitive condition (ACSC) discharges were substantially higher in HSAs of maximum penalty hospitals relative to nonpenalty hospitals (108 vs 63 per 1000 Medicare enrollees, P < 0.001). ACSC discharges have been used to measure primary care quality for 30 years, with the assumption being that admission for chronic conditions, such as HF, can be prevented with effective primary care.^26,27 Moreover, patients discharged from maximum-penalty hospitals were more likely to have an emergency room visit within 30 days of discharge (20.8% vs 18.4%, P < 0.001). Higher rates of ACSCs and postdischarge emergency department visits suggest outpatient resources in maximum-penalty service areas struggle to manage the disease burden of high-risk populations. Geography may be a contributor; maximum-penalty hospitals were more likely to be rural than no-penalty hospitals (24% vs 5%, P = 0.022).

Our findings suggest hospitals providing care to vulnerable communities (defined by low income, low education, and high rates of ambulatory sensitive discharges) are disproportionately penalized. McHugh et al. revealed high nurse staffing levels to be protective against readmission penalties²⁸, yet high penalties to low-margin hospitals may encourage reduced rather than increased staff. It may be better policy to direct resources rather than penalties to underserved communities; our findings echo others with concern about disproportionate penalties to hospitals serving low SES patients.^2,5,6,29

Secondary Analysis: Geographic Matching

Geographic matching paired each maximum-penalty hospital to the nearest no-penalty hospital in an attempt to control for unmeasured regional factors that may confound an association between socioeconomic profile and health outcomes. For example, cost of living ^{30, 31} and obesity ^32,33 vary regionally. Our study was unequipped to assess potential regional confounders; we attempted to control for them with geographical matching.

Median distance between maximum-penalty and no-penalty hospitals was 42.5 miles. Seven pairs were located within the same county, thus both case and control were assigned the same socioeconomic profile. Despite close proximity and identical SES profile in 7 of 39 pairs, maximum-penalty hospitals were in counties with lower income and lower rates of education, strengthening the association between SES and maximum readmission penalties.

Implications and Future Directions

In response to criticism surrounding the HRRP, the National Quality Forum endorsed the general concept of SES adjustment for hospital quality measures.³⁴ Subsequently, in a briefing dated March 24, 2015, MedPAC, a government agency which provides Medicare policy analysis to Congress, proposed an SES adjustment methodology of “dividing hospitals into peer groups based on their overall share of low-income Medicare patients, and then setting a benchmark readmissions target for each peer group”;³⁵ in other words, lower standards for hospitals that serve low-income populations. MedPAC’s proposal will reduce penalties to “safety net” institutions, which is progress but not a solution. Although the HRRP appears to be working, according to the US Department of Health and Human Services, readmissions fell by 150,000 between January 2012 and February 2013,³⁶ we are concerned neither the HRRP nor the MedPac revision proposal considers geographic and environmental components of readmission. The HRRP promotes national improvement in exchange for regional regression.

Fair quality measures are key to value-based reimbursement models; yet, implicit in penalties for excess readmissions is the assumed ability to calculate hospital performance targets. Benchmarks for safety, timely care, and patient satisfaction can be uniform; rates of central line infections should not be influenced by patient mix. However, 9 of the 39 maximum-penalty hospitals under the HRRP are in rural Kentucky; one could hypothesize many reasons why rural Kentucky is a hotbed for excess readmission, including the regional production of tobacco and bourbon.

The fundamental question raised by our study is whether poor performance on quality measures is a function of underperforming hospitals or a manifestation of underserved communities. Moving forward, we encourage data systems and study designs that focus research on geospatial distribution of population health within the context of social and behavioral health determinants.³⁷ Small-area studies of factors that drive health outcomes will inform rational alignment of healthcare policies and resources (including penalties and incentives) with underlying population needs.

 

Strengths and Weaknesses

Matching is a strength of the study. Primary analysis matched case and controls by hospital characteristics, generating cohorts similar across a spectrum of hospital metrics. Therefore, variation in readmission rates was less likely confounded by hospital characteristics. The secondary analysis was matched by geography in an effort to adjust for unmeasured, regional factors, including obesity and cost of living that may confound an association between SES and health outcomes. Geographic matching adds strength to our assertion that SES drives distinction between maximum-penalty hospitals and nonpenalty hospitals.

One weakness was the regional unit of analysis for socioeconomic and Dartmouth Atlas data, which is not a precise profile of the corresponding hospital. Each hospital was assigned a county-level socioeconomic profile. A more robust methodology would analyze patient-level SES data; this was impractical given a cohort of 78 hospitals. Regional health outcomes data limits analysis of readmission penalties as a function of hospital quality. Instead, regional data facilitated associations between readmission and population health, consistent with the aim of our study.

We analyzed 116 of 3668 hospitals eligible for the HRRP (3.2%), limiting the generalizability of our findings. Eighty-four percent of hospitals in the primary analysis have below the median number of beds, and none of them are teaching hospitals. Our analysis, restricted to maximum-penalty and no-penalty cohorts, does not address potential association between submaximal readmission penalties and socioeconomics.

Both matching techniques potentially controlled for similar SES factors and skewed our results towards null, especially in terms of race and ethnicity. Geographic matching generated 7 pairs (18%) within in the same county; both maximum-penalty and no-penalty hospitals were assigned the same socioeconomic profile, as well as 6 pairs (15%) within the same HSA, and both cases and controls had identical Dartmouth Atlas health outcomes profiles. We retained these pairs in our analysis to avoid artificially inflating SES and population health differences between cohorts.

Thirty-nine hospitals received a maximum penalty in the 3rd year of the HRRP. Relative to geographically matched no-penalty hospitals, maximum-penalty hospitals were more likely to be rural and located in counties with less educational attainment, more poverty and more poorly controlled chronic disease. In contrast to nationwide studies, a matched analysis plan revealed no difference between the cohorts in terms of race and ethnicity and provided evidence that maximum penalty hospitals had higher rates of age-, sex-, and race-adjusted hospital-wide mortality.

Our results highlight potential consequences of nationally derived benchmarks for phenomena underpinned by social, behavioral, and environmental factors and raise the question of whether maximum HRRP penalties are a consequence of underperforming hospitals or a manifestation of underserved communities. We are encouraged by MedPAC’s proposal to stratify HRRP by SES, yet recommend further small-area geographic analyses to better align quality measures, penalties, and incentives with resources and needs of distinct populations.

Acknowledgments

The authors thank William Hisey, who laid the foundation for the analysis and without whom the project would not have been possible.

DISCLOSURE

The authors certify that none of the material in this manuscript has been previously published and that none of this material is currently under consideration for publication elsewhere. This project received no funding. None of the authors on this manuscript have any commercial relationships to disclose in relation to this manuscript. All authors have reviewed and approved this manuscript and have contributed significantly to the design, conduct, and/or analysis of the research. No authors have any financial interests to disclose. No authors have any potential conflicts of interest to disclose. No authors have financial or personal relationships with any of the subject material presented in the manuscript.

INTRODUCTION

According to Centers for Medicare & Medicaid Services (CMS), approximately 1 in 5 patients discharged from a hospital will be readmitted within 30 days.¹ The Hospital Readmission Reduction Program (HRRP) is designed to reduce readmission by withholding up to 3% of all Medicare reimbursement from hospitals with “excess” readmissions; however, absent from the HRRP is adjustment for socioeconomic status (SES), which CMS holds may undermine incentives to reduce health disparities and institutionalize lower standards for hospitals serving disadvantaged populations.²

Lack of SES adjustment has been criticized by those who point to evidence highlighting postdischarge environment and patient SES as drivers of readmission and suggest hospitals that serve low SES individuals will bear a disproportionate share of penalties.^3-6 Single-center,^3,7,8 regional,^9,10 and nationwide^6,11 studies highlight census tract level socioeconomic variables as predictive of readmission. Single-center studies, robust in controlling for confounders, including staffing, training, electronic medical record utilization, and transitional care processes, do not allow comparisons between hospitals, limiting utility in HRRP evaluation. Multicenter cohorts, on the other hand, allow for comparisons between high and low penalty hospitals, pioneered by Joynt et al¹² after the first round of HRRP penalties; yet this technique may not account for confounding caused by extensive demographic, socioeconomic, and hospital characteristic heterogeneity inherent in a national cohort. Analysis of the 2015 HRRP penalty data by Sjoding et al.⁶ revealed higher chronic obstructive pulmonary disease (COPD) readmission rates in the Mid-Atlantic, Midwest, and South relative to other regions; however, the magnitude of small-area variation and its relationship to population SES have yet to be characterized.

Therefore, we conducted a matched case-control design, whereby each maximum penalty hospital was matched to a nonpenalty hospital using key hospital characteristics. We then used geographic matching to isolate SES factors predictive of readmission within specific geographies in an effort to control for regional population differences. We hypothesized that, among both matched and localized hospital pairs, the disparities in population SES are the most significant predictors of a maximum penalty. Now in the 3rd year of the HRRP with approximately 75% of eligible hospitals to receive penalties worth an estimated $428 million in the 2015 fiscal year,¹³ we offer a small-area analysis of bipolar extremes to inform debate surrounding the HRRP with evidence regarding the causes and implications of readmission penalties.

METHODS

Study Design and Sample

This study relies on a case-control design. The cases were defined as US hospitals to receive the maximum 3% HRRP penalty in fiscal year 2015. Controls were drawn from the cohort of hospitals potentially subject to HRRP penalties that received no readmission penalty in the 2015 fiscal year with at least 1 admission for any of the following conditions: heart failure (HF), acute myocardial infarction (AMI), pneumonia (PN), total knee arthroscopy or total hip arthroscopy (THA/TKA), or chronic obstructive pulmonary disease (COPD).

Data Sources

Penalty data were drawn from the 2015 master penalty file,¹⁴ which were accessed via CMS.gov. County-level demographic and socioeconomic data were gathered from the 2015 American Community Survey (ACS), a subsidiary of the US Census. Data on hospital characteristics, capacity, and regional healthcare utilization were drawn from 2012 Dartmouth Atlas,¹⁵ 2012 Medicare Cost Report,¹⁶ 2012 American Hospital Association Hospital Statistics Database, and 2014 Hospital Care Downloadable Database.

Hospital-level CMS data were linked to the master 2015 penalty file. Dartmouth Atlas data were subsequently linked to the file using the Dartmouth Atlas “Hospital to HSA/HRR Crosswalk” file (accessed via DartmouthAtlas.org.) Each hospital was assigned the profile of the hospital service area (HSA) and hospital referral region (HRR) in which it is located. An HSA is a geographic region defined by hospital admissions; the majority, but not entirety, of residents of a given HSA utilize the corresponding hospital. Similarly, an HRR is a geographic region defined by referrals for major cardiovascular and neurosurgery procedures. County-level socioeconomic data were linked to the dataset by county name; thus, hospital socioeconomic profiles are based on the county in which they are located.

 

Case-Control Matching

In the primary analysis, coarsened exact matching (CEM) matched controls to cases by potential confounding hospital characteristics, including the following: ownership, number of beds, case mix index (measure of acuity), ambulatory care visit rates within 14 days of discharge, and total number of penalty-eligible cases, including HF, AMI, COPD, PN, and THA/TKA.

In the secondary analysis, hospitals were geocoded by zip code. Geographic Information Systems mapping software (ESRI ArcGIS, Redlands, CA) relied upon Euclidean allocation distance spatial analysis^17,18 to match each maximum-penalty hospital to the nearest nonpenalty hospital. Each case was matched to a distinct control; duplicate controls were replaced with the nearest unmatched no-penalty hospital.

Statistical Analysis

Univariate analyses utilized unpaired Student t tests (primary analysis) and paired Student t tests (secondary analysis). The CEM algorithm matches by strata rather than pairs, precluding paired Student t tests in the primary analysis. Statistical analyses were conducted using STATA (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, TX).

RESULTS

Maximum Penalty and Nonpenalty Hospital Matching

Of 3383 hospitals eligible for the HRRP, 39 received the maximum penalty and 770 received no penalty. Thirty-eight control hospitals were identified using CEM algorithm; 1 maximum-penalty hospital could not be matched and was excluded from primary analy

sis.

Hospital Characteristics

Case and control profiles are presented in Table 1. Cases and controls were matched by characteristics which may impact readmission rates (Table 1). CEM yielded cohorts similar across a spectrum of metrics, and identical in terms of matching criteria including ownership, beds (quartile), case mix index (above median), ambulatory care visit within 14 days of discharge (above median), and total number of penalty-eligible cases (above median). Relative to no-penalty hospitals, maximum-penalty hospitals were more likely rural (n = 9 vs n = 2, P = 0.022) and have a less profitable operating margin (0.1% vs 6.9%), and location within HSAs with higher age, sex, and race adjusted hospital-wide mortality rate (5.3% vs 4.9%, P = 0.009) and higher rates of discharge for ambulatory care sensitive conditions (108 vs 63 discharges per 1000 Medicare enrollees).

Demographic and Socioeconomic Characteristics

As presented in Table 2, cases a

nd controls are in counties with similar age, sex, and ethnicity profiles. Per capita income was similar between cohorts. However, relative to non-penalty hospitals, maximum-penalty hospitals are in counties with a larger percentage of individuals below the poverty line (19.1% vs 15.5%, P = 0.015), a larger percentage of individuals qualifying for food stamp benefits (16.8% vs 12.7%, P = 0.005), lower rates of labor force participation (57.0% vs 63.6%), and lower rates of high school graduation (82.2% vs 87.5%, P = 0.0011).

Secondary Analysis: Geographical Matching

Secondary analysis matched each maximum-penalty hospital to the nearest no-penalty hospital using a global information system vector analysis algorithm. As shown in the Figure, median distance between the case and the control was 42.5 miles (interquartile range: 25th percentile, 15.4 miles; 75th percentile, 98.4 miles). Seventeen pairs (44%) were in the same HRR, 6 of which were in the same HSA. Seven pairs (18%) were within the same county

.

Secondary Analysis: Economic and Demographic Profiles of Geographically Matched Pairs

Demographic and socioeconomic profiles are presented in Table 3. The cases and controls are in counties with similar age, sex, and ethnicity distributions. Relative to no-penalty hospitals, maximum-penalty hospitals are in counties with lower socioeconomic profiles, including increased rates of poverty (15.6% vs 19.2%, P = 0.007) and lower rates of high school (86.4% vs 82.1%, P = 0.005) or college graduation (22.3% vs 28.1%, P = 0.002). Seven pairs were in the same county; a sensitivity analysis excluding these hospitals revealed similarly lower SES profile in cases relative to controls (Supplementary Table 1).

DISCUSSION

Our analysis reveals that county-level socioeconomic profiles are predictors of maximum HRRP penalties. Specifically, after matching cases and controls on 5 hospital characteristics that may influence readmission, maximum-penalty hospitals were more likely to be in rural counties with higher rates of poverty and lower rates of education relative to no-penalty hospitals. We observed no difference between cases and controls with respect to age, sex, or ethnicity.

Our study complement

s that of Joynt et al.,¹² whose analysis of the first year of the HRRP revealed safety net hospitals (top quartile in disproportionate share index) had nearly double the odds to receive a high penalty (highest 50% of penalties). We add to current literature with evidence that national and regional variation in readmission penalties is associated with income and education but not race and ethnicity. Others have shown racial and ethnic disparities in readmission rates even after adjusting for income and disease severity,^19,20 leading the American Hospital Association to call for race and ethnicity adjustments of HRRP penalties.²¹ In contrast, we offer evidence that maximum penalties are not a function of race or ethnicity.

 

Maximum Penalties as a Function of Population Health

The Dartmouth Atlas of Healthcare measures health outcomes, which are regionally aggregated among local hospitals by either HSA or HRR; see Methods. Such small-area aggregation does not precisely reflect outcomes from a specific hospital, but rather it describes the health status of localities. Disparities in health outcomes exist between maximum-penalty and no-penalty HSAs. Complication rates were slightly higher in maximum penalty HSAs, consistent with studies highlighting complications as drivers of surgical readmissions.^22,23 Moreover, hospital-wide mortality rates were higher in maximum-penalty areas relative to nonpenalty HSAs (5.3 vs 4.9, P = 0.009).

Using national data, Krumholz et al. found no correlation between rates of readmission and mortality for HF, AMI, and PN²⁴, which is a phenomenon acknowledged by the Medicare Payment Advisory Commission (MedPac) in a 2013 report titled, “Refining the hospital readmission reduction program.”²⁵ In large national studies, others have shown low SES to be associated with elevated readmission but not mortality.^10,11 In contrast, we restricted our analysis to matched cohorts and are, to our knowledge, the first to present evidence of an association between readmission and hospital-wide mortality adjusted for age, sex, and ethnicity.

Our results suggest maximum readmission penalties are a function of population health and public health capacity. The rates of ambulatory care sensitive condition (ACSC) discharges were substantially higher in HSAs of maximum penalty hospitals relative to nonpenalty hospitals (108 vs 63 per 1000 Medicare enrollees, P < 0.001). ACSC discharges have been used to measure primary care quality for 30 years, with the assumption being that admission for chronic conditions, such as HF, can be prevented with effective primary care.^26,27 Moreover, patients discharged from maximum-penalty hospitals were more likely to have an emergency room visit within 30 days of discharge (20.8% vs 18.4%, P < 0.001). Higher rates of ACSCs and postdischarge emergency department visits suggest outpatient resources in maximum-penalty service areas struggle to manage the disease burden of high-risk populations. Geography may be a contributor; maximum-penalty hospitals were more likely to be rural than no-penalty hospitals (24% vs 5%, P = 0.022).

Our findings suggest hospitals providing care to vulnerable communities (defined by low income, low education, and high rates of ambulatory sensitive discharges) are disproportionately penalized. McHugh et al. revealed high nurse staffing levels to be protective against readmission penalties²⁸, yet high penalties to low-margin hospitals may encourage reduced rather than increased staff. It may be better policy to direct resources rather than penalties to underserved communities; our findings echo others with concern about disproportionate penalties to hospitals serving low SES patients.^2,5,6,29

Secondary Analysis: Geographic Matching

Geographic matching paired each maximum-penalty hospital to the nearest no-penalty hospital in an attempt to control for unmeasured regional factors that may confound an association between socioeconomic profile and health outcomes. For example, cost of living ^{30, 31} and obesity ^32,33 vary regionally. Our study was unequipped to assess potential regional confounders; we attempted to control for them with geographical matching.

Median distance between maximum-penalty and no-penalty hospitals was 42.5 miles. Seven pairs were located within the same county, thus both case and control were assigned the same socioeconomic profile. Despite close proximity and identical SES profile in 7 of 39 pairs, maximum-penalty hospitals were in counties with lower income and lower rates of education, strengthening the association between SES and maximum readmission penalties.

Implications and Future Directions

In response to criticism surrounding the HRRP, the National Quality Forum endorsed the general concept of SES adjustment for hospital quality measures.³⁴ Subsequently, in a briefing dated March 24, 2015, MedPAC, a government agency which provides Medicare policy analysis to Congress, proposed an SES adjustment methodology of “dividing hospitals into peer groups based on their overall share of low-income Medicare patients, and then setting a benchmark readmissions target for each peer group”;³⁵ in other words, lower standards for hospitals that serve low-income populations. MedPAC’s proposal will reduce penalties to “safety net” institutions, which is progress but not a solution. Although the HRRP appears to be working, according to the US Department of Health and Human Services, readmissions fell by 150,000 between January 2012 and February 2013,³⁶ we are concerned neither the HRRP nor the MedPac revision proposal considers geographic and environmental components of readmission. The HRRP promotes national improvement in exchange for regional regression.

Fair quality measures are key to value-based reimbursement models; yet, implicit in penalties for excess readmissions is the assumed ability to calculate hospital performance targets. Benchmarks for safety, timely care, and patient satisfaction can be uniform; rates of central line infections should not be influenced by patient mix. However, 9 of the 39 maximum-penalty hospitals under the HRRP are in rural Kentucky; one could hypothesize many reasons why rural Kentucky is a hotbed for excess readmission, including the regional production of tobacco and bourbon.

The fundamental question raised by our study is whether poor performance on quality measures is a function of underperforming hospitals or a manifestation of underserved communities. Moving forward, we encourage data systems and study designs that focus research on geospatial distribution of population health within the context of social and behavioral health determinants.³⁷ Small-area studies of factors that drive health outcomes will inform rational alignment of healthcare policies and resources (including penalties and incentives) with underlying population needs.

 

Strengths and Weaknesses

Matching is a strength of the study. Primary analysis matched case and controls by hospital characteristics, generating cohorts similar across a spectrum of hospital metrics. Therefore, variation in readmission rates was less likely confounded by hospital characteristics. The secondary analysis was matched by geography in an effort to adjust for unmeasured, regional factors, including obesity and cost of living that may confound an association between SES and health outcomes. Geographic matching adds strength to our assertion that SES drives distinction between maximum-penalty hospitals and nonpenalty hospitals.

One weakness was the regional unit of analysis for socioeconomic and Dartmouth Atlas data, which is not a precise profile of the corresponding hospital. Each hospital was assigned a county-level socioeconomic profile. A more robust methodology would analyze patient-level SES data; this was impractical given a cohort of 78 hospitals. Regional health outcomes data limits analysis of readmission penalties as a function of hospital quality. Instead, regional data facilitated associations between readmission and population health, consistent with the aim of our study.

We analyzed 116 of 3668 hospitals eligible for the HRRP (3.2%), limiting the generalizability of our findings. Eighty-four percent of hospitals in the primary analysis have below the median number of beds, and none of them are teaching hospitals. Our analysis, restricted to maximum-penalty and no-penalty cohorts, does not address potential association between submaximal readmission penalties and socioeconomics.

Both matching techniques potentially controlled for similar SES factors and skewed our results towards null, especially in terms of race and ethnicity. Geographic matching generated 7 pairs (18%) within in the same county; both maximum-penalty and no-penalty hospitals were assigned the same socioeconomic profile, as well as 6 pairs (15%) within the same HSA, and both cases and controls had identical Dartmouth Atlas health outcomes profiles. We retained these pairs in our analysis to avoid artificially inflating SES and population health differences between cohorts.

Thirty-nine hospitals received a maximum penalty in the 3rd year of the HRRP. Relative to geographically matched no-penalty hospitals, maximum-penalty hospitals were more likely to be rural and located in counties with less educational attainment, more poverty and more poorly controlled chronic disease. In contrast to nationwide studies, a matched analysis plan revealed no difference between the cohorts in terms of race and ethnicity and provided evidence that maximum penalty hospitals had higher rates of age-, sex-, and race-adjusted hospital-wide mortality.

Our results highlight potential consequences of nationally derived benchmarks for phenomena underpinned by social, behavioral, and environmental factors and raise the question of whether maximum HRRP penalties are a consequence of underperforming hospitals or a manifestation of underserved communities. We are encouraged by MedPAC’s proposal to stratify HRRP by SES, yet recommend further small-area geographic analyses to better align quality measures, penalties, and incentives with resources and needs of distinct populations.

Acknowledgments

The authors thank William Hisey, who laid the foundation for the analysis and without whom the project would not have been possible.

DISCLOSURE

The authors certify that none of the material in this manuscript has been previously published and that none of this material is currently under consideration for publication elsewhere. This project received no funding. None of the authors on this manuscript have any commercial relationships to disclose in relation to this manuscript. All authors have reviewed and approved this manuscript and have contributed significantly to the design, conduct, and/or analysis of the research. No authors have any financial interests to disclose. No authors have any potential conflicts of interest to disclose. No authors have financial or personal relationships with any of the subject material presented in the manuscript.

INTRODUCTION

According to Centers for Medicare & Medicaid Services (CMS), approximately 1 in 5 patients discharged from a hospital will be readmitted within 30 days.¹ The Hospital Readmission Reduction Program (HRRP) is designed to reduce readmission by withholding up to 3% of all Medicare reimbursement from hospitals with “excess” readmissions; however, absent from the HRRP is adjustment for socioeconomic status (SES), which CMS holds may undermine incentives to reduce health disparities and institutionalize lower standards for hospitals serving disadvantaged populations.²

Lack of SES adjustment has been criticized by those who point to evidence highlighting postdischarge environment and patient SES as drivers of readmission and suggest hospitals that serve low SES individuals will bear a disproportionate share of penalties.^3-6 Single-center,^3,7,8 regional,^9,10 and nationwide^6,11 studies highlight census tract level socioeconomic variables as predictive of readmission. Single-center studies, robust in controlling for confounders, including staffing, training, electronic medical record utilization, and transitional care processes, do not allow comparisons between hospitals, limiting utility in HRRP evaluation. Multicenter cohorts, on the other hand, allow for comparisons between high and low penalty hospitals, pioneered by Joynt et al¹² after the first round of HRRP penalties; yet this technique may not account for confounding caused by extensive demographic, socioeconomic, and hospital characteristic heterogeneity inherent in a national cohort. Analysis of the 2015 HRRP penalty data by Sjoding et al.⁶ revealed higher chronic obstructive pulmonary disease (COPD) readmission rates in the Mid-Atlantic, Midwest, and South relative to other regions; however, the magnitude of small-area variation and its relationship to population SES have yet to be characterized.

Therefore, we conducted a matched case-control design, whereby each maximum penalty hospital was matched to a nonpenalty hospital using key hospital characteristics. We then used geographic matching to isolate SES factors predictive of readmission within specific geographies in an effort to control for regional population differences. We hypothesized that, among both matched and localized hospital pairs, the disparities in population SES are the most significant predictors of a maximum penalty. Now in the 3rd year of the HRRP with approximately 75% of eligible hospitals to receive penalties worth an estimated $428 million in the 2015 fiscal year,¹³ we offer a small-area analysis of bipolar extremes to inform debate surrounding the HRRP with evidence regarding the causes and implications of readmission penalties.

METHODS

Study Design and Sample

This study relies on a case-control design. The cases were defined as US hospitals to receive the maximum 3% HRRP penalty in fiscal year 2015. Controls were drawn from the cohort of hospitals potentially subject to HRRP penalties that received no readmission penalty in the 2015 fiscal year with at least 1 admission for any of the following conditions: heart failure (HF), acute myocardial infarction (AMI), pneumonia (PN), total knee arthroscopy or total hip arthroscopy (THA/TKA), or chronic obstructive pulmonary disease (COPD).

Data Sources

Penalty data were drawn from the 2015 master penalty file,¹⁴ which were accessed via CMS.gov. County-level demographic and socioeconomic data were gathered from the 2015 American Community Survey (ACS), a subsidiary of the US Census. Data on hospital characteristics, capacity, and regional healthcare utilization were drawn from 2012 Dartmouth Atlas,¹⁵ 2012 Medicare Cost Report,¹⁶ 2012 American Hospital Association Hospital Statistics Database, and 2014 Hospital Care Downloadable Database.

Hospital-level CMS data were linked to the master 2015 penalty file. Dartmouth Atlas data were subsequently linked to the file using the Dartmouth Atlas “Hospital to HSA/HRR Crosswalk” file (accessed via DartmouthAtlas.org.) Each hospital was assigned the profile of the hospital service area (HSA) and hospital referral region (HRR) in which it is located. An HSA is a geographic region defined by hospital admissions; the majority, but not entirety, of residents of a given HSA utilize the corresponding hospital. Similarly, an HRR is a geographic region defined by referrals for major cardiovascular and neurosurgery procedures. County-level socioeconomic data were linked to the dataset by county name; thus, hospital socioeconomic profiles are based on the county in which they are located.

 

Case-Control Matching

In the primary analysis, coarsened exact matching (CEM) matched controls to cases by potential confounding hospital characteristics, including the following: ownership, number of beds, case mix index (measure of acuity), ambulatory care visit rates within 14 days of discharge, and total number of penalty-eligible cases, including HF, AMI, COPD, PN, and THA/TKA.

In the secondary analysis, hospitals were geocoded by zip code. Geographic Information Systems mapping software (ESRI ArcGIS, Redlands, CA) relied upon Euclidean allocation distance spatial analysis^17,18 to match each maximum-penalty hospital to the nearest nonpenalty hospital. Each case was matched to a distinct control; duplicate controls were replaced with the nearest unmatched no-penalty hospital.

Statistical Analysis

Univariate analyses utilized unpaired Student t tests (primary analysis) and paired Student t tests (secondary analysis). The CEM algorithm matches by strata rather than pairs, precluding paired Student t tests in the primary analysis. Statistical analyses were conducted using STATA (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, TX).

RESULTS

Maximum Penalty and Nonpenalty Hospital Matching

Of 3383 hospitals eligible for the HRRP, 39 received the maximum penalty and 770 received no penalty. Thirty-eight control hospitals were identified using CEM algorithm; 1 maximum-penalty hospital could not be matched and was excluded from primary analy

sis.

Hospital Characteristics

Case and control profiles are presented in Table 1. Cases and controls were matched by characteristics which may impact readmission rates (Table 1). CEM yielded cohorts similar across a spectrum of metrics, and identical in terms of matching criteria including ownership, beds (quartile), case mix index (above median), ambulatory care visit within 14 days of discharge (above median), and total number of penalty-eligible cases (above median). Relative to no-penalty hospitals, maximum-penalty hospitals were more likely rural (n = 9 vs n = 2, P = 0.022) and have a less profitable operating margin (0.1% vs 6.9%), and location within HSAs with higher age, sex, and race adjusted hospital-wide mortality rate (5.3% vs 4.9%, P = 0.009) and higher rates of discharge for ambulatory care sensitive conditions (108 vs 63 discharges per 1000 Medicare enrollees).

Demographic and Socioeconomic Characteristics

As presented in Table 2, cases a

nd controls are in counties with similar age, sex, and ethnicity profiles. Per capita income was similar between cohorts. However, relative to non-penalty hospitals, maximum-penalty hospitals are in counties with a larger percentage of individuals below the poverty line (19.1% vs 15.5%, P = 0.015), a larger percentage of individuals qualifying for food stamp benefits (16.8% vs 12.7%, P = 0.005), lower rates of labor force participation (57.0% vs 63.6%), and lower rates of high school graduation (82.2% vs 87.5%, P = 0.0011).

Secondary Analysis: Geographical Matching

Secondary analysis matched each maximum-penalty hospital to the nearest no-penalty hospital using a global information system vector analysis algorithm. As shown in the Figure, median distance between the case and the control was 42.5 miles (interquartile range: 25th percentile, 15.4 miles; 75th percentile, 98.4 miles). Seventeen pairs (44%) were in the same HRR, 6 of which were in the same HSA. Seven pairs (18%) were within the same county

.

Secondary Analysis: Economic and Demographic Profiles of Geographically Matched Pairs

Demographic and socioeconomic profiles are presented in Table 3. The cases and controls are in counties with similar age, sex, and ethnicity distributions. Relative to no-penalty hospitals, maximum-penalty hospitals are in counties with lower socioeconomic profiles, including increased rates of poverty (15.6% vs 19.2%, P = 0.007) and lower rates of high school (86.4% vs 82.1%, P = 0.005) or college graduation (22.3% vs 28.1%, P = 0.002). Seven pairs were in the same county; a sensitivity analysis excluding these hospitals revealed similarly lower SES profile in cases relative to controls (Supplementary Table 1).

DISCUSSION

Our analysis reveals that county-level socioeconomic profiles are predictors of maximum HRRP penalties. Specifically, after matching cases and controls on 5 hospital characteristics that may influence readmission, maximum-penalty hospitals were more likely to be in rural counties with higher rates of poverty and lower rates of education relative to no-penalty hospitals. We observed no difference between cases and controls with respect to age, sex, or ethnicity.

Our study complement

s that of Joynt et al.,¹² whose analysis of the first year of the HRRP revealed safety net hospitals (top quartile in disproportionate share index) had nearly double the odds to receive a high penalty (highest 50% of penalties). We add to current literature with evidence that national and regional variation in readmission penalties is associated with income and education but not race and ethnicity. Others have shown racial and ethnic disparities in readmission rates even after adjusting for income and disease severity,^19,20 leading the American Hospital Association to call for race and ethnicity adjustments of HRRP penalties.²¹ In contrast, we offer evidence that maximum penalties are not a function of race or ethnicity.

 

Maximum Penalties as a Function of Population Health

The Dartmouth Atlas of Healthcare measures health outcomes, which are regionally aggregated among local hospitals by either HSA or HRR; see Methods. Such small-area aggregation does not precisely reflect outcomes from a specific hospital, but rather it describes the health status of localities. Disparities in health outcomes exist between maximum-penalty and no-penalty HSAs. Complication rates were slightly higher in maximum penalty HSAs, consistent with studies highlighting complications as drivers of surgical readmissions.^22,23 Moreover, hospital-wide mortality rates were higher in maximum-penalty areas relative to nonpenalty HSAs (5.3 vs 4.9, P = 0.009).

Using national data, Krumholz et al. found no correlation between rates of readmission and mortality for HF, AMI, and PN²⁴, which is a phenomenon acknowledged by the Medicare Payment Advisory Commission (MedPac) in a 2013 report titled, “Refining the hospital readmission reduction program.”²⁵ In large national studies, others have shown low SES to be associated with elevated readmission but not mortality.^10,11 In contrast, we restricted our analysis to matched cohorts and are, to our knowledge, the first to present evidence of an association between readmission and hospital-wide mortality adjusted for age, sex, and ethnicity.

Our results suggest maximum readmission penalties are a function of population health and public health capacity. The rates of ambulatory care sensitive condition (ACSC) discharges were substantially higher in HSAs of maximum penalty hospitals relative to nonpenalty hospitals (108 vs 63 per 1000 Medicare enrollees, P < 0.001). ACSC discharges have been used to measure primary care quality for 30 years, with the assumption being that admission for chronic conditions, such as HF, can be prevented with effective primary care.^26,27 Moreover, patients discharged from maximum-penalty hospitals were more likely to have an emergency room visit within 30 days of discharge (20.8% vs 18.4%, P < 0.001). Higher rates of ACSCs and postdischarge emergency department visits suggest outpatient resources in maximum-penalty service areas struggle to manage the disease burden of high-risk populations. Geography may be a contributor; maximum-penalty hospitals were more likely to be rural than no-penalty hospitals (24% vs 5%, P = 0.022).

Our findings suggest hospitals providing care to vulnerable communities (defined by low income, low education, and high rates of ambulatory sensitive discharges) are disproportionately penalized. McHugh et al. revealed high nurse staffing levels to be protective against readmission penalties²⁸, yet high penalties to low-margin hospitals may encourage reduced rather than increased staff. It may be better policy to direct resources rather than penalties to underserved communities; our findings echo others with concern about disproportionate penalties to hospitals serving low SES patients.^2,5,6,29

Secondary Analysis: Geographic Matching

Geographic matching paired each maximum-penalty hospital to the nearest no-penalty hospital in an attempt to control for unmeasured regional factors that may confound an association between socioeconomic profile and health outcomes. For example, cost of living ^{30, 31} and obesity ^32,33 vary regionally. Our study was unequipped to assess potential regional confounders; we attempted to control for them with geographical matching.

Median distance between maximum-penalty and no-penalty hospitals was 42.5 miles. Seven pairs were located within the same county, thus both case and control were assigned the same socioeconomic profile. Despite close proximity and identical SES profile in 7 of 39 pairs, maximum-penalty hospitals were in counties with lower income and lower rates of education, strengthening the association between SES and maximum readmission penalties.

Implications and Future Directions

In response to criticism surrounding the HRRP, the National Quality Forum endorsed the general concept of SES adjustment for hospital quality measures.³⁴ Subsequently, in a briefing dated March 24, 2015, MedPAC, a government agency which provides Medicare policy analysis to Congress, proposed an SES adjustment methodology of “dividing hospitals into peer groups based on their overall share of low-income Medicare patients, and then setting a benchmark readmissions target for each peer group”;³⁵ in other words, lower standards for hospitals that serve low-income populations. MedPAC’s proposal will reduce penalties to “safety net” institutions, which is progress but not a solution. Although the HRRP appears to be working, according to the US Department of Health and Human Services, readmissions fell by 150,000 between January 2012 and February 2013,³⁶ we are concerned neither the HRRP nor the MedPac revision proposal considers geographic and environmental components of readmission. The HRRP promotes national improvement in exchange for regional regression.

Fair quality measures are key to value-based reimbursement models; yet, implicit in penalties for excess readmissions is the assumed ability to calculate hospital performance targets. Benchmarks for safety, timely care, and patient satisfaction can be uniform; rates of central line infections should not be influenced by patient mix. However, 9 of the 39 maximum-penalty hospitals under the HRRP are in rural Kentucky; one could hypothesize many reasons why rural Kentucky is a hotbed for excess readmission, including the regional production of tobacco and bourbon.

The fundamental question raised by our study is whether poor performance on quality measures is a function of underperforming hospitals or a manifestation of underserved communities. Moving forward, we encourage data systems and study designs that focus research on geospatial distribution of population health within the context of social and behavioral health determinants.³⁷ Small-area studies of factors that drive health outcomes will inform rational alignment of healthcare policies and resources (including penalties and incentives) with underlying population needs.

 

Strengths and Weaknesses

Matching is a strength of the study. Primary analysis matched case and controls by hospital characteristics, generating cohorts similar across a spectrum of hospital metrics. Therefore, variation in readmission rates was less likely confounded by hospital characteristics. The secondary analysis was matched by geography in an effort to adjust for unmeasured, regional factors, including obesity and cost of living that may confound an association between SES and health outcomes. Geographic matching adds strength to our assertion that SES drives distinction between maximum-penalty hospitals and nonpenalty hospitals.

One weakness was the regional unit of analysis for socioeconomic and Dartmouth Atlas data, which is not a precise profile of the corresponding hospital. Each hospital was assigned a county-level socioeconomic profile. A more robust methodology would analyze patient-level SES data; this was impractical given a cohort of 78 hospitals. Regional health outcomes data limits analysis of readmission penalties as a function of hospital quality. Instead, regional data facilitated associations between readmission and population health, consistent with the aim of our study.

We analyzed 116 of 3668 hospitals eligible for the HRRP (3.2%), limiting the generalizability of our findings. Eighty-four percent of hospitals in the primary analysis have below the median number of beds, and none of them are teaching hospitals. Our analysis, restricted to maximum-penalty and no-penalty cohorts, does not address potential association between submaximal readmission penalties and socioeconomics.

Both matching techniques potentially controlled for similar SES factors and skewed our results towards null, especially in terms of race and ethnicity. Geographic matching generated 7 pairs (18%) within in the same county; both maximum-penalty and no-penalty hospitals were assigned the same socioeconomic profile, as well as 6 pairs (15%) within the same HSA, and both cases and controls had identical Dartmouth Atlas health outcomes profiles. We retained these pairs in our analysis to avoid artificially inflating SES and population health differences between cohorts.

Thirty-nine hospitals received a maximum penalty in the 3rd year of the HRRP. Relative to geographically matched no-penalty hospitals, maximum-penalty hospitals were more likely to be rural and located in counties with less educational attainment, more poverty and more poorly controlled chronic disease. In contrast to nationwide studies, a matched analysis plan revealed no difference between the cohorts in terms of race and ethnicity and provided evidence that maximum penalty hospitals had higher rates of age-, sex-, and race-adjusted hospital-wide mortality.

Our results highlight potential consequences of nationally derived benchmarks for phenomena underpinned by social, behavioral, and environmental factors and raise the question of whether maximum HRRP penalties are a consequence of underperforming hospitals or a manifestation of underserved communities. We are encouraged by MedPAC’s proposal to stratify HRRP by SES, yet recommend further small-area geographic analyses to better align quality measures, penalties, and incentives with resources and needs of distinct populations.

Acknowledgments

The authors thank William Hisey, who laid the foundation for the analysis and without whom the project would not have been possible.

DISCLOSURE

The authors certify that none of the material in this manuscript has been previously published and that none of this material is currently under consideration for publication elsewhere. This project received no funding. None of the authors on this manuscript have any commercial relationships to disclose in relation to this manuscript. All authors have reviewed and approved this manuscript and have contributed significantly to the design, conduct, and/or analysis of the research. No authors have any financial interests to disclose. No authors have any potential conflicts of interest to disclose. No authors have financial or personal relationships with any of the subject material presented in the manuscript.

There is growing recognition that patients and family members have critical insights into healthcare experiences. As consumers of healthcare, patient experience is the definitive gauge of whether healthcare is patient centered. In addition, patients may know things about their healthcare that the care team does not. Several studies have demonstrated that patients have knowledge of adverse events and medical errors that are not detected by other methods.^1-5 For these reasons, systems designed to elicit patient perspectives of care and detect patient-perceived breakdowns in care could be used to improve healthcare safety and quality, including the patient experience.

Historically, hospitals have relied on patient-initiated reporting via complaints or legal action as the main source of information regarding patient-perceived breakdowns in care. However, many patients are hesitant to speak up about problems or uncertain about how to report concerns.^6-8 As a result, healthcare systems often only learn of the most severe breakdowns in care from a subset of activated patients, thus underestimating how widespread patient-perceived breakdowns are.

To overcome these limitations of patient-initiated reporting, hospitals could conduct outreach to patients to actively identify and learn about patient-perceived breakdowns in care. Potential benefits of outreach to patients include more reliable detection of patient-perceived breakdowns in care, identification of a broader range of types of breakdowns commonly experienced by patients, and recognition of problems in real-time when there is more opportunity for redress. Indeed, some hospitals have adopted active outreach programs such as structured nurse manager rounding or postdischarge phone calls.⁹

It is possible that outreach will not overcome patients’ reluctance to speak up, or patients may not share serious or actionable breakdowns. The manner in which outreach is conducted is likely to influence the information patients are willing to share. Prior studies examining patient perspectives of healthcare have primarily taken a structured approach with close-ended questions or a focus on specific aspects of care.^1,10,11 Limited data collected using an open-ended approach suggest patient-perceived breakdowns in care may be very common.^2,12,13 However, the impact of such breakdowns on patients has not been well characterized.

In order to design systems that can optimally detect patient-perceived breakdowns in care, additional information is needed to understand whether patients will report breakdowns in response to outreach programs, what types of problems they will report, and how these problems impact them. Understanding such issues will allow healthcare systems to respond to calls by federal health agencies to develop mechanisms for patients to report concerns about breakdowns in care, thereby providing truly patient-centered care.¹⁴ Therefore, we undertook this study with the overall goal of describing what may be learned from an open-ended outreach approach that directly asks patients about problems they have encountered during hospitalization. Specifically, we aim to (1) describe the types of problems reported by patients in response to this outreach approach and (2) characterize patients’ perceptions of the impact of these events.

METHODS 

Setting

We conducted this study in 2 hospitals between June 2014 and February 2015. One participating hospital is a large, urban, tertiary care medical center serving a predominantly white (78%) patient population in Baltimore, Maryland. The second hospital is a large, inner city, tertiary care medical center serving a predominantly African-American (71%) patient population in Washington, DC.

Three medical-surgical units (MSUs) at each hospital participated. We selected MSUs because MSU patients interact with a variety of clinicians, often have long stays, and are at risk for adverse events. Hospitalists were part of the clinical care team in each of the participating units, serving either as the attending of record or by comanaging patients.

Patient Eligibility

Patients were potentially eligible if they were at least 18 years old, able to speak English or Spanish, and admitted to the hospital for more than 24 hours. Ineligibility criteria included the following: imminent discharge, observation (noninpatient) status, on hospice, on infection precautions (for inpatient interviews only), psychiatric or violence concerns, prisoner status, significant confusion, or inability to provide informed consent.

Eligible patients in each unit were randomized. Interviewers consecutively approached patients according to their random assignment. If a patient was not available, the interviewer proceeded to the next room. Interviewers returned to rooms of missed patients when possible. Recruitment in the unit ended when the recruitment target for that unit was achieved.

 

Interviewers

Five interviewers conducted interviews. One author (KS) provided interviewer training that included didactic instruction, observation, feedback, and modeling. Interviewers participated in weekly debriefing sessions. One interviewer speaks Spanish fluently and was able to conduct interviews in Spanish. Translator services were available for the other interviewers.

Interview Process

Interviews were conducted in person while the patients were hospitalized or via telephone 7 - 30 days postdischarge. A patient who had completed an interview while hospitalized was not eligible for a postdischarge telephone interview. Family members or friends present at the time of the interviews could also participate in addition to or in lieu of the patients with the patients’ assent. Interviewers obtained verbal, informed consent at the start of each interview.

The interview domains and sample questions were developed specifically for the current study and are listed in Table 1. The goal of the interview was to elicit the patient’s (or family member’s) perception of their care experiences and their perceptions of the consequences of any problems with their care. The interviewer sought to obtain sufficient detail to understand the patient’s concerns and to determine what, if any, action might be needed to remediate problems reported by patients. Interviewers captured patient responses by taking detailed notes on a case report form or by directly entering patient responses using a computer or iPad at the time of interview at the discretion of the interviewer.

We defined a patient-perceived breakdown as something that went wrong during the hospitalization according to the patient. If a patient-perceived breakdown in care was identified, the interviewer attempted to resolve the concern. Some breakdowns had occurred in the past, making further resolution impossible (eg, a long wait in the emergency department). Other breakdowns were active and addressable, such as the patient having clinical questions that had not been answered. In such cases, the interviewer attempted to address the patient’s concerns, typically by working with unit nursing staff. For patients interviewed postdischarge, the interviewer worked to resolve ongoing patient concerns with the assistance of the patient safety, quality, and compliance teams as needed. The interviewer provided a brief narrative summary of all interviews to unit nursing leadership within 24 hours. Positive comments were sent to leadership but not captured systematically for research purposes. Further details of the process of responding to patients’ concerns will be reported elsewhere. All data were entered into REDCap to facilitate data management and reporting.¹⁵

The MedStar Health Research Institute Institutional Review Board reviewed and approved this study.

Categorizing Patients’ Responses: The Patient Experience Coding Tool

Using directed content analysis,¹⁶ we deductively created the Patient Experience Coding Tool (PECT) in order to summarize the narrative information captured during the interviews and categorize patient-perceived breakdowns in care. First, we referred to our prior interviews of patients’ views on breakdowns in cancer care⁶ and surrogate decision-makers’ views on breakdowns in intensive care units¹³ to create the initial categories. We then applied the resultant framework to the interviews in the present study and refined the categories. This involved applying the categorization to an initial set of interviews to check the sufficiency of the coding categories. We clarified the scope of each category (ie, what types of events fit under each category) and created additional categories (eg, medication-related problems) to capture patient experiences that were not included in the initial framework.

We then coded each interview using the PECT. A minimum of 2 readers reviewed the narrative notes for each interview. The first reader provided an initial categorization; the second reader reviewed the narrative and confirmed or questioned the initial categorization to improve coding accuracy. If a reader was uncertain about the correct categorization, it was discussed by three readers until an agreement was achieved. Because facilities-related problems (eg, food or parking) fall outside the realm of provider-based hospital care, such comments were not the focus of the outreach efforts and were not consistently recorded. Therefore, they were not included in the PECT and are not reported here.

Analyses

We computed simple, descriptive statistics including the number and percentage of patients identifying at least one breakdown, as well as the number and percent reporting each type of breakdown. We also computed the number and percentage of patients reporting any harm and each type of harm. We computed the percentage of patients reporting at least 1 breakdown by hospital, type of interview (postdischarge vs inpatient), selected patient demographic characteristics (eg, gender, age, education, race), and interviewee (patient vs someone other than the patient interviewed or present during the interview) using the chi-square statistic to test the statistical significance of the resulting differences. All statistical analyses were performed using SPSS version 22.

 

RESULTS

A total of 979 outreach interviews were conducted. Of these, 349 were conducted via telephone postdischarge, and 630 were conducted in person during hospitalization. The average interview duration was 8.5 minutes for telephone interviews and 12.2 minutes for in-person interviews. Of the patients approached to participate, 67% completed an interview (61% in person, 83% via telephone). Patient characteristics are summarized in Table 2.

Overall, 386 of 979 interviewees (39.4%) believed they had experienced at least one breakdown in care. The types of patient-perceived breakdowns reported were categorized using the PECT and are summarized in Table 3 and the Figure. The most common concern involved information exchange. Approximately 1 in 10 patients (n = 105, 10.7%) felt that they had not received the information they needed when they needed it. Medication-related concerns were reported by 12.3% (n = 120) of interviewees and predominantly included concerns about what medications were being administered (5.7%) and inadequately treated pain (5.6%). Many of the patients expressing concerns about what medications were administered questioned why they were not receiving their outpatient medications or did not understand why a different medication was being administered, suggesting that many of these instances were related to breakdowns in communication as well. Other relatively common concerns were delays in the admissions process (reported by 9.2% of interviewees), poor team communication (reported by 6.6% of interviewees), healthcare providers with a rude or uncaring manner (reported by 6.3% of interviewees), and problems related to discharge (reported by 5.7% of interviewees).

Of the 386 interviewees who perceived a breakdown in care, 140 (36.3%) perceived harm associated with the event (Table 3). The most common harms were physical (eg, pain; n = 66) and emotional (eg, distress, worry; n = 60). In addition, patients reported instances of damage to relationships with providers (n = 28) resulting in a loss of trust, with participants citing breakdowns as a reason for not coming back to a particular hospital or provider. In other cases, patients believed that breakdowns in care resulted in the need for additional care or a prolonged hospital stay.

We found no difference between the 2 hospitals where the study was conducted in the percentage of interviewees reporting at least 1 breakdown (39.1% vs 39.9%, P = 0.80). We also found no difference between interview method, (ie, in person vs telephone; 40.6% vs 37.2%, respectively, P = 0.30), patient gender (40.6% and 38.8% for men and women, respectively, P = 0.57), race (41.0% and 36.8% for white and black or African-American, respectively, P = 0.20) or between interviewers (P = 0.37). We did identify differences in rates of reporting at least 1 breakdown in care related to age (45.4% of patients aged 60 years or younger vs 34.5% of patients older than 60 years, P < 0.001) and education (32.7% of patients with a high school education or less vs 46.8% of those with at least some college education, P < 0.001). Patients interviewed alone reported fewer breakdowns than if another person was present during the interview or was interviewed in lieu of the patient (37.8% vs 53.4%, P = 0.002). The rate of reporting breakdowns for patients interviewed alone in the hospital is very similar to the rates of those interviewed via telephone (37.8% vs 37.2%). For most types of breakdowns, there were no differences in reporting for in-person vs postdischarge interviews. Discharge-related problems were more frequently reported among patients interviewed postdischarge (8.9% postdischarge vs 4.0% in person, P = 0.002). Patients interviewed in person were more likely to report problems with information exchange compared to patients interviewed postdischarge (17.6% vs 13.5%, respectively; P = 0.09), although this did not reach statistical significance.

DISCUSSION

Through interviews with nearly 1000 patients, we have found that approximately 4 in 10 hospitalized patients believed they experienced a breakdown in care. Not only are patient-perceived breakdowns in care distressingly common, more than one-third of these events resulted in harm according to the patient. Patients reported a diverse range of breakdowns, including problems related to patient experience, as well as breakdowns in technical aspects of medical care. Collectively, these findings illustrate a striking need to identify and address these frequent and potentially harmful breakdowns.

Our findings are consistent with prior studies in which 20% to 50% of patients identified a problem during hospitalization. For example, Weingart et al^. interviewed patients in a single general medical unit and found that 20% experienced an adverse event, near miss, or medical error, while nearly 40% experienced what was defined as a service quality incident.^2,12 Of note, both our study and the study by Weingart et al. systematically elicited patients’ perspectives of breakdowns in care with explicit questions about problems or breakdowns in care.^2,12 Because patients are often reluctant to speak up about problems in care,without such efforts to actively identify problems, providers and leaders are likely to be unaware of the majority of these concerns.^6-8 These findings suggest that hospital-based providers should at least consider routinely asking patients about breakdowns in care to identify and respond to patients’ concerns.

Not only are patient-perceived breakdowns common, more than one-third of patients who experienced a breakdown considered it to be harmful. This suggests that our outreach approach identified predominantly nontrivial concerns. We adopted a broad definition of harm that includes emotional distress, damage to the relationship with providers, and life disruption. This differs from other studies examining patient reports of breakdowns in care, in which harm was restricted to physical injury.^1,2 We consider this inclusive definition of harm to be a strength of the present study as it provides the most complete picture of the impact of such events on patients. This approach is supported by other studies demonstrating that patients place great emphasis on the psychological consequences of adverse events.^17-19 Thus, it is clear from our work and other studies that nonphysical harm is important and warrants concerted efforts to address.

Patients in our study reported a variety of breakdowns, including breakdowns related to patient experience (eg, communication, relationship with providers) and technical aspects of healthcare delivery (eg, diagnosis, treatment). This is consistent with other studies examining patient perspectives of breakdowns in care. Weingart et al.found that hospitalized patients reported a broad range of problems, including adverse events, medical errors, communication breakdowns, and problems with food.^2,12 This variety of events suggests a need for integration between the various hospital groups that handle patient-perceived breakdowns, including bedside providers, risk management, patient relations, patient advocates, and quality and safety groups, in order to provide a coordinated and effective response to the full spectrum of patient-perceived breakdowns in care.

Patients in our study were more likely to report breakdowns related to communication and relationships with providers than those related to technical aspects of care. Similarly, Kuzel et al.found that the most common problems cited by patients in the primary care setting were breakdowns in the clinician-patient relationship and access-related problems.¹⁷This is not surprising, as patients are likely to have more direct knowledge about communication and interpersonal relationships than about technical aspects of care.

We identified several factors associated with the likelihood of reporting a breakdown in care. Most strikingly, involving a friend or family member in the interview was strongly associated with reporting a breakdown. Other work has also suggested that patients’ friends and family members are an important source of information about safety concerns.^20,21 In addition, several patient characteristics were associated with an increased likelihood of reporting a breakdown, including being younger and better educated. These findings highlight the importance of engaging patients’ friends and families in efforts to elicit patient concerns about healthcare, and they confirm recommendations for patients to bring a friend or family member to healthcare encounters.²² In addition, they illustrate the need to better understand how patient characteristics affect perceptions of breakdowns in care and their willingness to speak up, as this could inform efforts to target outreach to especially vulnerable patients.

A strength of this study is the number of interviews completed (almost 1000), which provides a diverse range of patient views and experiences, as evidenced by the demographic characteristics of participants. Interviews were conducted at two hospitals that differ substantially with regard to populations served, further enhancing the generalizability of our findings. Despite the large number of interviews and diverse patient characteristics, patients were drawn from only 3 units at 2 hospitals, which may limit generalizability.

We did not conduct medical record reviews to validate patients’ reports of problems, which may be viewed as a limitation. While such a comparison would be informative, the intent of the current study was to elicit patients’ perceptions of care, including aspects of care that are not typically captured in the medical record, such as communication. Other studies have demonstrated that patients’ reports of medical errors and adverse events tend to differ from providers’ reports of the same subjects.^19,23 Therefore, we considered the patients’ perceptions of care to be a useful endpoint in and of itself. We did not determine the extent to which providers were already aware of patients’ concerns or whether they considered patients’ concerns valid. A related limitation is our inability to determine whether the differences we identified in the rates of breakdown reporting based on patient characteristics reflect differences in willingness to report or differences in experiences. Because we included patients in an MSU, it is possible that breakdowns were related to medical care, surgical care, or both. We did not follow patients longitudinally and therefore only captured harm perceived by a patient at the time of the interview. It is possible that patients may have experienced harm later in their hospitalization or following discharge that was not measured. Lastly, we did not measure interrater reliability of the interview coding and therefore do not present the PECT as a validated instrument. These important questions should be targeted for future study.

 

CONCLUSION

When directly asked about their experiences, almost 4 out of 10 hospitalized patients reported a breakdown in their care, many of which were perceived to be harmful. Not all hospitals will have the resources to implement the intensive approach used in this study to elicit patient-perceived breakdowns. Therefore, further work is needed to develop sustainable methods to overcome patients’ reluctance to report breakdowns in care. Engaging patients’ families and friends may be a particularly fruitful strategy. We offer the PECT as a tool that hospitals could use to summarize a variety of sources of patient feedback such as complaints, responses to surveys, and consumer reviews. Hospitals that effectively encourage patients and their family members to speak up about perceived breakdowns will identify many opportunities to address patient concerns, potentially leading to improved patient safety and experience.