Keiko A. Fukuda, BA

Delirium is characterized by fluctuating disturbances in cognition and consciousness and is a common complication of hospitalization in medical and surgical patients. Studies estimate the prevalence of delirium in hospitalized patients[1] to be 14% to 56%, and up to 70% in critically ill elderly patients.[2] Estimates of total healthcare costs associated with delirium range from $38 to $152 billion per year in the United States.[3] Delirious patients are more likely to be discharged to a nursing home and have increased hospital mortality and longer lengths of stay.[4, 5, 6] Recent data suggest long‐term effects of delirium including cognitive impairments up to 1 year following the illness[7] and an increased likelihood of developing[8] or worsening dementia.[9]

It is estimated that one‐third of hospital‐acquired delirium cases could be prevented with appropriate interventions.[10] A prediction rule that easily and accurately identifies high‐risk patients upon admission could therefore have a substantial clinical impact. In addition, a prediction rule could be used to identify patients in whom new targeted interventions for delirium prevention could be investigated. A number of risk factors for delirium have been identified, including older age, preexisting cognitive dysfunction, vision and hearing impairment, severe illness, dehydration, electrolyte abnormalities, overmedication, and alcohol abuse.[11, 12, 13, 14, 15, 16] Existing prediction rules using various combinations of these measures have been limited by their complexity,[17] do not predict incident delirium,[18, 19] or are restricted to surgical[20, 21, 22] or intensive care[23] patients and therefore are not broadly applicable to the general medical population, which is particularly susceptible to developing delirium.

We conducted this study to develop a simple, efficient, and accurate prediction rule for hospital‐acquired delirium in adult medical inpatients assessed at the time of admission. Our a priori hypothesis was that a delirium prediction rule would consist of a combination of known risk factors and most likely incorporate old age, illness severity, and preexisting cognitive dysfunction.

METHODS

RESULTS

The derivation cohort consisted of elderly patients (mean age, 68.0811.96 years; interquartile range, 5096 years), and included more males than females (54.1% vs 45.9%). Subjects were predominantly white (73.7%) and lived at home (90%) (Table 1). The mean admission MMSE score was 27.0 (standard deviation [SD], 3.4; range, 730). Median follow‐up was 2 days (interquartile range, 13). Delirium developed in 12% (n=25) of the cohort.

Univariate analysis of the derivation study identified 10 variables significantly associated (P<0.05) with delirium (Table 2). Predictors of delirium included abnormal scores on 4 subtests of the MMSE, low score on the Mini‐Cog, living in an assisted living or skilled nursing facility, moderate to severe illness, old age, a past history of dementia, and hearing loss as assessed by the whisper test. These predictors were then entered into a stepwise logistic regression analysis that identified 4 independent predictors of delirium (Table 3).

These 4 independent predictors were assigned 1 point each if present to create a prediction rule with a range of possible scores from 0 to 4. There was a significant trend predicting higher rates of delirium with higher scores, with no subjects who scored 0 becoming delirious, compared to 40% of those subjects scoring 3 or 4 (P for trend<0.001) (Table 4).

The validation cohort consisted of adults with a mean age of 70.7210.6 years, (interquartile range, 5194 years), who were predominantly white (75.8%) and overwhelmingly male (95.2%) (Table 1). The mean admission MMSE score was 26.75 (SD, 2.8; range, 1730). Median follow‐up was 2 days (interquartile range, 15). Delirium developed in 8.5% (n=14) of the cohort. In the validation cohort, 4% of subjects with a score of 0 became delirious, whereas 19% of those scoring 3 or 4 became delirious (P for trend 0.025) (Table 4).

ROC curves were compared for the derivation and validation cohorts. The area under the ROC curve for the derivation cohort (0.81, 95% confidence interval [CI]: 0.720.90) was slightly better than that in the validation cohort (0.69, 95% CI: 0.540.83), but the difference did not reach statistical significance (P=0.14) (Figure 1).

Figure 1

DISCUSSION

We derived and validated a prediction rule to assess the risk of developing delirium in hospitalized adult medical patients. Four variables easily assessed on admission in a screen lasting less than 2 minutes were independently associated with the development of delirium. The prediction rule can be remembered with the following mnemonic: AWOL (Age80 years; unable to spell World backward; not fully Oriented to place; and moderate or severe iLlness severity).

It is estimated up to a third of hospital acquired delirium cases can be prevented.[10] Recent guidelines recommend the use of a multicomponent intervention to prevent delirium and provide evidence that such a strategy would be cost‐effective.[39] Nevertheless, such interventions are resource intense, requiring specialized nurse training and staffing[40] and have not been widely implemented. Acute care for the elderly units, where interventions to prevent delirium might logically be implemented, also require physical remodeling to provide carpeted hallways, handrails, and elevated toilet seats and door levers.[41] A method of risk stratification to identify the patients who would benefit most from resource‐intensive prevention strategies would be valuable.

The AWOL tool may provide a practical alternative to existing delirium prediction rules for adult medical inpatients. Because it can be completed by a nurse in <2 minutes, the AWOL tool may be easier to apply and disseminate than a previously described score relying on the MMSE, Acute Physiology and Chronic Health Evaluation scores, and measured visual acuity.[17] Two other tools, 1 based on chart abstraction[18] and the other based on clinical variables measured at admission,[19] are similarly easy to apply but only predict prevalent and not incident delirium, making them less clinically useful.

This study's strengths include its prospective cohort design and the derivation and validation being performed in different hospitals. The derivation cohort consisted of patients admitted to a tertiary care academic medical center or an affiliated hospital where routine mixed gender general medical patients are treated, whereas validation was performed at the SFVAMC, where patients are predominantly older men with a high incidence of vascular risk factors. The outcome was assessed on a daily basis, and the likelihood any cases were missed was low. Although there is some potential for bias because the outcome was assessed by a research assistant not blinded to baseline characteristics, this was mitigated by having each outcome validated by a blinded neurologist and in the validation cohort having the research assistant blinded to the AWOL score. Other strengths are the broad inclusion criteria, with both middle‐aged and elderly patients having a wide range of medical and neurological conditions, allowing for wide application of the results. Although many studies of delirium focus on patients over age 70 years, we chose to include patients aged 50 years or older because hospital‐acquired delirium still occurs in this age group (17 of 195 [8%] patients aged 5069 years became delirious in this study), and risk factors such as severe illness and cognitive dysfunction are likely to be predictors of delirium even at younger ages. Additionally, the inclusion of nurses' clinical judgment to assess illness severity using a straightforward rating scale allows bedside nurses to readily administer the prediction rule in practice.[34]

This study has several potential limitations. The number of outcomes in the derivation cohort was small compared to the number of predictors chosen for the prediction rule. This could potentially have led to overfitting the model in the derivation cohort and thus an overly optimistic estimation of the model's performance. In the validation cohort, the area under the ROC curve was lower than in the derivation cohort, and although the difference did not reach statistical significance, this may have been due to the small sample size. In addition, none of the 4 subjects with an AWOL score of 4 became delirious, potentially reflecting poor calibration of the prediction rule. However, the trend of higher rates of delirium among subjects with higher AWOL scores still reached statistical significance, and the prediction rule demonstrated good discrimination between patients at high and low risk for developing delirium.

To test whether a better prediction tool could be derived from our data, we combined the derivation and validation cohorts and repeated a stepwise multivariable logistic regression with the same variables used for derivation of the AWOL tool (with the exception of the whisper test of hearing and a past medical history of dementia, because these data were not collected in the validation cohort). This model produced the same 4 independent predictors of delirium used in the AWOL tool. We then used bootstrapping to internally validate the prediction rule, suggesting that the predictors in the AWOL tool were the best fit for the available data. However, given the small number of outcomes in our study, the AWOL tool may benefit from further validation in a larger independent cohort to more precisely calibrate the number of expected outcomes with each score.

Although the majority of medical inpatients were eligible for enrollment in our study, some populations were excluded, and our results may not generalize to these populations. Non‐English speaking patients were excluded to preserve the validity of our study instruments. In addition, patients with profound aphasia or an admission diagnosis of alcohol withdrawal were excluded. Patients discharged on the first day of their hospitalization were excluded either because they were discharged prior to screening or prior to their first follow‐up visit. Therefore, our results may only be valid in patients who remained in the hospital for over 24 hours. In addition, because we only included medical patients, our results cannot necessarily be generalized to the surgical population.

Finally, parts of the prediction rule (orientation and spelling world backward) are also components of the CAM and were used in the assessment of the outcome, and this may introduce a potential tautology: if patients are disoriented or have poor attention because they cannot spell world backward at admission, they already have fulfilled part of the criteria for delirium. However, a diagnosis of delirium using the CAM involves a comprehensive patient and caregiver interview, and in addition to poor attention, requires the presence of an acute change in mental status and disorganized thinking or altered level of consciousness. Therefore, it is possible, and common, for patients to be disoriented to place and/or unable to spell world backward, yet not be delirious, and predicting a subsequent change in cognition during the hospitalization is still clinically important. It is possible the AWOL tool works by identifying patients with impaired attention and subclinical delirium, but one could argue this makes a strong case for its validity because these patients especially should be triaged to an inpatient unit that specializes in delirium prevention. It is also possible the cognitive tasks that are part of the AWOL tool detect preexisting cognitive impairment, which is in turn a major risk factor for delirium.

Recognizing and classifying the risk of delirium during hospitalization is imperative, considering the illness' significant contribution to healthcare costs, morbidity, and mortality. The cost‐effectiveness of proven interventions to detect and prevent delirium could be magnified with focused implementation in those patients at highest risk.[39, 40, 41] Further research is required to determine whether the combination of delirium prediction rules such as those developed here and prevention strategies will result in decreased rates of delirium and economic savings for the healthcare system.

Disclosures

Drs. Douglas, Hessler, Dhaliwal, Betjemann, Lucatorto, Johnston, Josephson, and Ms. Fukuda and Ms. Alameddine have no conflicts of interest or financial disclosures. This research was made possible by the Ruth E. Raskin Fund and a UCSF Dean's Research Scholarship. These funding agencies had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.

Delirium is characterized by fluctuating disturbances in cognition and consciousness and is a common complication of hospitalization in medical and surgical patients. Studies estimate the prevalence of delirium in hospitalized patients[1] to be 14% to 56%, and up to 70% in critically ill elderly patients.[2] Estimates of total healthcare costs associated with delirium range from $38 to $152 billion per year in the United States.[3] Delirious patients are more likely to be discharged to a nursing home and have increased hospital mortality and longer lengths of stay.[4, 5, 6] Recent data suggest long‐term effects of delirium including cognitive impairments up to 1 year following the illness[7] and an increased likelihood of developing[8] or worsening dementia.[9]

It is estimated that one‐third of hospital‐acquired delirium cases could be prevented with appropriate interventions.[10] A prediction rule that easily and accurately identifies high‐risk patients upon admission could therefore have a substantial clinical impact. In addition, a prediction rule could be used to identify patients in whom new targeted interventions for delirium prevention could be investigated. A number of risk factors for delirium have been identified, including older age, preexisting cognitive dysfunction, vision and hearing impairment, severe illness, dehydration, electrolyte abnormalities, overmedication, and alcohol abuse.[11, 12, 13, 14, 15, 16] Existing prediction rules using various combinations of these measures have been limited by their complexity,[17] do not predict incident delirium,[18, 19] or are restricted to surgical[20, 21, 22] or intensive care[23] patients and therefore are not broadly applicable to the general medical population, which is particularly susceptible to developing delirium.

We conducted this study to develop a simple, efficient, and accurate prediction rule for hospital‐acquired delirium in adult medical inpatients assessed at the time of admission. Our a priori hypothesis was that a delirium prediction rule would consist of a combination of known risk factors and most likely incorporate old age, illness severity, and preexisting cognitive dysfunction.

METHODS

RESULTS

The derivation cohort consisted of elderly patients (mean age, 68.0811.96 years; interquartile range, 5096 years), and included more males than females (54.1% vs 45.9%). Subjects were predominantly white (73.7%) and lived at home (90%) (Table 1). The mean admission MMSE score was 27.0 (standard deviation [SD], 3.4; range, 730). Median follow‐up was 2 days (interquartile range, 13). Delirium developed in 12% (n=25) of the cohort.

Univariate analysis of the derivation study identified 10 variables significantly associated (P<0.05) with delirium (Table 2). Predictors of delirium included abnormal scores on 4 subtests of the MMSE, low score on the Mini‐Cog, living in an assisted living or skilled nursing facility, moderate to severe illness, old age, a past history of dementia, and hearing loss as assessed by the whisper test. These predictors were then entered into a stepwise logistic regression analysis that identified 4 independent predictors of delirium (Table 3).

These 4 independent predictors were assigned 1 point each if present to create a prediction rule with a range of possible scores from 0 to 4. There was a significant trend predicting higher rates of delirium with higher scores, with no subjects who scored 0 becoming delirious, compared to 40% of those subjects scoring 3 or 4 (P for trend<0.001) (Table 4).

The validation cohort consisted of adults with a mean age of 70.7210.6 years, (interquartile range, 5194 years), who were predominantly white (75.8%) and overwhelmingly male (95.2%) (Table 1). The mean admission MMSE score was 26.75 (SD, 2.8; range, 1730). Median follow‐up was 2 days (interquartile range, 15). Delirium developed in 8.5% (n=14) of the cohort. In the validation cohort, 4% of subjects with a score of 0 became delirious, whereas 19% of those scoring 3 or 4 became delirious (P for trend 0.025) (Table 4).

ROC curves were compared for the derivation and validation cohorts. The area under the ROC curve for the derivation cohort (0.81, 95% confidence interval [CI]: 0.720.90) was slightly better than that in the validation cohort (0.69, 95% CI: 0.540.83), but the difference did not reach statistical significance (P=0.14) (Figure 1).

Figure 1

DISCUSSION

We derived and validated a prediction rule to assess the risk of developing delirium in hospitalized adult medical patients. Four variables easily assessed on admission in a screen lasting less than 2 minutes were independently associated with the development of delirium. The prediction rule can be remembered with the following mnemonic: AWOL (Age80 years; unable to spell World backward; not fully Oriented to place; and moderate or severe iLlness severity).

It is estimated up to a third of hospital acquired delirium cases can be prevented.[10] Recent guidelines recommend the use of a multicomponent intervention to prevent delirium and provide evidence that such a strategy would be cost‐effective.[39] Nevertheless, such interventions are resource intense, requiring specialized nurse training and staffing[40] and have not been widely implemented. Acute care for the elderly units, where interventions to prevent delirium might logically be implemented, also require physical remodeling to provide carpeted hallways, handrails, and elevated toilet seats and door levers.[41] A method of risk stratification to identify the patients who would benefit most from resource‐intensive prevention strategies would be valuable.

The AWOL tool may provide a practical alternative to existing delirium prediction rules for adult medical inpatients. Because it can be completed by a nurse in <2 minutes, the AWOL tool may be easier to apply and disseminate than a previously described score relying on the MMSE, Acute Physiology and Chronic Health Evaluation scores, and measured visual acuity.[17] Two other tools, 1 based on chart abstraction[18] and the other based on clinical variables measured at admission,[19] are similarly easy to apply but only predict prevalent and not incident delirium, making them less clinically useful.

This study's strengths include its prospective cohort design and the derivation and validation being performed in different hospitals. The derivation cohort consisted of patients admitted to a tertiary care academic medical center or an affiliated hospital where routine mixed gender general medical patients are treated, whereas validation was performed at the SFVAMC, where patients are predominantly older men with a high incidence of vascular risk factors. The outcome was assessed on a daily basis, and the likelihood any cases were missed was low. Although there is some potential for bias because the outcome was assessed by a research assistant not blinded to baseline characteristics, this was mitigated by having each outcome validated by a blinded neurologist and in the validation cohort having the research assistant blinded to the AWOL score. Other strengths are the broad inclusion criteria, with both middle‐aged and elderly patients having a wide range of medical and neurological conditions, allowing for wide application of the results. Although many studies of delirium focus on patients over age 70 years, we chose to include patients aged 50 years or older because hospital‐acquired delirium still occurs in this age group (17 of 195 [8%] patients aged 5069 years became delirious in this study), and risk factors such as severe illness and cognitive dysfunction are likely to be predictors of delirium even at younger ages. Additionally, the inclusion of nurses' clinical judgment to assess illness severity using a straightforward rating scale allows bedside nurses to readily administer the prediction rule in practice.[34]

This study has several potential limitations. The number of outcomes in the derivation cohort was small compared to the number of predictors chosen for the prediction rule. This could potentially have led to overfitting the model in the derivation cohort and thus an overly optimistic estimation of the model's performance. In the validation cohort, the area under the ROC curve was lower than in the derivation cohort, and although the difference did not reach statistical significance, this may have been due to the small sample size. In addition, none of the 4 subjects with an AWOL score of 4 became delirious, potentially reflecting poor calibration of the prediction rule. However, the trend of higher rates of delirium among subjects with higher AWOL scores still reached statistical significance, and the prediction rule demonstrated good discrimination between patients at high and low risk for developing delirium.

To test whether a better prediction tool could be derived from our data, we combined the derivation and validation cohorts and repeated a stepwise multivariable logistic regression with the same variables used for derivation of the AWOL tool (with the exception of the whisper test of hearing and a past medical history of dementia, because these data were not collected in the validation cohort). This model produced the same 4 independent predictors of delirium used in the AWOL tool. We then used bootstrapping to internally validate the prediction rule, suggesting that the predictors in the AWOL tool were the best fit for the available data. However, given the small number of outcomes in our study, the AWOL tool may benefit from further validation in a larger independent cohort to more precisely calibrate the number of expected outcomes with each score.

Although the majority of medical inpatients were eligible for enrollment in our study, some populations were excluded, and our results may not generalize to these populations. Non‐English speaking patients were excluded to preserve the validity of our study instruments. In addition, patients with profound aphasia or an admission diagnosis of alcohol withdrawal were excluded. Patients discharged on the first day of their hospitalization were excluded either because they were discharged prior to screening or prior to their first follow‐up visit. Therefore, our results may only be valid in patients who remained in the hospital for over 24 hours. In addition, because we only included medical patients, our results cannot necessarily be generalized to the surgical population.

Finally, parts of the prediction rule (orientation and spelling world backward) are also components of the CAM and were used in the assessment of the outcome, and this may introduce a potential tautology: if patients are disoriented or have poor attention because they cannot spell world backward at admission, they already have fulfilled part of the criteria for delirium. However, a diagnosis of delirium using the CAM involves a comprehensive patient and caregiver interview, and in addition to poor attention, requires the presence of an acute change in mental status and disorganized thinking or altered level of consciousness. Therefore, it is possible, and common, for patients to be disoriented to place and/or unable to spell world backward, yet not be delirious, and predicting a subsequent change in cognition during the hospitalization is still clinically important. It is possible the AWOL tool works by identifying patients with impaired attention and subclinical delirium, but one could argue this makes a strong case for its validity because these patients especially should be triaged to an inpatient unit that specializes in delirium prevention. It is also possible the cognitive tasks that are part of the AWOL tool detect preexisting cognitive impairment, which is in turn a major risk factor for delirium.

Recognizing and classifying the risk of delirium during hospitalization is imperative, considering the illness' significant contribution to healthcare costs, morbidity, and mortality. The cost‐effectiveness of proven interventions to detect and prevent delirium could be magnified with focused implementation in those patients at highest risk.[39, 40, 41] Further research is required to determine whether the combination of delirium prediction rules such as those developed here and prevention strategies will result in decreased rates of delirium and economic savings for the healthcare system.

Disclosures

Drs. Douglas, Hessler, Dhaliwal, Betjemann, Lucatorto, Johnston, Josephson, and Ms. Fukuda and Ms. Alameddine have no conflicts of interest or financial disclosures. This research was made possible by the Ruth E. Raskin Fund and a UCSF Dean's Research Scholarship. These funding agencies had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.

Delirium is characterized by fluctuating disturbances in cognition and consciousness and is a common complication of hospitalization in medical and surgical patients. Studies estimate the prevalence of delirium in hospitalized patients[1] to be 14% to 56%, and up to 70% in critically ill elderly patients.[2] Estimates of total healthcare costs associated with delirium range from $38 to $152 billion per year in the United States.[3] Delirious patients are more likely to be discharged to a nursing home and have increased hospital mortality and longer lengths of stay.[4, 5, 6] Recent data suggest long‐term effects of delirium including cognitive impairments up to 1 year following the illness[7] and an increased likelihood of developing[8] or worsening dementia.[9]

It is estimated that one‐third of hospital‐acquired delirium cases could be prevented with appropriate interventions.[10] A prediction rule that easily and accurately identifies high‐risk patients upon admission could therefore have a substantial clinical impact. In addition, a prediction rule could be used to identify patients in whom new targeted interventions for delirium prevention could be investigated. A number of risk factors for delirium have been identified, including older age, preexisting cognitive dysfunction, vision and hearing impairment, severe illness, dehydration, electrolyte abnormalities, overmedication, and alcohol abuse.[11, 12, 13, 14, 15, 16] Existing prediction rules using various combinations of these measures have been limited by their complexity,[17] do not predict incident delirium,[18, 19] or are restricted to surgical[20, 21, 22] or intensive care[23] patients and therefore are not broadly applicable to the general medical population, which is particularly susceptible to developing delirium.

We conducted this study to develop a simple, efficient, and accurate prediction rule for hospital‐acquired delirium in adult medical inpatients assessed at the time of admission. Our a priori hypothesis was that a delirium prediction rule would consist of a combination of known risk factors and most likely incorporate old age, illness severity, and preexisting cognitive dysfunction.

METHODS

RESULTS

The derivation cohort consisted of elderly patients (mean age, 68.0811.96 years; interquartile range, 5096 years), and included more males than females (54.1% vs 45.9%). Subjects were predominantly white (73.7%) and lived at home (90%) (Table 1). The mean admission MMSE score was 27.0 (standard deviation [SD], 3.4; range, 730). Median follow‐up was 2 days (interquartile range, 13). Delirium developed in 12% (n=25) of the cohort.

Univariate analysis of the derivation study identified 10 variables significantly associated (P<0.05) with delirium (Table 2). Predictors of delirium included abnormal scores on 4 subtests of the MMSE, low score on the Mini‐Cog, living in an assisted living or skilled nursing facility, moderate to severe illness, old age, a past history of dementia, and hearing loss as assessed by the whisper test. These predictors were then entered into a stepwise logistic regression analysis that identified 4 independent predictors of delirium (Table 3).

These 4 independent predictors were assigned 1 point each if present to create a prediction rule with a range of possible scores from 0 to 4. There was a significant trend predicting higher rates of delirium with higher scores, with no subjects who scored 0 becoming delirious, compared to 40% of those subjects scoring 3 or 4 (P for trend<0.001) (Table 4).

The validation cohort consisted of adults with a mean age of 70.7210.6 years, (interquartile range, 5194 years), who were predominantly white (75.8%) and overwhelmingly male (95.2%) (Table 1). The mean admission MMSE score was 26.75 (SD, 2.8; range, 1730). Median follow‐up was 2 days (interquartile range, 15). Delirium developed in 8.5% (n=14) of the cohort. In the validation cohort, 4% of subjects with a score of 0 became delirious, whereas 19% of those scoring 3 or 4 became delirious (P for trend 0.025) (Table 4).

ROC curves were compared for the derivation and validation cohorts. The area under the ROC curve for the derivation cohort (0.81, 95% confidence interval [CI]: 0.720.90) was slightly better than that in the validation cohort (0.69, 95% CI: 0.540.83), but the difference did not reach statistical significance (P=0.14) (Figure 1).

Figure 1

DISCUSSION

We derived and validated a prediction rule to assess the risk of developing delirium in hospitalized adult medical patients. Four variables easily assessed on admission in a screen lasting less than 2 minutes were independently associated with the development of delirium. The prediction rule can be remembered with the following mnemonic: AWOL (Age80 years; unable to spell World backward; not fully Oriented to place; and moderate or severe iLlness severity).

It is estimated up to a third of hospital acquired delirium cases can be prevented.[10] Recent guidelines recommend the use of a multicomponent intervention to prevent delirium and provide evidence that such a strategy would be cost‐effective.[39] Nevertheless, such interventions are resource intense, requiring specialized nurse training and staffing[40] and have not been widely implemented. Acute care for the elderly units, where interventions to prevent delirium might logically be implemented, also require physical remodeling to provide carpeted hallways, handrails, and elevated toilet seats and door levers.[41] A method of risk stratification to identify the patients who would benefit most from resource‐intensive prevention strategies would be valuable.

The AWOL tool may provide a practical alternative to existing delirium prediction rules for adult medical inpatients. Because it can be completed by a nurse in <2 minutes, the AWOL tool may be easier to apply and disseminate than a previously described score relying on the MMSE, Acute Physiology and Chronic Health Evaluation scores, and measured visual acuity.[17] Two other tools, 1 based on chart abstraction[18] and the other based on clinical variables measured at admission,[19] are similarly easy to apply but only predict prevalent and not incident delirium, making them less clinically useful.

This study's strengths include its prospective cohort design and the derivation and validation being performed in different hospitals. The derivation cohort consisted of patients admitted to a tertiary care academic medical center or an affiliated hospital where routine mixed gender general medical patients are treated, whereas validation was performed at the SFVAMC, where patients are predominantly older men with a high incidence of vascular risk factors. The outcome was assessed on a daily basis, and the likelihood any cases were missed was low. Although there is some potential for bias because the outcome was assessed by a research assistant not blinded to baseline characteristics, this was mitigated by having each outcome validated by a blinded neurologist and in the validation cohort having the research assistant blinded to the AWOL score. Other strengths are the broad inclusion criteria, with both middle‐aged and elderly patients having a wide range of medical and neurological conditions, allowing for wide application of the results. Although many studies of delirium focus on patients over age 70 years, we chose to include patients aged 50 years or older because hospital‐acquired delirium still occurs in this age group (17 of 195 [8%] patients aged 5069 years became delirious in this study), and risk factors such as severe illness and cognitive dysfunction are likely to be predictors of delirium even at younger ages. Additionally, the inclusion of nurses' clinical judgment to assess illness severity using a straightforward rating scale allows bedside nurses to readily administer the prediction rule in practice.[34]

This study has several potential limitations. The number of outcomes in the derivation cohort was small compared to the number of predictors chosen for the prediction rule. This could potentially have led to overfitting the model in the derivation cohort and thus an overly optimistic estimation of the model's performance. In the validation cohort, the area under the ROC curve was lower than in the derivation cohort, and although the difference did not reach statistical significance, this may have been due to the small sample size. In addition, none of the 4 subjects with an AWOL score of 4 became delirious, potentially reflecting poor calibration of the prediction rule. However, the trend of higher rates of delirium among subjects with higher AWOL scores still reached statistical significance, and the prediction rule demonstrated good discrimination between patients at high and low risk for developing delirium.

To test whether a better prediction tool could be derived from our data, we combined the derivation and validation cohorts and repeated a stepwise multivariable logistic regression with the same variables used for derivation of the AWOL tool (with the exception of the whisper test of hearing and a past medical history of dementia, because these data were not collected in the validation cohort). This model produced the same 4 independent predictors of delirium used in the AWOL tool. We then used bootstrapping to internally validate the prediction rule, suggesting that the predictors in the AWOL tool were the best fit for the available data. However, given the small number of outcomes in our study, the AWOL tool may benefit from further validation in a larger independent cohort to more precisely calibrate the number of expected outcomes with each score.

Although the majority of medical inpatients were eligible for enrollment in our study, some populations were excluded, and our results may not generalize to these populations. Non‐English speaking patients were excluded to preserve the validity of our study instruments. In addition, patients with profound aphasia or an admission diagnosis of alcohol withdrawal were excluded. Patients discharged on the first day of their hospitalization were excluded either because they were discharged prior to screening or prior to their first follow‐up visit. Therefore, our results may only be valid in patients who remained in the hospital for over 24 hours. In addition, because we only included medical patients, our results cannot necessarily be generalized to the surgical population.

Finally, parts of the prediction rule (orientation and spelling world backward) are also components of the CAM and were used in the assessment of the outcome, and this may introduce a potential tautology: if patients are disoriented or have poor attention because they cannot spell world backward at admission, they already have fulfilled part of the criteria for delirium. However, a diagnosis of delirium using the CAM involves a comprehensive patient and caregiver interview, and in addition to poor attention, requires the presence of an acute change in mental status and disorganized thinking or altered level of consciousness. Therefore, it is possible, and common, for patients to be disoriented to place and/or unable to spell world backward, yet not be delirious, and predicting a subsequent change in cognition during the hospitalization is still clinically important. It is possible the AWOL tool works by identifying patients with impaired attention and subclinical delirium, but one could argue this makes a strong case for its validity because these patients especially should be triaged to an inpatient unit that specializes in delirium prevention. It is also possible the cognitive tasks that are part of the AWOL tool detect preexisting cognitive impairment, which is in turn a major risk factor for delirium.

Recognizing and classifying the risk of delirium during hospitalization is imperative, considering the illness' significant contribution to healthcare costs, morbidity, and mortality. The cost‐effectiveness of proven interventions to detect and prevent delirium could be magnified with focused implementation in those patients at highest risk.[39, 40, 41] Further research is required to determine whether the combination of delirium prediction rules such as those developed here and prevention strategies will result in decreased rates of delirium and economic savings for the healthcare system.

Disclosures

Drs. Douglas, Hessler, Dhaliwal, Betjemann, Lucatorto, Johnston, Josephson, and Ms. Fukuda and Ms. Alameddine have no conflicts of interest or financial disclosures. This research was made possible by the Ruth E. Raskin Fund and a UCSF Dean's Research Scholarship. These funding agencies had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.