Ilene Timko Burns, MD, MPH

Although immunization rates have increased to record levels, they remain suboptimal. State laws requiring immunizations for school entry help ensure that school-aged children are adequately immunized, but younger children are at risk of inadequate immunization. In 1995, only 74% of children in the United States between the ages of 19 and 35 months had received all immunizations recommended for children 18 months of age.¹ A number of studies have focused on the causes of low immunization rates, which include poverty, inadequate clinician knowledge of contraindications, and missed opportunities to vaccinate.^2-8 In contrast, higher rates have been found when proactive office procedures-such as reminder systems,^9-12 audit and feedback systems,¹³ and provider prompts,^11,14-16-are implemented. The Task Force on Community Preventive Services found that provider reminder/recall systems improve vaccination coverage in adults, adolescents, and children and strongly recommends their use.¹⁷

The aim of this study was to determine whether a chart prompt, ie, a reminder to the patients’ physicians, is an effective method to increase pediatric immunization rates by decreasing missed opportunities. The physicians’ offices in this study had neither a reminder system in place for immunizations nor an existing database that would permit computer generation of such a reminder system for existing pediatric patients. Manual chart review offered the only method to determine immunization rates and to generate prompts.

Methods

Study population

The setting for this study was the St. Margaret Memorial Hospital Family Practice Residency Program. During the study period, 36 residents and 4 fellows saw patients in the 2 Family Health Centers (FHC) of the program. The Lawrenceville Family Health Center (LFHC) is located in the financially disadvantaged Lawrenceville neighborhood of urban Pittsburgh, serving a predominantly white population. In 1994, staff at the LFHC provided care for approximately 715 children younger than 6 years. The Bloomfield-Garfield Family Health Center (BGFHC) serves the urban Pittsburgh neighborhoods of Bloomfield and Garfield. The population served by this FHC is predominantly African American with significant minorities of Asian American and white patients. In 1994, staff at the BGFHC provided care for approximately 315 children younger than 6 years. Previous assessments of immunization rates showed that the LFHC and the BGFHC had significantly different pediatric immunization rates (47% of children fully immunized by their second birthday at LFHC compared with 33% at BGFHC; chart audit performed by I.T.B. in 1993, unpublished data). Charts of children born on or after May 1, 1989, were included in the study. The Institutional Review Board of the University of Pittsburgh approved the study.

Study protocol

The intervention phase of the project was initiated on July 1, 1995, and completed on June 30, 1996. Each patient chart was systematically assigned to the control or intervention group based on the chart number. A random-number table was used to generate the following scheme: children with a chart number ending in 0, 1, 2, 5, or 6 were assigned to the intervention group, and children with a chart number ending in 3, 4, 7, 8, or 9 were assigned to the control group. Each morning, a designated nurse at each FHC generated a list of patients scheduled to be seen that day. The names of patients eligible for the study were highlighted. For each child in the intervention group, the nurse reviewed the immunization record present on the child’s chart. Each participating nurse received instruction in chart review and immunization guidelines from the principal investigator. Based on this chart review, the nurse determined if the child was eligible for any immunizations on that day’s office visit. The 1995 Recommended Childhood Immunization Schedule of the Centers for Disease Control and Prevention¹⁸Table 1 was used as the primary guideline. The 1994 RedBook¹⁹ was used as the source for answers to questions about the eligibility for immunizations when the 1995 schedule was ambiguous or inappropriate (eg, for a child with delayed immunization or special medical circumstances).

The nurse completed 1-page forms and attached them to charts of children who might be eligible for immunization on that day’s visit. No form was completed for children who were up to date on immunizations. No chart review was performed that day on children in the control group, and no form was attached to the control group’s charts. The form acted as a prompt for the physician to ask the parent or guardian if the child had received the immunizations in question. If the physician determined there was a valid reason not to administer the vaccines on that visit, the physician noted the reason on the form and checked off the “Do not immunize” option on the form. If there were no contraindications to immunization, and if the parent or guardian gave informed consent for immunization, the physician checked off the “Immunize” option.

Evaluation of intervention

Review of records for all eligible children seen during the study period began in October 1996 and continued through January 1997. All reviews were done at the clinic using a laptop computer for direct entry into an onscreen data entry form. The chart reviewer was blinded as to intervention or control status. The data collected included (1) child’s name; (2) record number; (3) date of birth; (4) dates of vaccine administration for hepatitis B (HEPB),diphtheria/tetanus/pertussis (DTP), oral polio vaccine (OPV), Haemophilus influenzae type B (HIB), and measles/mumps/rubella (MMR); (5) dates of clinic visits within the study period; (6) dates of canceled appointments within the study period; and (7) dates of appointments not canceled and not attended within the study period. The principal investigator (I.T.B.), blinded to control or intervention status, then assessed the completeness of immunization for all vaccines recommended by the 1995 Recommended Childhood Immunization Schedule based on the child’s age at the end of the study period.

Statistical methods

Chi-square tests were used to test for association between on-time vaccination status and intervention-versus-control group membership. To test for a possible difference in mean age between the 2 groups, a t-test was performed. On-time vaccination for DTP3 was defined as vaccination occurring between the ages of 3.5 months (the earliest age at which properly spaced vaccinations could be accomplished) and 7 months (ie, from the day of the 3.5-month birthday to the day the child turned 7 months old). On-time vaccination for DTP4 was defined as vaccination occurring between the ages of 12 and 19 months (ie, from the day of the first birthday to the day the child turned 19 months old). For MMR1, on-time vaccination was defined as vaccination occurring between the ages of 12 and 16 months; for HEPB3, between the ages of 5.9 and 19 months (HEPB3 is not recommended before 6 months of age, but immunization 2 to 3 days early is still likely to be immunogenic); and for OPV3, between the ages of 3.5 and 19 months. On-time vaccination for DTP5 and OPV4 was defined as vaccination occurring between then ages of 4 and 7 years (ie, from the day of the 4-year birthday to the day the child turned 7 years old).

Analyses of on-time vs not-on-time vaccination for DTP3, DTP4, DTP5, MMR1, HEPB3, OPV3, and OPV4 were performed within the subgroups of children who were eligible to receive the vaccine during the study period. All analyses were performed using SAS software (SAS Institute Inc, Cary, NC).

Immunizations

DTP.Children considered eligible for DTP3 immunization had not yet been immunized with DTP3 before the beginning of the study period; had been immunized with DTP2 by 3 months before the end of the study period; and were at least 3.5 months old by 1 month before the end of the study. Children considered eligible for DTP4 immunization had not yet been immunized with DTP4 before the beginning of the study; had been immunized with DTP3 by 7 months before the end of the study period; and were at least 12 months old by 1 month before the end of the study. Children considered eligible for DTP5 immunization had not yet been immunized with DTP5 by the beginning of the study; had been immunized with DTP4 before age 4 years; and were at least 4 years old by 1 month before the end of the study period.

MMR. Children considered eligible for MMR1 immunization had not yet been immunized with MMR1 by the beginning of the study and were at least 12 months old by 1 month before the end of the study.

HEBP. Children considered eligible for HEPB3 immunization had not yet been immunized with HEPB3 before the beginning of the study; had been immunized with HEPB2 by 3 months before the end of the study period; and were at least 5.9 months old by 1 month before the end of the study.

OPV. Children considered eligible for OPV3 immunization had not yet been immunized with OPV3 before the beginning of the study; had been immunized with OPV2 by 3 months before the end of the study period; and were at least 3.5 months old by 1 month before the end of the study period. Children considered eligible for OPV4 immunization had not been immunized with OPV4 by the beginning of the study; had been immunized with OPV3 before age 4 years; and were at least 4 years old by 1 month before the end of the study.

Results

A total of 977 charts were reviewed. At the LFHC, 637 charts were reviewed; at the BGFHC, 340 charts were reviewed. Among these 977 children, 448 had been assigned to the intervention group, and 529 had been assigned to the control group. The intervention group did not differ from the control group in mean age at the midpoint of the study (39.1 months vs 38.1 months, respectively; P = .33). The age distribution in each group is provided in Table 2. No statistically significant association was noted between assignment to group and FHC site. At the LFHC, 47% of children had been assigned to the intervention group, and at the BGFHC, 44% of children had been assigned to the intervention group (P = .35).

The intervention and control groups were compared with regard to the timeliness of receipt of DTP3, DTP4, DTP5, HEPB3, MMR1, OPV3, and OPV4 vaccinations as well as completeness of immunization for age. “Up to date for age” was defined as receipt of all immunizations as recommended by the 1995 Childhood Immunization Schedule for the child’s age at the end of the study period. Results of this comparison are shown in Table 3. Among the subgroup of children eligible to receive DTP4 vaccination during the study period (n = 224), 51% of the intervention group vs 36% of the control group received DTP4 vaccination on time (P = .03). For the subgroup of children eligible to receive MMR1 vaccination during the study period (n = 238), 42% of the intervention group compared with 26% of the control group received MMR1 on time (P = .01). For the subgroup of children eligible to receive OPV3 vaccination during the study period (n = 200), 70% of the intervention group compared with 56% of the control group received OPV3 vaccination on time (P = .04). For DTP3, DTP5, HEPB3, and OPV4 vaccination, no statistically significant difference was noted between intervention and control groups in the percent of children receiving vaccinations on time. Neither was there a statistically significant difference between the intervention and control groups for “up to date for age” (85% vs 80%; P = .09).

The total number of office visits during the study period did not differ between the 2 groups; both the intervention and control groups had a mean of 4 ± 0.1 visits during the study period. The groups also did not differ in the number appointments canceled or number of appointments not kept. A total of 54% of children received at least 1 immunization during the study period; there was no difference between the intervention and control groups (56% and 54%, respectively, P = .51).

Discussion

We found that a nurse-initiated prompt increased on-time vaccinations for DTP4, OPV3, and MMR1 by 14% to 16% by decreasing the number of missed opportunities for vaccination. Multiple studies show missed opportunities to vaccinate at acute care visits for mild illnesses.^3,6,20-22 Reasons for such missed opportunities include practice policies against vaccination at acute care visits, time limitations of acute care visits, focus on the initial agenda of the visit, concern about parental expectations, or overly cautious interpretation of contraindications.²³ Overcoming missed opportunities to vaccinate can involve a reminder prompt for the clinician as well as clinician education that vaccines may be given during mild acute illness and during most chronic illnesses. The Standards for Pediatric Immunization Practice urge that “providers utilize all clinical encounters to screen for needed vaccines and, when indicated, immunize children.”²⁴

Several educational materials-including the Standards for Pediatric Immunization Practice, the Guide to Contraindications to Childhood Vaccinations, and the Teaching Immunization for Medical Education (TIME) project^24-26-address missed opportunities. Gyorkos et al11 found that provider-oriented strategies, primarily chart reminders, increased pooled influenza vaccination rates by 18% (95% CI, 16-20) and pneumococcal vaccination rates by 7.5% (95% CI, 3-12). Other data show increases in pneumococcal vaccination rates of 10% and 41% and in influenza vaccine of 47%.^14-16 In 1 private practice, a systematic health maintenance protocol resulted in 95% of patients being offered vaccination compared with 45% of controls.²⁷ The Task Force on Community Preventive Services found that provider reminders and recall systems improved vaccination coverage and strongly recommends their use.¹⁷ Prompts have increased use of some other preventive services such as smoking cessation counseling as well as mammography and colorectal cancer screening.^16,28-30

Strengths of this study include (1) it was a randomized, controlled trial with blinding of the investigators to the patients’ group status and (2) it was conducted in the real-life setting of health centers that serve underprivileged persons in the inner city. One of the limitations of the study is that nurse staffing became short midway through the project, and the prompt sheets were not prepared on some days. Thus, the results likely underestimate the magnitude of the true effect if the intervention had been perfectly staffed; however, the positive effect despite this problem shows the robustness of prompts. Because the study took place only in urban health centers in Pittsburgh, generalizability may be limited.

Another limitation is the length of time since the study was performed. However, national immunization statistics from 2000 (the most recent available data) show that only 76% of children in the United States between 19 and 35 months of age received all the immunizations recommended for children by 18 months of age.¹ This is a mere increase of 2 percentage points since 1995, the year in which this study was started. Since that time more vaccines have come into routine use, making it more difficult for children to be up to date on their immunizations. The health center sites used for this study still do not have computerized databases that allow for computer generation of vaccine reminders or recalls.

Because the greatest improvement was seen regarding immunizations given during the second year of life, when rates of age-appropriate immunizations tend to decrease, it might be reasonable to implement this type of an intervention focused on the age group of 12 to 24 months. This would allow limited resources to be concentrated on the population most likely to benefit.

Although immunization rates have increased to record levels, they remain suboptimal. State laws requiring immunizations for school entry help ensure that school-aged children are adequately immunized, but younger children are at risk of inadequate immunization. In 1995, only 74% of children in the United States between the ages of 19 and 35 months had received all immunizations recommended for children 18 months of age.¹ A number of studies have focused on the causes of low immunization rates, which include poverty, inadequate clinician knowledge of contraindications, and missed opportunities to vaccinate.^2-8 In contrast, higher rates have been found when proactive office procedures-such as reminder systems,^9-12 audit and feedback systems,¹³ and provider prompts,^11,14-16-are implemented. The Task Force on Community Preventive Services found that provider reminder/recall systems improve vaccination coverage in adults, adolescents, and children and strongly recommends their use.¹⁷

The aim of this study was to determine whether a chart prompt, ie, a reminder to the patients’ physicians, is an effective method to increase pediatric immunization rates by decreasing missed opportunities. The physicians’ offices in this study had neither a reminder system in place for immunizations nor an existing database that would permit computer generation of such a reminder system for existing pediatric patients. Manual chart review offered the only method to determine immunization rates and to generate prompts.

Methods

Study population

The setting for this study was the St. Margaret Memorial Hospital Family Practice Residency Program. During the study period, 36 residents and 4 fellows saw patients in the 2 Family Health Centers (FHC) of the program. The Lawrenceville Family Health Center (LFHC) is located in the financially disadvantaged Lawrenceville neighborhood of urban Pittsburgh, serving a predominantly white population. In 1994, staff at the LFHC provided care for approximately 715 children younger than 6 years. The Bloomfield-Garfield Family Health Center (BGFHC) serves the urban Pittsburgh neighborhoods of Bloomfield and Garfield. The population served by this FHC is predominantly African American with significant minorities of Asian American and white patients. In 1994, staff at the BGFHC provided care for approximately 315 children younger than 6 years. Previous assessments of immunization rates showed that the LFHC and the BGFHC had significantly different pediatric immunization rates (47% of children fully immunized by their second birthday at LFHC compared with 33% at BGFHC; chart audit performed by I.T.B. in 1993, unpublished data). Charts of children born on or after May 1, 1989, were included in the study. The Institutional Review Board of the University of Pittsburgh approved the study.

Study protocol

The intervention phase of the project was initiated on July 1, 1995, and completed on June 30, 1996. Each patient chart was systematically assigned to the control or intervention group based on the chart number. A random-number table was used to generate the following scheme: children with a chart number ending in 0, 1, 2, 5, or 6 were assigned to the intervention group, and children with a chart number ending in 3, 4, 7, 8, or 9 were assigned to the control group. Each morning, a designated nurse at each FHC generated a list of patients scheduled to be seen that day. The names of patients eligible for the study were highlighted. For each child in the intervention group, the nurse reviewed the immunization record present on the child’s chart. Each participating nurse received instruction in chart review and immunization guidelines from the principal investigator. Based on this chart review, the nurse determined if the child was eligible for any immunizations on that day’s office visit. The 1995 Recommended Childhood Immunization Schedule of the Centers for Disease Control and Prevention¹⁸Table 1 was used as the primary guideline. The 1994 RedBook¹⁹ was used as the source for answers to questions about the eligibility for immunizations when the 1995 schedule was ambiguous or inappropriate (eg, for a child with delayed immunization or special medical circumstances).

The nurse completed 1-page forms and attached them to charts of children who might be eligible for immunization on that day’s visit. No form was completed for children who were up to date on immunizations. No chart review was performed that day on children in the control group, and no form was attached to the control group’s charts. The form acted as a prompt for the physician to ask the parent or guardian if the child had received the immunizations in question. If the physician determined there was a valid reason not to administer the vaccines on that visit, the physician noted the reason on the form and checked off the “Do not immunize” option on the form. If there were no contraindications to immunization, and if the parent or guardian gave informed consent for immunization, the physician checked off the “Immunize” option.

Evaluation of intervention

Review of records for all eligible children seen during the study period began in October 1996 and continued through January 1997. All reviews were done at the clinic using a laptop computer for direct entry into an onscreen data entry form. The chart reviewer was blinded as to intervention or control status. The data collected included (1) child’s name; (2) record number; (3) date of birth; (4) dates of vaccine administration for hepatitis B (HEPB),diphtheria/tetanus/pertussis (DTP), oral polio vaccine (OPV), Haemophilus influenzae type B (HIB), and measles/mumps/rubella (MMR); (5) dates of clinic visits within the study period; (6) dates of canceled appointments within the study period; and (7) dates of appointments not canceled and not attended within the study period. The principal investigator (I.T.B.), blinded to control or intervention status, then assessed the completeness of immunization for all vaccines recommended by the 1995 Recommended Childhood Immunization Schedule based on the child’s age at the end of the study period.

Statistical methods

Chi-square tests were used to test for association between on-time vaccination status and intervention-versus-control group membership. To test for a possible difference in mean age between the 2 groups, a t-test was performed. On-time vaccination for DTP3 was defined as vaccination occurring between the ages of 3.5 months (the earliest age at which properly spaced vaccinations could be accomplished) and 7 months (ie, from the day of the 3.5-month birthday to the day the child turned 7 months old). On-time vaccination for DTP4 was defined as vaccination occurring between the ages of 12 and 19 months (ie, from the day of the first birthday to the day the child turned 19 months old). For MMR1, on-time vaccination was defined as vaccination occurring between the ages of 12 and 16 months; for HEPB3, between the ages of 5.9 and 19 months (HEPB3 is not recommended before 6 months of age, but immunization 2 to 3 days early is still likely to be immunogenic); and for OPV3, between the ages of 3.5 and 19 months. On-time vaccination for DTP5 and OPV4 was defined as vaccination occurring between then ages of 4 and 7 years (ie, from the day of the 4-year birthday to the day the child turned 7 years old).

Analyses of on-time vs not-on-time vaccination for DTP3, DTP4, DTP5, MMR1, HEPB3, OPV3, and OPV4 were performed within the subgroups of children who were eligible to receive the vaccine during the study period. All analyses were performed using SAS software (SAS Institute Inc, Cary, NC).

Immunizations

DTP.Children considered eligible for DTP3 immunization had not yet been immunized with DTP3 before the beginning of the study period; had been immunized with DTP2 by 3 months before the end of the study period; and were at least 3.5 months old by 1 month before the end of the study. Children considered eligible for DTP4 immunization had not yet been immunized with DTP4 before the beginning of the study; had been immunized with DTP3 by 7 months before the end of the study period; and were at least 12 months old by 1 month before the end of the study. Children considered eligible for DTP5 immunization had not yet been immunized with DTP5 by the beginning of the study; had been immunized with DTP4 before age 4 years; and were at least 4 years old by 1 month before the end of the study period.

MMR. Children considered eligible for MMR1 immunization had not yet been immunized with MMR1 by the beginning of the study and were at least 12 months old by 1 month before the end of the study.

HEBP. Children considered eligible for HEPB3 immunization had not yet been immunized with HEPB3 before the beginning of the study; had been immunized with HEPB2 by 3 months before the end of the study period; and were at least 5.9 months old by 1 month before the end of the study.

OPV. Children considered eligible for OPV3 immunization had not yet been immunized with OPV3 before the beginning of the study; had been immunized with OPV2 by 3 months before the end of the study period; and were at least 3.5 months old by 1 month before the end of the study period. Children considered eligible for OPV4 immunization had not been immunized with OPV4 by the beginning of the study; had been immunized with OPV3 before age 4 years; and were at least 4 years old by 1 month before the end of the study.

Results

A total of 977 charts were reviewed. At the LFHC, 637 charts were reviewed; at the BGFHC, 340 charts were reviewed. Among these 977 children, 448 had been assigned to the intervention group, and 529 had been assigned to the control group. The intervention group did not differ from the control group in mean age at the midpoint of the study (39.1 months vs 38.1 months, respectively; P = .33). The age distribution in each group is provided in Table 2. No statistically significant association was noted between assignment to group and FHC site. At the LFHC, 47% of children had been assigned to the intervention group, and at the BGFHC, 44% of children had been assigned to the intervention group (P = .35).

The intervention and control groups were compared with regard to the timeliness of receipt of DTP3, DTP4, DTP5, HEPB3, MMR1, OPV3, and OPV4 vaccinations as well as completeness of immunization for age. “Up to date for age” was defined as receipt of all immunizations as recommended by the 1995 Childhood Immunization Schedule for the child’s age at the end of the study period. Results of this comparison are shown in Table 3. Among the subgroup of children eligible to receive DTP4 vaccination during the study period (n = 224), 51% of the intervention group vs 36% of the control group received DTP4 vaccination on time (P = .03). For the subgroup of children eligible to receive MMR1 vaccination during the study period (n = 238), 42% of the intervention group compared with 26% of the control group received MMR1 on time (P = .01). For the subgroup of children eligible to receive OPV3 vaccination during the study period (n = 200), 70% of the intervention group compared with 56% of the control group received OPV3 vaccination on time (P = .04). For DTP3, DTP5, HEPB3, and OPV4 vaccination, no statistically significant difference was noted between intervention and control groups in the percent of children receiving vaccinations on time. Neither was there a statistically significant difference between the intervention and control groups for “up to date for age” (85% vs 80%; P = .09).

The total number of office visits during the study period did not differ between the 2 groups; both the intervention and control groups had a mean of 4 ± 0.1 visits during the study period. The groups also did not differ in the number appointments canceled or number of appointments not kept. A total of 54% of children received at least 1 immunization during the study period; there was no difference between the intervention and control groups (56% and 54%, respectively, P = .51).

Discussion

We found that a nurse-initiated prompt increased on-time vaccinations for DTP4, OPV3, and MMR1 by 14% to 16% by decreasing the number of missed opportunities for vaccination. Multiple studies show missed opportunities to vaccinate at acute care visits for mild illnesses.^3,6,20-22 Reasons for such missed opportunities include practice policies against vaccination at acute care visits, time limitations of acute care visits, focus on the initial agenda of the visit, concern about parental expectations, or overly cautious interpretation of contraindications.²³ Overcoming missed opportunities to vaccinate can involve a reminder prompt for the clinician as well as clinician education that vaccines may be given during mild acute illness and during most chronic illnesses. The Standards for Pediatric Immunization Practice urge that “providers utilize all clinical encounters to screen for needed vaccines and, when indicated, immunize children.”²⁴

Several educational materials-including the Standards for Pediatric Immunization Practice, the Guide to Contraindications to Childhood Vaccinations, and the Teaching Immunization for Medical Education (TIME) project^24-26-address missed opportunities. Gyorkos et al11 found that provider-oriented strategies, primarily chart reminders, increased pooled influenza vaccination rates by 18% (95% CI, 16-20) and pneumococcal vaccination rates by 7.5% (95% CI, 3-12). Other data show increases in pneumococcal vaccination rates of 10% and 41% and in influenza vaccine of 47%.^14-16 In 1 private practice, a systematic health maintenance protocol resulted in 95% of patients being offered vaccination compared with 45% of controls.²⁷ The Task Force on Community Preventive Services found that provider reminders and recall systems improved vaccination coverage and strongly recommends their use.¹⁷ Prompts have increased use of some other preventive services such as smoking cessation counseling as well as mammography and colorectal cancer screening.^16,28-30

Strengths of this study include (1) it was a randomized, controlled trial with blinding of the investigators to the patients’ group status and (2) it was conducted in the real-life setting of health centers that serve underprivileged persons in the inner city. One of the limitations of the study is that nurse staffing became short midway through the project, and the prompt sheets were not prepared on some days. Thus, the results likely underestimate the magnitude of the true effect if the intervention had been perfectly staffed; however, the positive effect despite this problem shows the robustness of prompts. Because the study took place only in urban health centers in Pittsburgh, generalizability may be limited.

Another limitation is the length of time since the study was performed. However, national immunization statistics from 2000 (the most recent available data) show that only 76% of children in the United States between 19 and 35 months of age received all the immunizations recommended for children by 18 months of age.¹ This is a mere increase of 2 percentage points since 1995, the year in which this study was started. Since that time more vaccines have come into routine use, making it more difficult for children to be up to date on their immunizations. The health center sites used for this study still do not have computerized databases that allow for computer generation of vaccine reminders or recalls.

Because the greatest improvement was seen regarding immunizations given during the second year of life, when rates of age-appropriate immunizations tend to decrease, it might be reasonable to implement this type of an intervention focused on the age group of 12 to 24 months. This would allow limited resources to be concentrated on the population most likely to benefit.

Although immunization rates have increased to record levels, they remain suboptimal. State laws requiring immunizations for school entry help ensure that school-aged children are adequately immunized, but younger children are at risk of inadequate immunization. In 1995, only 74% of children in the United States between the ages of 19 and 35 months had received all immunizations recommended for children 18 months of age.¹ A number of studies have focused on the causes of low immunization rates, which include poverty, inadequate clinician knowledge of contraindications, and missed opportunities to vaccinate.^2-8 In contrast, higher rates have been found when proactive office procedures-such as reminder systems,^9-12 audit and feedback systems,¹³ and provider prompts,^11,14-16-are implemented. The Task Force on Community Preventive Services found that provider reminder/recall systems improve vaccination coverage in adults, adolescents, and children and strongly recommends their use.¹⁷

The aim of this study was to determine whether a chart prompt, ie, a reminder to the patients’ physicians, is an effective method to increase pediatric immunization rates by decreasing missed opportunities. The physicians’ offices in this study had neither a reminder system in place for immunizations nor an existing database that would permit computer generation of such a reminder system for existing pediatric patients. Manual chart review offered the only method to determine immunization rates and to generate prompts.

Methods

Study population

The setting for this study was the St. Margaret Memorial Hospital Family Practice Residency Program. During the study period, 36 residents and 4 fellows saw patients in the 2 Family Health Centers (FHC) of the program. The Lawrenceville Family Health Center (LFHC) is located in the financially disadvantaged Lawrenceville neighborhood of urban Pittsburgh, serving a predominantly white population. In 1994, staff at the LFHC provided care for approximately 715 children younger than 6 years. The Bloomfield-Garfield Family Health Center (BGFHC) serves the urban Pittsburgh neighborhoods of Bloomfield and Garfield. The population served by this FHC is predominantly African American with significant minorities of Asian American and white patients. In 1994, staff at the BGFHC provided care for approximately 315 children younger than 6 years. Previous assessments of immunization rates showed that the LFHC and the BGFHC had significantly different pediatric immunization rates (47% of children fully immunized by their second birthday at LFHC compared with 33% at BGFHC; chart audit performed by I.T.B. in 1993, unpublished data). Charts of children born on or after May 1, 1989, were included in the study. The Institutional Review Board of the University of Pittsburgh approved the study.

Study protocol

The intervention phase of the project was initiated on July 1, 1995, and completed on June 30, 1996. Each patient chart was systematically assigned to the control or intervention group based on the chart number. A random-number table was used to generate the following scheme: children with a chart number ending in 0, 1, 2, 5, or 6 were assigned to the intervention group, and children with a chart number ending in 3, 4, 7, 8, or 9 were assigned to the control group. Each morning, a designated nurse at each FHC generated a list of patients scheduled to be seen that day. The names of patients eligible for the study were highlighted. For each child in the intervention group, the nurse reviewed the immunization record present on the child’s chart. Each participating nurse received instruction in chart review and immunization guidelines from the principal investigator. Based on this chart review, the nurse determined if the child was eligible for any immunizations on that day’s office visit. The 1995 Recommended Childhood Immunization Schedule of the Centers for Disease Control and Prevention¹⁸Table 1 was used as the primary guideline. The 1994 RedBook¹⁹ was used as the source for answers to questions about the eligibility for immunizations when the 1995 schedule was ambiguous or inappropriate (eg, for a child with delayed immunization or special medical circumstances).

The nurse completed 1-page forms and attached them to charts of children who might be eligible for immunization on that day’s visit. No form was completed for children who were up to date on immunizations. No chart review was performed that day on children in the control group, and no form was attached to the control group’s charts. The form acted as a prompt for the physician to ask the parent or guardian if the child had received the immunizations in question. If the physician determined there was a valid reason not to administer the vaccines on that visit, the physician noted the reason on the form and checked off the “Do not immunize” option on the form. If there were no contraindications to immunization, and if the parent or guardian gave informed consent for immunization, the physician checked off the “Immunize” option.

Evaluation of intervention

Review of records for all eligible children seen during the study period began in October 1996 and continued through January 1997. All reviews were done at the clinic using a laptop computer for direct entry into an onscreen data entry form. The chart reviewer was blinded as to intervention or control status. The data collected included (1) child’s name; (2) record number; (3) date of birth; (4) dates of vaccine administration for hepatitis B (HEPB),diphtheria/tetanus/pertussis (DTP), oral polio vaccine (OPV), Haemophilus influenzae type B (HIB), and measles/mumps/rubella (MMR); (5) dates of clinic visits within the study period; (6) dates of canceled appointments within the study period; and (7) dates of appointments not canceled and not attended within the study period. The principal investigator (I.T.B.), blinded to control or intervention status, then assessed the completeness of immunization for all vaccines recommended by the 1995 Recommended Childhood Immunization Schedule based on the child’s age at the end of the study period.

Statistical methods

Chi-square tests were used to test for association between on-time vaccination status and intervention-versus-control group membership. To test for a possible difference in mean age between the 2 groups, a t-test was performed. On-time vaccination for DTP3 was defined as vaccination occurring between the ages of 3.5 months (the earliest age at which properly spaced vaccinations could be accomplished) and 7 months (ie, from the day of the 3.5-month birthday to the day the child turned 7 months old). On-time vaccination for DTP4 was defined as vaccination occurring between the ages of 12 and 19 months (ie, from the day of the first birthday to the day the child turned 19 months old). For MMR1, on-time vaccination was defined as vaccination occurring between the ages of 12 and 16 months; for HEPB3, between the ages of 5.9 and 19 months (HEPB3 is not recommended before 6 months of age, but immunization 2 to 3 days early is still likely to be immunogenic); and for OPV3, between the ages of 3.5 and 19 months. On-time vaccination for DTP5 and OPV4 was defined as vaccination occurring between then ages of 4 and 7 years (ie, from the day of the 4-year birthday to the day the child turned 7 years old).

Analyses of on-time vs not-on-time vaccination for DTP3, DTP4, DTP5, MMR1, HEPB3, OPV3, and OPV4 were performed within the subgroups of children who were eligible to receive the vaccine during the study period. All analyses were performed using SAS software (SAS Institute Inc, Cary, NC).

Immunizations

DTP.Children considered eligible for DTP3 immunization had not yet been immunized with DTP3 before the beginning of the study period; had been immunized with DTP2 by 3 months before the end of the study period; and were at least 3.5 months old by 1 month before the end of the study. Children considered eligible for DTP4 immunization had not yet been immunized with DTP4 before the beginning of the study; had been immunized with DTP3 by 7 months before the end of the study period; and were at least 12 months old by 1 month before the end of the study. Children considered eligible for DTP5 immunization had not yet been immunized with DTP5 by the beginning of the study; had been immunized with DTP4 before age 4 years; and were at least 4 years old by 1 month before the end of the study period.

MMR. Children considered eligible for MMR1 immunization had not yet been immunized with MMR1 by the beginning of the study and were at least 12 months old by 1 month before the end of the study.

HEBP. Children considered eligible for HEPB3 immunization had not yet been immunized with HEPB3 before the beginning of the study; had been immunized with HEPB2 by 3 months before the end of the study period; and were at least 5.9 months old by 1 month before the end of the study.

OPV. Children considered eligible for OPV3 immunization had not yet been immunized with OPV3 before the beginning of the study; had been immunized with OPV2 by 3 months before the end of the study period; and were at least 3.5 months old by 1 month before the end of the study period. Children considered eligible for OPV4 immunization had not been immunized with OPV4 by the beginning of the study; had been immunized with OPV3 before age 4 years; and were at least 4 years old by 1 month before the end of the study.

Results

A total of 977 charts were reviewed. At the LFHC, 637 charts were reviewed; at the BGFHC, 340 charts were reviewed. Among these 977 children, 448 had been assigned to the intervention group, and 529 had been assigned to the control group. The intervention group did not differ from the control group in mean age at the midpoint of the study (39.1 months vs 38.1 months, respectively; P = .33). The age distribution in each group is provided in Table 2. No statistically significant association was noted between assignment to group and FHC site. At the LFHC, 47% of children had been assigned to the intervention group, and at the BGFHC, 44% of children had been assigned to the intervention group (P = .35).

The intervention and control groups were compared with regard to the timeliness of receipt of DTP3, DTP4, DTP5, HEPB3, MMR1, OPV3, and OPV4 vaccinations as well as completeness of immunization for age. “Up to date for age” was defined as receipt of all immunizations as recommended by the 1995 Childhood Immunization Schedule for the child’s age at the end of the study period. Results of this comparison are shown in Table 3. Among the subgroup of children eligible to receive DTP4 vaccination during the study period (n = 224), 51% of the intervention group vs 36% of the control group received DTP4 vaccination on time (P = .03). For the subgroup of children eligible to receive MMR1 vaccination during the study period (n = 238), 42% of the intervention group compared with 26% of the control group received MMR1 on time (P = .01). For the subgroup of children eligible to receive OPV3 vaccination during the study period (n = 200), 70% of the intervention group compared with 56% of the control group received OPV3 vaccination on time (P = .04). For DTP3, DTP5, HEPB3, and OPV4 vaccination, no statistically significant difference was noted between intervention and control groups in the percent of children receiving vaccinations on time. Neither was there a statistically significant difference between the intervention and control groups for “up to date for age” (85% vs 80%; P = .09).

The total number of office visits during the study period did not differ between the 2 groups; both the intervention and control groups had a mean of 4 ± 0.1 visits during the study period. The groups also did not differ in the number appointments canceled or number of appointments not kept. A total of 54% of children received at least 1 immunization during the study period; there was no difference between the intervention and control groups (56% and 54%, respectively, P = .51).

Discussion

We found that a nurse-initiated prompt increased on-time vaccinations for DTP4, OPV3, and MMR1 by 14% to 16% by decreasing the number of missed opportunities for vaccination. Multiple studies show missed opportunities to vaccinate at acute care visits for mild illnesses.^3,6,20-22 Reasons for such missed opportunities include practice policies against vaccination at acute care visits, time limitations of acute care visits, focus on the initial agenda of the visit, concern about parental expectations, or overly cautious interpretation of contraindications.²³ Overcoming missed opportunities to vaccinate can involve a reminder prompt for the clinician as well as clinician education that vaccines may be given during mild acute illness and during most chronic illnesses. The Standards for Pediatric Immunization Practice urge that “providers utilize all clinical encounters to screen for needed vaccines and, when indicated, immunize children.”²⁴

Several educational materials-including the Standards for Pediatric Immunization Practice, the Guide to Contraindications to Childhood Vaccinations, and the Teaching Immunization for Medical Education (TIME) project^24-26-address missed opportunities. Gyorkos et al11 found that provider-oriented strategies, primarily chart reminders, increased pooled influenza vaccination rates by 18% (95% CI, 16-20) and pneumococcal vaccination rates by 7.5% (95% CI, 3-12). Other data show increases in pneumococcal vaccination rates of 10% and 41% and in influenza vaccine of 47%.^14-16 In 1 private practice, a systematic health maintenance protocol resulted in 95% of patients being offered vaccination compared with 45% of controls.²⁷ The Task Force on Community Preventive Services found that provider reminders and recall systems improved vaccination coverage and strongly recommends their use.¹⁷ Prompts have increased use of some other preventive services such as smoking cessation counseling as well as mammography and colorectal cancer screening.^16,28-30

Strengths of this study include (1) it was a randomized, controlled trial with blinding of the investigators to the patients’ group status and (2) it was conducted in the real-life setting of health centers that serve underprivileged persons in the inner city. One of the limitations of the study is that nurse staffing became short midway through the project, and the prompt sheets were not prepared on some days. Thus, the results likely underestimate the magnitude of the true effect if the intervention had been perfectly staffed; however, the positive effect despite this problem shows the robustness of prompts. Because the study took place only in urban health centers in Pittsburgh, generalizability may be limited.

Another limitation is the length of time since the study was performed. However, national immunization statistics from 2000 (the most recent available data) show that only 76% of children in the United States between 19 and 35 months of age received all the immunizations recommended for children by 18 months of age.¹ This is a mere increase of 2 percentage points since 1995, the year in which this study was started. Since that time more vaccines have come into routine use, making it more difficult for children to be up to date on their immunizations. The health center sites used for this study still do not have computerized databases that allow for computer generation of vaccine reminders or recalls.

Because the greatest improvement was seen regarding immunizations given during the second year of life, when rates of age-appropriate immunizations tend to decrease, it might be reasonable to implement this type of an intervention focused on the age group of 12 to 24 months. This would allow limited resources to be concentrated on the population most likely to benefit.