Vaccination Programs: Client Reminder and Recall Systems

This review was conducted on behalf of the CPSTF by a team of specialists in systematic review methods, and in research, practice, and policy related to increasing appropriate vaccination.

Twenty-nine studies were included in the systematic review.

• Overall vaccination rates increased by a median of 11 percentage points.
  • Client reminder and recall interventions used alone: median increase of 6 percentage points (14 studies).
  • Interventions implemented with additional components: median increase of 12 percentage points (15 studies).

Evidence indicates client reminder and recall systems can reach a large number of clients and increase vaccination rates with relatively few economic resources.

Twenty-four studies were included in the review (search period 1980 2012). All monetary values are presented in 2013 U.S. dollars.

• Median intervention group size: 654 (23 studies).
• Median cost per person per year: $2.13 (23 studies)
• Median cost per additional person vaccinated: $15 (22 studies)

• Children, adolescents, and adults
• Various populations
• Clinical and community settings at different levels of scale from individual practice settings to entire communities
• Programs that use a range of intervention characteristics (e.g., reminder or recall, content, theoretical basis, and method of delivery)
• Programs used alone or with additional components
• Assorted vaccinations

• How effective are client reminders that use emerging communication technologies such as text messaging?
• Are interventions implemented by individual providers or healthcare systems, or through immunization information systems (IIS) effective? If so, are they cost-effective?
• How effective are interventions with adolescent populations?
• How effective are client reminders in communities with disparities in vaccination rates?

• Evaluated interventions were used alone (14 studies) or with additional interventions (15 studies).
• Studies were conducted in the United States (18 studies), Canada (3 studies), Denmark (1 study), New Zealand (1 study), and the United Kingdom (1 study).
• Studies evaluated programs to address influenza (8 studies), influenza/pneumococcal (1 study), pneumococcal (1 study), DTP/DTaP (1 study), MMR (2 studies), tetanus (2 studies), and various childhood series (9 studies).

Effectiveness Review

• Logic Model [PDF (Print Only) – 285 kB]
• Text Description [PDF – 171 kB]

When starting an effectiveness review, the systematic review team develops an analytic framework. The analytic framework illustrates how the intervention approach is thought to affect public health. It guides the search for evidence and may be used to summarize the evidence collected. The analytic framework often includes intermediate outcomes, potential effect modifiers, potential harms, and potential additional benefits.

Economic Review

No content is available for this section.

Effectiveness Review

• Summary Evidence Table – Search Period: 1997-2007 [PDF – 548 kB]
• Summary Evidence Table – Search Period: 2007-2012 [PDF – 172 kB]

Economic Review

No content is available for this section.

Effectiveness Review

Baker AM, McCarthy B, Gurley VF, Yood MU. Influenza immunization in a managed care organization. Journal of General Internal Medicine 1998;13:469-75.

Britto MT, Pandzik GM, Meeks CS, Kotagal UR. Combining evidence and diffusion of innovation theory to enhance influenza immunization. Joint Commission Journal on Quality and Patient Safety 2006;32(8):426-32.

Daley MF, Barrow J, Pearson K, et al. Identification and recall of children with chronic medical conditions for influenza vaccination. Pediatrics 2004;113:e26-e33.

Daley MF, Steiner JF, Kempe A, et al. Quality improvement in immunization delivery following an unsuccessful immunization recall. Ambulatory Pediatrics 2004;4:217-23.

Dini EF, Linkins RW, Sigafoos J. The impact of computer-generated messages on childhood immunization coverage. American Journal of Preventive Medicine 2000;18:132-9.

Findley SE, Sanchez M, Mejia M, Ferreira R, Pena O, Matos S et al. Effective strategies for integrating immunization promotion into community programs. Health Promotion Practice 2009;10(2 Suppl):128S-37S.

Gill JM, Saldarriaga AM. The impact of a computerized physician reminder and a mailed patient reminder on influenza immunizations for older patients. Delaware Medical Journal 2000;72(10):425-30.

Hambidge SJ, Davidson AJ, Phibbs SL, Chandramouli V, Zerbe G, LeBaron CW et al. Strategies to improve immunization rates and well-child care in a disadvantaged population: a cluster randomized controlled trial. Archives of Pediatrics & Adolescent Medicine 2004; 158:162-9.

Hambidge SJ, Phibbs SL, Chandramouli V, Fairclough D, Steiner JF. A stepped intervention increases well-child care and immunization rates in a disadvantaged population. Pediatrics 2009;124(2):455-64.

Hogg W, Bass MJ, Colange N, Crouch H, Satenstein G. Randomized controlled study of customized preventive medicine reminder letters in a community practice. Canadian Family Physician 1998;44:81-8.

Hull S, Hagdrup N, Hart B, Griffiths C, Hennessy E. Boosting uptake of influenza immunisation: a randomised controlled trial of telephone appointing in general practice. British Journal of General Practice 2002;52:712-6.

Humiston SG, Bennett NM, Long C, et al. Increasing inner-city adult influenza vaccination rates: a randomized controlled trial. Public Health Reports 2011;126:39-47.

Irigoyen MM, Findley S, Wang D, et al. Challenges and successes of immunization registry reminders at inner-city practices. Ambulatory Pediatrics 2006;6:100-4.

Kempe A, Daley MF, Barrow J, et al. Implementation of universal influenza immunization recommendations for healthy young children: results of a randomized, controlled trial with registry-based recall. Pediatrics 2005;115:146-54.

Kharbanda EO1, Stockwell MS, Fox HW, Andres R, Lara M, Rickert VI. Text message reminders to promote human papillomavirus vaccination. Vaccine 2011;29(14):2537-41. doi: 10.1016/j.vaccine.2011.01.065.

LeBaron CW, Starnes D, Dini EF, Chambliss JW, Chaney M. The impact of interventions by a community-based organization on inner-city vaccination coverage: Fulton County, Georgia, 1992-1993. Arch Pediatr Adolesc Med 1998;152(4):327-32.

LeBaron CW, Starnes DM, Rask KJ. The impact of reminder-recall interventions on low vaccination coverage in an inner-city population. Archives of Pediatrics & Adolescent Medicine 2004;158:255-61.

Lemstra M, Rajakumar D, Thompson A, Moraros J. The effectiveness of telephone reminders and home visits to improve measles, mumps and rubella immunization coverage rates in children. Paediatrics and Child Health 2011;16(1):e1-5.

Lieu TA, Capra AM, Makol J, Black SB, Shinefield HR. Effectiveness and cost-effectiveness of letters, automated telephone messages, or both for underimmunized children in a health maintenance organization. Pediatrics 1998;101:E3.

Muehleisen B, Baer G, Schaad UB, Heininger U. Assessment of immunization status in hospitalized children followed by counseling of parents and primary care physicians improves vaccination coverage: an interventional study. Journal of Pediatrics 2007; 151(6):704-6.

Nowalk MP, Zimmerman RK, Lin CJ, et al. Raising adult vaccination rates over 4 years among racially diverse patients at inner-city health centers. Journal of the American Geriatrics Society 2008;56(7):1177-82

Puech M, Ward J, Lajoie V. Postcard reminders from GPs for influenza vaccine: are they more effective than an ad hoc approach? Australian and New Zealand Journal of Public Health 1998;22:254-6.

Rodewald LE, Szilagyi PG, Humiston SG, Barth R, Kraus R, Raubertas RF. A randomized study of tracking with outreach and provider prompting to improve immunization coverage and primary care. Pediatrics 1999;103:31-8.

Sansom S, Rudy E, Strine T, Douglas W. Hepatitis A and B vaccination in a sexually transmitted disease clinic for men who have sex with men. Sexually Transmitted Diseases 2003;30:685-8.

Stockwell MS1, Kharbanda EO, Martinez RA, Vargas CY, Vawdrey DK, Camargo S. Effect of a text messaging intervention on influenza vaccination in an urban, low-income pediatric and adolescent population: a randomized controlled trial. JAMA 2012;307(16):1702-8. doi: 10.1001/jama.2012.502.

Szilagyi PG, Schaffer S, Barth R, et al. Effect of telephone reminder/recall on adolescent immunization and preventive visits: results from a randomized clinical trial. Archives of Pediatrics & Adolescent Medicine 2006;160:157-63.

Szilagyi PG, Humiston SG, Gallivan S, Albertin C, Sandler M, Blumkin A. Effectiveness of a citywide patient immunization navigator program on improving adolescent immunizations and preventive care visit rates. Archives of Pediatrics & Adolescent Medicine 2011;165(6):547-53.

Vora S, Verber L, Potts S, Dozier T, Daum RS. Effect of a novel birth intervention and reminder-recall on on-time immunization compliance in high-risk children. Human Vaccines 2009; 5(6):395-402.

Winston CA, Mims AD, Leatherwood KA. Increasing pneumococcal vaccination in managed care through telephone outreach. American Journal of Managed Care 2007;13(10):581-8.

Wood D, Halfon N, Donald SC, et al. Increasing immunization rates among inner-city, African American children. A randomized trial of case management. JAMA 1998;279:29-34.

Economic Review

Baker AM, McCarthy B, Gurley VF, Yood MU. Influenza immunization in a managed care organization. J Gen Intern Med. 1998;13(7):469-475. http://dx.doi.org/10.1046/j.1525-1497.1998.00136.x.

Berg GD, Thomas E, Silverstein S, Neel CL, Mireles M. Reducing medical service utilization by encouraging vaccines Randomized controlled trial. Am J Prev Med 2004;27(4):284-8.

Buchner DM, Larson EB, White RF. Influenza vaccination in community elderly. A controlled trial of postcard reminders. J Am Geriatr Soc 1987;35(8):755. http://dx.doi.org/10.1111/j.1532-5415.1987.tb06354.x.

Chiu TT. Community mobilization for preschool immunizations: the Shots by Two Project. Am J Public Health 1997;87(3):462-3.

Dini EF, Linkins RW, Sigafoos J. The impact of computer-generated messages on childhood immunization coverage. Am J Prev Med 2000;18(2):132-9. http://dx.doi.org/10.1016/S0749-3797(99)00086-0.

Frame PS, Zimmer JG, Werth PL, Hall WJ, Eberly SW. Computer-based vs manual health maintenance tracking: a controlled trial. Arch Fam Med 1994;3(7):581. http://dx.doi.org/10.1001/archfami.3.7.581.

Frank JW, McMurray L, Henderson M. Influenza vaccination in the elderly: 2. The economics of sending reminder letters. CMAJ 1985;132(5):516.

Franzini L, Rosenthal J, Spears W, et al. Cost-effectiveness of childhood immunization reminder/recall systems in urban private practices. Pediatrics 2000;106(Suppl 1):177-83.

Grabenstein JD, Hartzema AG, Guess HA, Johnston WP, Rittenhouse BE. Community pharmacists as immunization advocates: cost-effectiveness of a cue to influenza vaccination. Med Care 1992:503-513. http://dx.doi.org/10.1097/00005650-199206000-00004.

Hull S, Hagdrup N, Hart B, Griffiths C, Hennessy E. Boosting uptake of influenza immunisation: a randomised controlled trial of telephone appointing in general practice. Br J Gen Pract 2002;52(482):712.

Irigoyen MM, Findley S, Earle B, Stambaugh K, Vaughan R. Impact of appointment reminders on vaccination coverage at an urban clinic. Pediatrics 2000;106(4):919.

Kreuter MW, Caburnay CA, Chen JJ, Donlin MJ. Effectiveness of individually tailored calendars in promoting childhood immunization in urban public health centers. Am J Public Health 2004;94(1):122. http://dx.doi.org/10.2105/AJPH.94.1.122.

LeBaron CW, Starnes DM, Rask KJ. The impact of reminder-recall interventions on low vaccination coverage in an inner-city population. Arch Pediatr Adolesc Med 2004;158(3):255-61. http://dx.doi.org/10.1001/archpedi.158.3.255.

Lemstra M, Rajakumar D, Thompson A, Moraros J. The effectiveness of telephone reminders and home visits to improve measles, mumps and rubella immunization coverage rates in children. Pediatr Child Health 2011;16(1):e1.

Lieu TA, Black SB, Ray P, et al. Computer-generated recall letters for underimmunized children: how cost-effective? Pediatr Infect Dis J 1997;16(1):28-33. http://dx.doi.org/10.1097/00006454-199701000-00007.

Lieu TA, Capra AM, Makol J, Black SB, Shinefield HR. Effectiveness and cost-effectiveness of letters, automated telephone messages, or both for underimmunized children in a health maintenance organization. Pediatrics 1998;101(4):e3. http://dx.doi.org/10.1542/peds.101.4.e3.

McLeod D, Bowie RD, Kljakovic M. The cost of childhood immunisation in general practice. N Z Med J 1998;111(1061):73.

Moran WP, Nelson K, Wofford JL, Velez R, Case LD. Increasing influenza immunization among high-risk patients: education or financial incentive? Am J Med. 1996;101(6):612-20. http://dx.doi.org/10.1016/S0002-9343(96)00327-0.

Nexoe JR, Kragstrup J, Ronne T. Impact of postal invitations and user fee on influenza vaccination rates among the elderly: a randomized controlled trial in general practice. Scand J Prim Health Care 1997;15(2):109-12. http://dx.doi.org/10.3109/02813439709018497.

Rask KJ, LeBaron CW, Starnes DM. The costs of registry-based immunization interventions. Am J Prev Med 2001;21(4):267-71. http://dx.doi.org/10.1016/S0749-3797(01)00370-1.

Rosser WW, Hutchison BG, McDowell I, Newell C. Use of reminders to increase compliance with tetanus booster vaccination. CMAJ 1992;146(6):911.

Smith DM, Zhou XH, Weinberger M, Smith F, McDonald RC. Mailed reminders for area-wide nfluenza immunization: a randomized controlled trial. J Am Geriatr Soc. 1999;47(1):1. http://dx.doi.org/10.1111/j.1532-5415.1999.tb01893.x.

Winston CA, Mims AD, Leatherwood KA. Increasing pneumococcal vaccination in managed care through telephone outreach. Am J Manag Care 2007;13(10):581.

Yokley JM, Glenwick DS. Increasing the immunization of preschool children; an evaluation of applied community interventions. J Appl Behav Anal 1984;17(3):313. http://dx.doi.org/10.1901/jaba.1984.17-313.

Effectiveness Review

The CPSTF findings are based on studies included in the original review (search period 1980-1997) combined with studies identified in the updated search (search period 1997- February 2012). Reference lists of articles reviewed as well as lists in review articles were also searched, and members of our coordination team were consulted for additional references.

Details of the original search (1980-1997)

The following five electronic databases were searched during the original review period of 1980 up to 1997: MEDLINE, Embase, Psychlit, CAB Health, and Sociological Abstracts. The team also reviewed reference lists in articles and consulted with immunization experts. To be included in the review, a study had to:

• have a publication date of 1980 1997;
• address universally recommended adult, adolescent, or childhood vaccinations;
• be a primary study rather than, for example, a guideline or review;
• take place in an industrialized country or countries;
• be written in English;
• meet the evidence review and Guide chapter development team’s definition of the interventions; provide information on one or more outcomes related to the analytic frameworks; and
• compare a group of persons who had been exposed to the intervention with a group who had not been exposed or who had been less exposed. In addition, we excluded studies with least suitable designs for two interventions (provider reminder/recall and client reminder/recall) where the literature was most extensive.

Details of the update search (1997- February 2012)

The team conducted a broad literature search to identify studies assessing the effectiveness of Vaccine Preventable Disease interventions in improving vaccination rates. The following nine databases were searched during the period of 1997 up to February 2012: CABI, CINAHL, The Cochrane Library, EMBASE, ERIC, MEDLINE, PSYCHINFO, Soci Abs and WOS. Reference lists of articles reviewed as well as lists in review articles were also searched, and subject matter experts consulted for additional references. To be included in the updated review, a study had to:

• have a publication date of 1997- February 2012;
• evaluate vaccinations with universal recommendations;
• meet the evidence review and Guide chapter development team’s definition of the interventions;
• be a primary research study with one or more outcomes related to the analytic frameworks;
• take place in an high income country or countries;
• be written in English
• compare a group of persons who had been exposed to the intervention with a group who had not been exposed or who had been less exposed. In addition, we excluded studies with least suitable designs for two interventions (provider reminder/recall and client reminder/recall) where the literature was most extensive

Search Terms

1. Immunization
2. Vaccination
3. Immunization Programs

Economic Review

The present review included studies that reported economic outcomes from the 2000 review (search period 1980-1997) combined with studies identified from updated searches (search period 1997- February 2012) within the standard medical and health-related research databases, Google Scholar, and databases specialized to economics and social sciences. The details of the two sets of searches are provided below.

Details of the Updated Search (1997- February 2012)

The team conducted a broad literature search to identify studies assessing interventions to improve vaccination rates. The following nine databases were searched during the period of 1997 up to February 2012: CABI, CINAHL, The Cochrane Library, EMBASE, ERIC, MEDLINE, PSYCHINFO, Soci Abs and WOS. In addition, Google Scholar and specialized databases (CRD-University of York: NHS EED, EconLit, and JSTOR), were also searched. Reference lists of articles reviewed as well as lists in review articles were also considered, and subject matter experts consulted for additional references.

Search Terms

• Immunization
• Vaccination
• Immunization Programs

To be included in the updated review, a study had to do the following:

• Have a publication date of 1997- February 2012
• Evaluate vaccinations with universal recommendations
• Meet the evidence review and Community Guide review team’s definition of the interventions
• Be a primary research study with one or more outcomes related to the analytic framework(s)
• Take place in a high income country or countries
• Be written in English; and
• Compare a group of persons who had been exposed to the intervention with a group who had not been exposed or who had been less exposed.

Details of the Original Search (1980-1997)

The following five electronic databases were searched during the original review period of 1980 up to 1997: MEDLINE, Embase, Psychlit, CAB Health, and Sociological Abstracts. The team also reviewed reference lists in articles and consulted with immunization experts. To be included, a study had to do the following:

• Have a publication date of 1980 1997
• Address universally recommended adult, adolescent, or childhood vaccinations
• Be a primary study rather than, for example, a guideline or review
• Take place in an industrialized country or countries
• Be written in English
• Meet the definition of the interventions
• Provide information on one or more outcomes related to the analytic frameworks; and
• Compare a group of persons who had been exposed to the intervention with a group who had not been exposed or who had been less exposed. In addition, we excluded studies with least suitable designs for two interventions (provider reminder/recall and client reminder/recall) where the literature was most extensive.

• Client reminder or recall systems can be included in two interventions approaches recommended by the CPSTF:
  • Health care system-based interventions implemented in combination
  • Community-based interventions implemented in combination
• Immunization information systems can be used to generate client reminder or recall notices.
• Client reminder and recall systems can be used for vaccination and additional preventive care notices.

Evidence-Based Cancer Control Programs (EBCCP)

Find programs from the EBCCP website that align with this systematic review. (What is EBCCP?)

Healthy People 2030

Healthy People 2030 includes the following objectives related to this CPSTF recommendation.

• Reduce the proportion of children who get no recommended vaccines by age 2 years — IID‑02
• Maintain the vaccination coverage level of 1 dose of the MMR vaccine in children by age 2 years — IID‑03
• Maintain the vaccination coverage level of 2 doses of the MMR vaccine for children in kindergarten — IID‑04
• Increase the coverage level of 4 doses of the DTaP vaccine in children by age 2 years — IID‑06
• Increase the proportion of people who get the flu vaccine every year — IID‑09
• Increase the proportion of adults age 19 years or older who get recommended vaccines — IID‑D03