Vaccination Programs: Health Care System-Based Interventions Implemented in Combination

Effectiveness Review

Analytic Framework [PDF - 186 KB]

When starting an effectiveness review, the systematic review team develops an analytic framework. The analytic framework illustrates how the intervention approach is thought to affect public health. It guides the search for evidence and may be used to summarize the evidence collected. The analytic framework often includes intermediate outcomes, potential effect modifiers, potential harms, and potential additional benefits.

Economic Review

No content is available for this section.

Effectiveness Review

Summary Evidence Table - Effectiveness Review [PDF - 524 KB]

Economic Review

Summary Evidence Table - Economic Review [PDF - 113 KB]

The number of studies and publications do not always correspond (e.g., a publication may include several studies or one study may be explained in several publications).

Effectiveness Review

Allison MA, Daley MF, Barrow J, et al. High influenza vaccination coverage in children with high-risk conditions during a vaccine shortage. Archives of Pediatrics & Adolescent Medicine 2009;163(5):426-31.

Barton MB, Schoenbaum SC. Improving influenza vaccination performance in an HMO setting: the use of computer-generated reminders and peer comparison feedback. Am J Public Health 1990;80(5):534-6.

Becker DM, Gomez EB, Kaiser DL, Yoshihasi A, Hodge RH, Jr. Improving preventive care at a medical clinic: how can the patient help? Am J Prev Med 1989;5(6):353-9.

Britto MT, Pandzik GM, Meeks CS, Kotagal UR. Combining evidence and diffusion of innovation theory to enhance influenza immunization. Joint Commission Journal on Quality and Patient Safety 2006;32(8):426-32.

Britto MT, Schoettker PJ, Pandzik GM, Weiland J, Mandel KE. Improving influenza immunisation for high-risk children and adolescents. Quality & Safety in Health Care 2007;16(5):363-8.

Browngoehl K, Kennedy K, Krotki K, Mainzer H. Increasing immunization: a Medicaid managed care model. Pediatrics 1997;99(1):E4.

Buffington J, Bell KM, LaForce FM. A target-based model for increasing influenza immunizations in private practice. Genesee Hospital Medical Staff. J Gen Intern Med 1991;6(3):204-9.

Coyle CM, Currie BP. Improving the rates of inpatient pneumococcal vaccination: impact of standing orders versus computerized reminders to physicians. Infection Control & Hospital Epidemiology 2004;25(11):904-7.

Daley MF, Steiner JF, Kempe A, et al. Quality improvement in immunization delivery following an unsuccessful immunization recall. Ambulatory Pediatrics 2004;4:217-23.

Dickey LL, Petitti D. A patient-held minirecord to promote adult preventive care. J Fam Pract 1992;34(4):457-63.

Donato AA, Motz LM, Wilson G, Lloyd BJ. Efficacy of multiple influenza vaccine delivery systems in a single facility. Infection Control & Hospital Epidemiology 2007;28(2):219-21.

Elster AB, Lamb ME, Tavare J, Ralston CW. The medical and psychosocial impact of comprehensive care on adolescent pregnancy and parenthood. JAMA 1987;258(9):1187-92.

Fairbrother G, Hanson KL, Friedman S, Butts GC. The impact of physician bonuses, enhanced fees, and feedback on childhood immunization coverage rates. American Journal of Public Health 1999;89:171-5.

Fiks AG, Grundmeier RW, Biggs LM, Localio AR, Alessandrini EA. Impact of clinical alerts within an electronic health record on routine childhood immunization in an urban pediatric population. Pediatrics 2007;120(4):707-14.

Fiks AG, Hunter KF, Localio AR, et al. Impact of electronic health record-based alerts on influenza vaccination for children with asthma. Pediatrics 2009;124(1):159-69.

Frame PS, Zimmer JG, Werth PL, Hall WJ, Eberly SW. Computer-based vs manual health maintenance tracking. A controlled trial. Arch Fam Med 1994;3(7):581-8.

Ginson SH, Malmberg C, French DJ. Impact on vaccination rates of a pharmacist-initiated influenza and pneumococcal vaccination program. Canadian Journal of Hospital Pharmacy 2000;53(4):270-5.

Hambidge SJ, Davidson AJ, Phibbs SL, et al. Strategies to improve immunization rates and well-child care in a disadvantaged population: a cluster randomized controlled trial. Archives of Pediatrics & Adolescent Medicine 2004;158:162-9.

Hambidge SJ, Phibbs SL, Chandramouli V, Fairclough D, Steiner JF. A stepped intervention increases well-child care and immunization rates in a disadvantaged population. Pediatrics 2009;124(2):455-64.

Harper PG, Madlon-Kay DJ, Luxenberg MG, Tempest R. A clinic system to improve preschool vaccinations in a low socioeconomic status population. Archives of Pediatrics and Adolescent Medicine 1997;151(12):1220-3.

Harper PG, Murray DM. An organizational strategy to improve adolescent measles-mumps-rubella vaccination in a low socioeconomic population. A method to reduce missed opportunities. Arch Fam Med 1994;3(3):257-62.

Herman CJ, Speroff T, Cebul RD. Improving compliance with immunization in the older adult: results of a randomized cohort study. J Am Geriatr Soc 1994;42(11):1154-9.

Hillman AL, Ripley K, Goldfarb N, Weiner J, Nuamah I, Lusk E. The use of physician financial incentives and feedback to improve pediatric preventive care in Medicaid managed care. Pediatrics 1999;104:931-5.

Humiston SG, Bennett NM, Long C, et al. Increasing inner-city adult influenza vaccination rates: a randomized controlled trial. Public Health Reports 2011;126:39-47.

Hutchison BG, Shannon HS. Effect of repeated annual reminder letters on influenza immunization among elderly patients. J Fam Pract 1991;33(2):187-9.

Irigoyen MM, Findley S, Wang D, et al. Challenges and successes of immunization registry reminders at inner-city practices. Ambulatory Pediatrics 2006;6:100-4.

Karuza J, Calkins E, Feather J, Hershey CO, Katz L, Majeroni B. Enhancing physician adoption of practice guidelines. Dissemination of influenza vaccination guideline using a small-group consensus process. Arch Intern Med 1995;155(6):625-32.

Kempe A, Daley MF, Barrow J, et al. Implementation of universal influenza immunization recommendations for healthy young children: results of a randomized, controlled trial with registry-based recall. Pediatrics 2005;115:146-54.

Kiefe CI, Allison JJ, Williams OD, Person SD, Weaver MT, Weissman NW. Improving quality improvement using achievable benchmarks for physician feedback: a randomized controlled trial. JAMA 2001;285:2871-9.

Korn JE, Schlossberg LA, Rich EC. Improved preventive care following an intervention during an ambulatory care rotation: carryover to a second setting. J Gen Intern Med 1988;3(2):156-60.

Latessa RA, Cummings DM, Lilley SH, Morrissey SL. Changing practices in the use of pneumococcal vaccine. Family Medicine 2000;32:196-200.

Lukasik MH, Pratt G. The telephone: an overlooked technology for prevention in family medicine. Can Fam Physician 1987;33:1997-2001.

Margolis KL, Nichol KL, Wuorenma J, Von Sternberg TL. Exporting a successful influenza vaccination program from a teaching hospital to a community outpatient setting. J Am Geriatr Soc 1992;40(10):1021-3.

Margolis PA, Lannon CM, Stuart JM, Fried BJ, Keyes EL, Moore DE. Practice based education to improve delivery systems for prevention in primary care: randomised trial. BMJ 2004;328:388.

Moran WP, Nelson K, Wofford JL, Velez R. Computer-generated mailed reminders for influenza immunization: a clinical trial. J Gen Intern Med 1992;7(5):535-7.

Moran WP, Nelson K, Wofford JL, Velez R, Case LD. Increasing influenza immunization among high-risk patients: education or financial incentive? Am J Med 1996;101(6):612-20.

Muehleisen B, Baer G, Schaad UB, Heininger U. Assessment of immunization status in hospitalized children followed by counseling of parents and primary care physicians improves vaccination coverage: an interventional study. Journal of Pediatrics 2007;151(6):704-6.

Nexoe J, Kragstrup J, Ronne T. Impact of postal invitations and user fee on influenza vaccination rates among the elderly. A randomized controlled trial in general practice. Scandinavian Journal of Primary Health Care 1997;15(2):109-12.

Nichol KL. Improving influenza vaccination rates for high-risk inpatients. Am J Med 1991;91(6):584-8.

Nichol KL, Korn JE, Margolis KL, Poland GA, Petzel RA, Lofgren RP. Achieving the national health objective for influenza immunization: success of an institution-wide vaccination program. Am J Med 1990;89(2):156-60.

Nowalk MP, Zimmerman RK, Lin CJ, et al. Raising adult vaccination rates over 4 years among racially diverse patients at inner-city health centers. Journal of the American Geriatrics Society 2008;56(7):1177-82.

Oeffinger KC, Roaten SP, Hitchcock MA, Oeffinger PK. The effect of patient education on pediatric immunization rates. J Fam Pract 1992;35(3):288-93.

Ornstein SM, Garr DR, Jenkins RG, Rust PF, Arnon A. Computer-generated physician and patient reminders. Tools to improve population adherence to selected preventive services. J Fam Pract 1991;32(1):82-90.

O'Sullivan AL, Jacobsen BS. A randomized trial of a health care program for first-time adolescent mothers and their infants. Nurs Res 1992;41(4):210-5.

Pappano D, Humiston S, Goepp J. Efficacy of a pediatric emergency department-based influenza vaccination program. Archives of Pediatrics and Adolescent Medicine 2004;158(11):1077-83.

Pierce C, Goldstein M, Suozzi K, Gallaher M, Dietz V, Stevenson J. The impact of the standards for pediatric immunization practices on vaccination coverage levels. JAMA 1996;276(8):626-30.

Quinley JC, Shih A. Improving physician coverage of pneumococcal vaccine: a randomized trial of a telephone intervention. Journal of Community Health 2004;29(2):103-15.

Rhew DC, Glassman PA, Goetz MB. Improving pneumococcal vaccine rates. Nurse protocols versus clinical reminders. Journal of General Internal Medicine 1999;14:351-6.

Rodewald LE, Szilagyi PG, Humiston SG, Barth R, Kraus R, Raubertas RF. A randomized study of tracking with outreach and provider prompting to improve immunization coverage and primary care. Pediatrics 1999;103:31-8.

Rust CT, Sisk FA, Kuo AR, Smith J, Miller R, Sullivan KM. Impact of resident feedback on immunization outcomes. Archives of Pediatrics & Adolescent Medicine 1999;153:1165-9.

Satterthwaite P. A randomised intervention study to examine the effect on immunisation coverage of making influenza vaccine available at no cost. New Zealand Medical Journal 1997;110:58-60.

Shaw JS, Samuels RC, Larusso EM, Bernstein HH. Impact of an encounter-based prompting system on resident vaccine administration performance and immunization knowledge. Pediatrics 2000;105:978-83.

Shevlin JD, Summers BC, Thomas D, Whitney CG, Todd D, Ray SM. A systematic approach for increasing pneumococcal vaccination rates at an inner-city public hospital. American Journal of Preventive Medicine 2002;22:92-7.

Slora EJ, Steffes JM, Harris D, et al. Improving pediatric practice immunization rates through distance-based quality improvement: a feasibility trial from PROS. Clinical Pediatrics 2008;47(1):25-36.

Soljak MA, Handford S. Early results from the Northland immunisation register. New Zealand Medical Journal 1987;100(822):244-6.

Swenson CJ, Appel A, Sheehan M, et al. Using information technology to improve adult immunization delivery in an integrated urban health system. Joint Commission Journal on Quality & Patient Safety 2012;38(1):15-23.

Szilagyi PG, Humiston SG, Gallivan S, Albertin C, Sandler M, Blumkin A. Effectiveness of a citywide patient immunization navigator program on improving adolescent immunizations and preventive care visit rates. Archives of Pediatrics & Adolescent Medicine 2011;165(6):547-53.

Szilagyi PG, Rodewald LE, Humiston SG, et al. Effect of 2 urban emergency department immunization programs on childhood immunization rates. Archives of Pediatrics and Adolescent Medicine 1997;151(10):999-1006.

Thomas P, Joseph TL, Menzies RI. Evaluation of a targeted immunisation program for Aboriginal and Torres Strait Islander infants in an urban setting. New South Wales Public Health Bulletin 2008(5-6):96-9.

Tierney WM, Hui SL, McDonald CJ. Delayed feedback of physician performance versus immediate reminders to perform preventive care. Effects on physician compliance. Med Care 1986;24(8):659-66.

Trick WE, Das K, Gerard MN, et al. Clinical trial of standing-orders strategies to increase the inpatient influenza vaccination rate. Infection Control and Hospital Epidemiology 2009;30(1):86-8.

Turner BJ, Day SC, Borenstein B. A controlled trial to improve delivery of preventive care: physician or patient reminders? J Gen Intern Med 1989;4(5):403-9.

Turner RC, Waivers LE, O'Brien K. The effect of patient-carried reminder cards on the performance of health maintenance measures. Arch Intern Med 1990;150(3):645-7.

Warner EA, Seleznick MJ. Using medical record reminders to improve pneumococcal vaccination rates. Joint Commission Journal on Quality & Patient Safety 2004;30(6):331-4.

Zimmerman RK, Hoberman A, Nowalk MP, et al. Improving influenza vaccination rates of high-risk inner-city children over 2 intervention years. Annals of Family Medicine 2006;4(6):534-40.

Economic Review

Ahmed F, Elbasha EE, Thompson BL, Harris JR, Sneller VP. Cost-benefit analysis of a new HEDIS performance measure for pneumococcal vaccination. Med Decis Making 2002;22(Suppl 1):s58-s66. http://dx.doi.org/10.1177/027298902237711.

Frame PS, Zimmer JG, Werth PL, Hall WJ, Eberly SW. Computer-based vs manual health maintenance tracking: a controlled trial. Arch Fam Med 1994;3(7):581. http://dx.doi.org/10.1001/archfami.3.7.581.

Hambidge SJ, Phibbs SL, Chandramouli V, Fairclough D, Steiner JF. A stepped intervention increases well-child care and immunization rates in a disadvantaged population. Pediatrics 2009;124(2):455. http://dx.doi.org/10.1542/peds.2008-0446.

Healy CM, Rench MA, Baker CJ. Implementation of cocooning against pertussis in a high-risk population. Clin Infect Dis 2011;52(2):157. http://dx.doi.org/10.1093/cid/ciq001.

Poirier B, De Wals P, Petit G, Erickson LJ, Pépin J. Cost-effectiveness of a 3-dose pneumococcal conjugate vaccine program in the province of Quebec, Canada. Vaccine 2009;27(50):7105-9. http://dx.doi.org/10.1016/j.vaccine.2009.09.057.

Westman S, Halbert RJ, Walton LG, Henneman CE. A clinic without walls: the Los Angeles Immunization Demonstration Project. Am J Public Health 1997;87(2):293-4.

Hong K, Leidner AJ, Tsai Y, Tang Z, Cho B, Stokley S. Costs of interventions to increase vaccination coverage among children in the United States: a systematic review . Academic Pediatrics 2021;21(4):S67-77.

Effectiveness Review

The CPSTF findings are based on studies included in the original review (search period 1980-1997) combined with studies identified in the updated search (search period 1997- February 2012). Reference lists of articles reviewed as well as lists in review articles were also searched, and members of our coordination team were consulted for additional references.

Details of the original search (1980-1997)

The following five electronic databases were searched during the original review period of 1980 up to 1997: MEDLINE, Embase, Psychlit, CAB Health, and Sociological Abstracts. The team also reviewed reference lists in articles and consulted with immunization experts. To be included in the review, a study had to:

• have a publication date of 1980–1997;
• address universally recommended adult, adolescent, or childhood vaccinations;
• be a primary study rather than, for example, a guideline or review;
• take place in an industrialized country or countries;
• be written in English;
• meet the evidence review and Guide chapter development team’s definition of the interventions; provide information on one or more outcomes related to the analytic frameworks; and
• compare a group of persons who had been exposed to the intervention with a group who had not been exposed or who had been less exposed. In addition, we excluded studies with least suitable designs for two interventions (provider reminder/recall and client reminder/recall) where the literature was most extensive.

Details of the update search (1997- February 2012)

The team conducted a broad literature search to identify studies assessing the effectiveness of Vaccine Preventable Disease interventions in improving vaccination rates. The following nine databases were searched during the period of 1997 up to February 2012: CABI, CINAHL, The Cochrane Library, EMBASE, ERIC, MEDLINE, PSYCHINFO, Soci Abs and WOS. Reference lists of articles reviewed as well as lists in review articles were also searched, and subject matter experts consulted for additional references. To be included in the updated review, a study had to:

• have a publication date of 1997- February 2012;
• evaluate vaccinations with universal recommendations;
• meet the evidence review and Guide chapter development team’s definition of the interventions;
• be a primary research study with one or more outcomes related to the analytic frameworks;
• take place in an high income country or countries;
• be written in English
• compare a group of persons who had been exposed to the intervention with a group who had not been exposed or who had been less exposed. In addition, we excluded studies with least suitable designs for two interventions (provider reminder/recall and client reminder/recall) where the literature was most extensive

Search Terms

1. Immunization
2. Vaccination
3. Immunization Programs

Economic Review

The present review included studies that reported economic outcomes from the 2000 review (search period 1980-1997) combined with studies identified from updated searches (search period 1997- February 2012) within the standard medical and health-related research databases, Google Scholar, and databases specialized to economics and social sciences. The details of the two sets of searches are provided below.

Details of the Updated Search (1997- February 2012)

The team conducted a broad literature search to identify studies assessing interventions to improve vaccination rates. The following nine databases were searched during the period of 1997 up to February 2012: CABI, CINAHL, The Cochrane Library, EMBASE, ERIC, MEDLINE, PSYCHINFO, Soci Abs and WOS. In addition, Google Scholar and specialized databases (CRD-University of York: NHS EED, EconLit, and JSTOR), were also searched. Reference lists of articles reviewed as well as lists in review articles were also considered, and subject matter experts consulted for additional references.

Search Terms

• Immunization
• Vaccination
• Immunization Programs

To be included in the updated review, a study had to do the following:

• Have a publication date of 1997- February 2012
• Evaluate vaccinations with universal recommendations
• Meet the evidence review and Community Guide review team's definition of the interventions
• Be a primary research study with one or more outcomes related to the analytic framework(s)
• Take place in a high income country or countries
• Be written in English; and
• Compare a group of persons who had been exposed to the intervention with a group who had not been exposed or who had been less exposed.

Details of the Original Search (1980-1997)

The following five electronic databases were searched during the original review period of 1980 up to 1997: MEDLINE, Embase, Psychlit, CAB Health, and Sociological Abstracts. The team also reviewed reference lists in articles and consulted with immunization experts. To be included, a study had to do the following:

• Have a publication date of 1980–1997
• Address universally recommended adult, adolescent, or childhood vaccinations
• Be a primary study rather than, for example, a guideline or review
• Take place in an industrialized country or countries
• Be written in English
• Meet the definition of the interventions
• Provide information on one or more outcomes related to the analytic frameworks; and
• Compare a group of persons who had been exposed to the intervention with a group who had not been exposed or who had been less exposed. In addition, we excluded studies with least suitable designs for two interventions (provider reminder/recall and client reminder/recall) where the literature was most extensive.