Home   |   About   |   Contact

   
Framework Development

At the request of the CDC Division of Nutrition, Physical Activity and Obesity, the Center TRT investigators and staff developed a framework to guide practitioners’ efforts to evaluate obesity prevention interventions, with a particular focus on interventions that target change at the policy and environment level. Policy and environmental interventions are challenging to evaluate because the process of planning and implementing them is multistep, cyclical, incremental, and influenced by many factors outside practitioners’ control.

The Center TRT Evaluation Framework draws on a range of theories and models related to policymaking and evaluation in public health (Table 1) and was developed with input from policy and evaluation experts at both the University of North Carolina and the Centers for Disease Control and Prevention.  The Center TRT identified three principles to guide evaluation. Evaluations will (1) address the entire intervention/policy planning process over time; (2) be guided by a program theory that takes a systems approach to addressing interactions across contexts, processes, outputs, and outcomes, and (3) not address the entire process at once, but rather focus evaluation on factors relevant to the current stage of the planning process (Olgilvie, 2011).

The Center TRT’s Evaluation Framework is organized around the four standard sections of a logic model: inputs, activities, outputs, and outcomes, but provides the flexibility for the logic model to evolve as a project progresses. The framework provides a comprehensive overview of the entire intervention process progressing from engaging stakeholders, to formulating a proposal and then to enacting, implementing, and maintaining an intervention or policy in practice. The expectation is that practitioners will carve out and adapt those components that are relevant to the current stage of their specific intervention to develop a logic model reflective of their intervention. The Center TRT Evaluation Framework logic model can be used as the basis for an evaluation plan that includes process and outcome evaluation questions that are linked to the logic model, potential data sources, and data collection methods.

The Center TRT Evaluation Framework logic model and evaluation plan were reviewed by stakeholders, including state-level obesity prevention program evaluators, and refined by Center TRT investigators and staff.  A complete description of the framework is in press [Leeman, J., Sommers, J., Vu, M., Jernigan, J., Payne, G., Thompson, D., Heiser. C., Farris, R., & Ammerman, A. (In press). An evaluation framework for obesity prevention policies. Preventing Chronic Disease.]

Table 1. Overview of frameworks and theories used in Center TRT Evaluation Framework

Framework or Theory

Central Constructs

CDC evaluation framework  (CDC, 1999)

Steps in an evaluation:

  • Engage stakeholders
  • Describe the policy initiative
  • Focus the evaluation
  • Gather credible evidence
  • Justify conclusions
  • Disseminate & utilize findings

Phases of evaluation:

  • Formative
  • Process
  • Outcome

 

Process of setting the policy agenda – creating a window 
of opportunity (Kingdon, 2003)

Components of process:

  • Problem
  • Solutions to problems
  • Politics

 

Process of making public 
policy in the US (Longest, 2006)

Steps in policy making:

  • Formulation
  • Enactment
  • Implementation
  • Modification

Sources of input and feedback:

  • Organizations
  • Interest groups
  • Socio-political environment

 

RE-AIM framework (Glasgow et al., 2003)

Public health impact criteria:

  • Reach
  • Effectiveness
  • Adoption
  • Implementation
  • Maintenance

 

Conceptual framework for environmental and policy strategies (Brennan et al., 2011)

Types of environments:

  • Physical
  • Economic
  • Social
  • Communication

 

Criteria for evaluating 
healthcare systems (Aday et al., 2004)

Criteria:

  • Effective
  • Equitable
  • Cost effective

References

Leeman, J., Sommers, J., Vu, M., Jernigan, J., Payne, G., Thompson, D., Heiser, C., Farris, R, & Ammerman, A. (2012). An evaluation framework for obesity prevention policies. Preventing Chronic Disease, 9, E120.

Aday LA, Begley C, Lairson D, Balkrishnan R. Evaluating the healthcare system. Chicago: Health Administration Press; 2004.

Brennan L, Castro S, Brownson RC, Claus J, Orleans CT. Accelerating evidence reviews and broadening evidence standards to identify effective, promising, and emerging policy and environmental strategies for childhood obesity prevention. Annu Rev Public Health 2011; 32:199-223.

Centers for Disease Control and Prevention. (1999). Framework for program evaluation in public health. MMWR 1999;48(RR-11):1-40.

Glasgow RE, Lichtenstein E, Marcus AC. Why don't we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. Am J Public Health 2003;93(8):1261-1267.

Kingdon JW. Agendas, alternatives, and public policies. New York: Longman; 2003.

Longest B B. Health policymaking in the United States. Chicago: Health Administration Press; 2006.

Ogilvie D, Cummins S, Petticrew M, White M, Jones A, Wheeler K. Assessing the evaluability of complex public health interventions: five questions for researchers, funders, and policymakers. Milbank Q 2011;89(2):206-25.