Module 4: Identifying and Selecting Appropriate Evidence-Based Interventions

Module Introduction

At this point in the process, the planning group has considered relevant data, defined the problem, and created a shared vision and goals for how to address the problem. The next step is to select interventions that can address the problem most effectively with available resources.

Module 4 discusses key concepts that will guide your coalition through the process of identifying and selecting appropriate evidence-based interventions to be implemented as part of your state plan. To follow the gardening analogy, this is the point at which you would identify and select seeds that will grow well in your garden, given the conditions in your area.

Module 4 Learning Objectives:

Upon completion of this module, you should be able to:

• Explain the difference between “research-tested evidence” and “practice-based evidence.”
• List key resources to identify evidence-based interventions.
• Explain how to match evidence-based interventions to the target population.
• Describe the factors to consider when matching evidence-based interventions to partners’ capacity.
• Explain how to adapt an intervention to specific populations.
• Describe how public health theories are used to develop interventions.

Time estimate for completion:

It should take approximately 75 minutes to complete this module.

Section I: Evidence-Based Interventions

Evidence-Based Interventions

The next step in the state planning process is to identify and select appropriate interventions, which are backed by evidence demonstrating their effectiveness at addressing the problem at hand.

Evidence is information that indicates whether a belief or perception is true or valid.

Historically, the most highly valued evidence for the effectiveness of interventions has been based on randomized controlled clinical trials. While appropriate for determining the biological causes of disease or the effectiveness of a medication or vaccine, these types of studies are difficult to conduct outside of the medical clinic or laboratory.

As public health has moved toward exploring chronic diseases with complex multi-causal pathways and interventions that focus on policy, environment, systems, and behavior change, research approaches have expanded to include quasi-experimental designs, mixed methods, natural experiments, and repeated implementations over time across multiple sites and multiple populations.

Section I: Evidence-Based Interventions cont'd

The Knowledge to Action Framework

The Knowledge to Action Framework demonstrates that there is a two-way flow of knowledge between researchers and practitioners. Evidence to support interventions is not generated exclusively by researchers and passively received by practitioners.

As research findings on effective interventions are disseminated, public health practitioners have launched their own implementations.  Adaptations to interventions and the results of repeated implementations with new target populations provide additional evidence for the effectiveness of interventions.  Dialogue between practitioners and researchers about these interventions creates a two-way flow of information.

To help demonstrate how this process works, the Centers for Disease Control and Prevention (CDC) developed the “Knowledge to Action Framework.” This framework graphically portrays the process by which evidence obtained through research (left side of diagram, in blue), is then translated into interventions through translation (middle of diagram, in red), and finally adopted in a widespread and sustainable manner through institutionalization (right side of diagram, in green). This diagram also shows how evidence gained through practice is fed back to stimulate additional research and the modification of interventions.

Knowledge to Action Framework

Download a PDF of the Knowledge to Action Framework with more information on each phase.

Section I: Evidence-Based Interventions cont'd

Knowledge to Action Framework: Research Phase

There are three main components of the Research Phase of the Knowledge to Action Framework, shown in blue in the diagram below.

Hover your mouse over each blue box for more information on the Research Phase.

Section I: Evidence-Based Interventions cont'd

Knowledge to Action Framework: Translation Phase

After successful Effectiveness and Implementation Studies have occurred, if the intervention is deemed to be effective and worth translating into a widely-disseminated format, it will move into the Translation and Institutionalization Phases.

During Translation, research studies are turned into easy-to-replicate interventions suitable for a wide range of potential implementers and target audiences. There are five main components of the Translation Phase of the Knowledge to Action Framework, shown in red in the diagram below.

Hover your mouse over each Red box for more information on the Translation Phase.

Section I: Evidence-Based Interventions cont'd

Knowledge to Action Framework: Institutionalization Phase and Evaluation

There is one main component to the Institutionalization Phase, shown in green. Evaluation, in yellow, is critical to the entire process.

Hover your mouse over each Green box for more information on the Institutionalization Phase, and Yellow box for more information on Evaluation.

Section I: Evidence-Based Interventions cont'd

Research-Tested and Practice-Based Evidence

What about “Best Practices”?
“Best practice” is a term that is often used to describe interventions that have been tested in several different settings and have varying degrees of evidence behind them. Because it is an imprecise term, this course uses Research-Tested and Practice-Based Evidence instead.

As demonstrated through the Knowledge to Action Framework, both research and practice are critical in building up the body of evidence in support of an intervention. The evidence gathered through these two fields can be described as “research-tested” and “practice-based” evidence:

Research-Tested Evidence refers to evidence gathered through research studies or a series of studies. These results have typically been presented at a scientific meeting or have been reported in a peer-reviewed journal.

Practice-Based Evidence refers to evidence gathered about interventions that have been developed, implemented, and evaluated in practice but have not been tested in a research study. Once these “promising practices” are subjected to more rigorous research and/or program evaluation, they then become “research-tested.”

Section I: Evidence-Based Interventions cont'd

Theory and Evidence-Based Interventions

Thought Provoker: Before clicking the link below, name an example of a theory that is commonly used in public health.

Evidence-based interventions are based in theory. A theory is a set of concepts that present a systematic view of how the relationships between particular factors produce certain results. A theory is applicable to all similar events or situations. A theory is also testable, in that new data collected should confirm or refute the theory. Theories give researchers and practitioners the concepts they need to examine health problems and to design and test interventions.

Theories:

• Guide research on why people do or do not practice certain health behaviors.
• Steer the gathering of information for public health program development.
• Suggest which program strategies best reach and impact target audiences.
• Help identify indicators to monitor and measure during program evaluation.

Click here for a brief overview of theories commonly used in health education and community health promotion, which you may encounter when researching evidence-based interventions.

Section I: Evidence-Based Interventions cont'd

Socio-Ecological Model and Evidence-Based Interventions

Since no single theory alone explains how the entire range of individual, societal, and environmental factors affect health, public health researchers and practitioners frequently use models. Models are frameworks that draw on multiple theories to explain complex problems affected by many levels of factors. The model used most frequently in chronic disease prevention and control is the Socio-Ecological Model (as described in Module 1.)  The model emphasizes the interaction between and the interdependence of factors across multiple levels.

The Socio-Ecological Model brings together theories that explain health and disease as the result of individual behavior with theories that explain health and disease as the result of social forces. To be the most effective, interventions should be planned that influence multiple levels of the Socio-Ecological Model.

Each level within the Socio-Ecological Model reflects the work of different social and behavioral disciplines, such as political science, sociology, anthropology, communications, organizational dynamics, and social psychology.  Public health researchers and practitioners have been working to demonstrate how coordinated interventions at multiple levels have stronger, more long-lasting effects on health status of populations using inter-disciplinary concepts and methods.  Advanced analytical approaches such as systems dynamic modeling have been developed to capture this complexity and predict how changes at different levels affect each other and produce changes in health outcomes.

Section I: Evidence-Based Interventions cont'd

Section I Summary & Activity

Section I introduced evidence-based interventions and described how evidence gathered from both research and practice is used to inform them. This section also briefly explained the importance of theory and the Socio-Ecological Model in guiding evidence-based interventions.

Now, think about how these concepts apply to your state planning process. Watch the video below, in which the Connecticut DPCP Coordinator discusses evidence-based interventions, and then answer the questions that follow.

Click here for a worksheet to record your answers.

1. What are three benefits of including evidence-based interventions in the state plan?
2. How can the evidence in support of interventions be used to make planning decisions with partners?

Section II: Sources to Identify Evidence-Based Interventions

Sources to Identify Evidence-Based Interventions

There are several reliable sources that planning group members can use to identify evidence-based interventions:

• Systematic reviews
• Recommendations from the CDC
• Academic centers and national partners
• Sources within the health promotion and health education literature

Section II: Sources to Identify Evidence-Based Interventions cont'd

Systematic Reviews and Practice Guidelines

A systematic review is a review of published literature on a specific topic that uses a formal, unbiased process to identify relevant studies, assess the quality of these studies, and summarize the evidence presented in them. Systematic reviews may be done by individuals or by groups of experts working together as an expert panel. In some cases, an expert panel translates the results of the review process into recommendations for action regarding policy and practice guidelines. The group arrives at recommendations and conclusions, which are then documented along with their rationale in a widely disseminated panel report.

Section II: Sources to Identify Evidence-Based Interventions cont'd

Systematic Review and Practice Guidelines Resources

A finding of “insufficient evidence” does not mean that an intervention has been proven to be ineffective, but that there is not enough existing evidence for the reviewers to determine whether it was effective or not.  Recommendations for or against these interventions can only come after further research is completed, analyzed, and published in the peer-reviewed literature.

Both the Guide to Clinical Preventive Services for Interventions in Clinical Settings and the Guide to Community Preventive Services (also known as The Community Guide) utilize systematic reviews to provide guidelines and recommendations about the effectiveness of tested interventions. Click here for more information about these critical resources.

While the Guides are very useful tools, they do come with some limitations. Not all health topics are included, because some have more published literature available than others. There is also lag time between when studies are conducted, published, and selected for systematic review, so the most up-to-date information may not be included. Because of these limitations, the recommendations in the Guides should be supplemented with other expert recommendations. 

Section II: Sources to Identify Evidence-Based Interventions cont'd

NCCDPHP Recommendations

The National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) at the CDC and its various divisions have issued their own recommendations for evidence-based interventions.

For example, the Division of Diabetes Translation has identified three evidence-based, core interventions that have the greatest potential to achieve diabetes prevention and control program goals. These core interventions are:

1. Improve quality of clinical care for populations with greatest diabetes burden and risk to improve control of A1C, blood pressure, and cholesterol, and to promote tobacco cessation.
2. Increase access to sustainable self-management education and support services for populations with greatest diabetes burden and risk to improve control of A1C, blood pressure, and cholesterol, and to promote tobacco cessation.
3. Increase use of lifestyle change programs that have achieved CDC recognition (or pending recognition) to prevent or delay onset of type 2 diabetes among people at high risk.

Section II: Sources to Identify Evidence-Based Interventions cont'd

Prevention Research Centers and Applied Projects

NCCDPHP supports several programs focused on developing interventions at research universities, state health departments, and non-governmental organizations. These programs, including Prevention Research Centers (PRCs) and Racial and Ethnic Approaches to Community Health (REACH) programs, are important resources to use when identifying evidence-based interventions.

Click here to read more about these NCCDPHP projects and resources.

Section II: Sources to Identify Evidence-Based Interventions cont'd

CDC Partners

NCCDPHP has worked with many national-level organizations to develop interventions and to expand expertise on implementing and evaluating interventions through distance learning, conferences, and professional meetings. Some of these organizations include:

• American Alliance for Health, Physical Education, Recreation and Dance (AAHPERD)
• American Association of Diabetes Educators (AADE)
• American Cancer Society (ACS)
• American Diabetes Association (ADA)
• American Heart Association (AHA)
• American Public Health Association (APHA)
• Arthritis Foundation (AF)
• Association of Schools of Public Health (ASPH)
• Association of State and Territorial Health Officials (ASTHO)
• National Association of Chronic Disease Directors (NACDD) and its various councils, including the Diabetes Council
• Society for Public Health Education (SOPHE)

These organizations’ websites or publications can be a good source of information about interventions.

Section II: Sources to Identify Evidence-Based Interventions cont'd

Other Sources of Information

Academic Institutions
Colleges and universities within your state can also be useful resources, especially schools of public health and medicine. Faculty at these institutions may provide expert consultation on selecting, adapting, implementing, and evaluating evidence-based interventions.

Thought Provoker: What types of resources and experience can your partners contribute to the process of selecting interventions?

Published Literature
Key literature sources that publish peer-reviewed articles describing public health interventions include the publications of the Society for Public Health Education (SOPHE), Health Education and Behavior and Health Promotion Practice, and the on-line journal produced by NCCDPHP, Preventing Chronic Disease. The fields of nutrition, physical activity, diabetes education, and policy have journals that may contain relevant literature.

Your Partners
Coalition members representing a variety of professional fields are good resources to include in the intervention identification process.

Section II: Sources to Identify Evidence-Based Interventions cont'd

Reviewing Previous or Ongoing Interventions

It is also important to review previous and existing interventions implemented through state and local health departments, as well as by partners represented on state and community coalitions. The planning group members should develop their own set of criteria to review these interventions and describe a standard process of documenting these interventions and the assessments made. 

Compare these interventions to what is recommended by the Community Guide, the research and programs developed by Prevention Research Centers, and guidance received from the CDC and other national institutions. Investigate what process and outcome data may have been collected and analyzed during implementation of these interventions – are these good examples of “practice-based evidence” even though they have not been reported in the scientific literature? 

As you review previous and existing interventions, some criteria to consider are:

• Do they closely resemble research-tested interventions? 
• Do they demonstrate some measurable outcomes?
• Are they cost-effective? 
• Can they be widely disseminated to many different partners?
• Are they culturally and linguistically appropriate?

While the decision may be reached to discontinue some previous and existing interventions, others may prove to be worth continuing as is or with some adaptations to make them conform more closely to the recommendations discussed.

Section II: Sources to Identify Evidence-Based Interventions cont'd

Section II Summary & Activity

Section II described several sources of information regarding evidence-based interventions.

Now, think about how your planning group can utilize these sources in order to identify interventions that are suitable for your state. Watch the video below, in which the Connecticut and Minnesota DPCP Coordinators discuss sources of information regarding evidence-based interventions, and then answer the questions that follow. Click here for a worksheet to record your answers.

1. What are two evidence-based interventions for diabetes prevention and control mentioned in the video?
2. What evidence-based interventions are currently being implemented in your state?

Section III: Selecting and Adapting Interventions

Criteria for Selecting Interventions

With all of the sources of information about evidence-based interventions that are available, how can your planning group narrow down and select interventions that will be most effective?

The information that you have already gathered about the problem, what causes and contributes to the problem, and what resources are available to you will inform your intervention selection. The overall compatibility between an intervention, the target population, and the implementing organization’s capacity to implement the intervention is referred to as “fit.”

Ultimately, the process of selecting interventions that “fit” comes down to balancing two criteria: 

1. The scope and nature of the problem, as defined by the epidemiological data describing the burden of the disease. From the data you have collected, you will have identified and prioritized chronic disease-related problems and target populations in which you want to make a positive change. With this information, you can find interventions that effectively address similar problems in similar groups. You can also assess which interventions are appropriate to implement as is, which could work with adaptations, and which are not suitable for the populations at all.
2. The capacity to implement interventions by coalition members, including the state and local health departments. Once you have determined which interventions could work for your problems and populations, you must also assess whether or not you and your partners have the capacity to implement the interventions as described in the literature.  In some cases, you and your partners may have to engage in some capacity-building activities before you can implement a chosen intervention successfully.

This section discusses the considerations that should be taken into account as your coalition develops criteria to select interventions, to ensure that the selected interventions fit your state’s circumstances.

Section III: Selecting and Adapting Interventions cont'd

Consider: The Target Population

The data that you have collected show the socioeconomic, demographic, and geographic characteristics of the populations with the highest rates of diabetes or chronic disease within your state. Data can illustrate which racial, ethnic, socioeconomic, and geographic subgroups are disparately affected by the problem.

When your planning group is examining interventions, you will need criteria to judge whether or not these interventions have been implemented with people who are similar to those in your target population. The target population for the intervention under consideration should be similar to yours in terms of:

You also will need criteria to judge whether or not the interventions you are considering have been implemented with people who have similar circumstances and available resources in the community (e.g. health care, self-management support, transportation, access to technology). You will need to consider the causes of the problem and the factors that contribute to the progression of disease in your target population. Interventions that have been implemented in similar circumstances and that have addressed similar contributing factors or missing resources will be a better fit than those that have not.

Section III: Selecting and Adapting Interventions cont'd

Consider: The Setting

When examining interventions, you also will need criteria to judge whether or not these interventions have been implemented in settings that are similar to the settings available in your state. The setting specified in the evidence-based intervention should match the settings available to you as much as possible. Some examples of intervention settings include:

• Worksites
• Health care clinics
• Schools
• Senior centers
• Faith-based organizations
• Group housing
• The community-at-large

Altering too much of the original intervention affects how well your intervention can achieve desired outcomes, but you may be able to transfer an intervention developed for one setting to a different setting if the age, health literacy, and socioeconomic status of the target populations are similar.

Section III: Selecting and Adapting Interventions cont'd

Consider: Reach

One of the most important characteristics to be considered when selecting interventions is their reach, or the extent to which the intended target population for the intervention is exposed to it. To estimate reach, compare the total number of persons that can receive or participate in an intervention to the total number of persons at risk for or already diagnosed with diabetes or other chronic diseases of interest in that particular group or community:

Calculating this will give you an idea of what proportion of the population will be affected by the intervention. Interventions with greater reach are preferable, as they are more likely to reach a large enough proportion of the target population to create real change.

Section III: Selecting and Adapting Interventions cont'd

Consider: Scalability

Closely related to reach is scalability, which refers to how easily an intervention can be widely or frequently implemented in order to gain a significant reach. Interventions that are easily implemented often, or that can be implemented for a large segment of the target population, are more scalable and will result in a greater reach. When thinking about scalability of potential interventions, the planning group should consider:

• How widely and frequently the intervention can be implemented, given existing and future capacity
• How often the intervention needs to be repeated
• The number of locations in which the intervention will need to be implemented
• The cost to expand or repeat the intervention

Some interventions are extremely effective but also are very expensive to implement, making wide scale implementation unlikely. 

Section III: Selecting and Adapting Interventions cont'd

Consider: Sustainability

Policy, environmental, and systems change interventions inherently have a broad reach, because they can affect large populations at once, and they typically can be sustained over time with fewer resources. Communications and education interventions are important when first making environmental, policy, and systems changes, in order to increase awareness, change perceptions, and motivate the target population to take action.

A final issue to consider is sustainability, which refers to the ongoing availability of resources required to implement or expand selected interventions.

Interventions can be more sustainable if they are built into the systems or ongoing programming of a partner. For example, professional education on chronic disease can become part of the ongoing curriculum offered at an Area Health Education Center, an entity whose function it is to provide continuing education to health care professionals.

Section III: Selecting and Adapting Interventions cont'd

Using the Socio-Ecological Model to Create Linked Sets of Interventions

While it may not be possible to always develop a linked bundle, aim for interventions that affect at least two layers of the Socio-Ecological Model.

When identifying interventions, the planning group can think in terms of building a linked “bundle” or set of interventions that address multiple layers of the Socio-Ecological Model. Sets of interventions that work together across multiple levels of the model can produce stronger and longer-lasting effects than single interventions that address one level.

Interventions can be bundled that focus on the same health issue at different levels. For example, a change in policy can be linked to a communications campaign that educates members of the target population about the change. The different interventions can be implemented at the same time or sequentially, and some elements may need to be repeated on a regular basis in order to reach everyone in the target population.

Section III: Selecting and Adapting Interventions cont'd

Assessing Your Capacity to Implement Interventions

An intervention must be matched to the ability and intent of the organization or agency that will be implementing it as much as it needs to be matched to the target audience that will be participating in it.  Program “fit” with an organization is tied to a number of considerations, such as:

• Interventions must align with the implementing organization’s mission, priorities, values, and ongoing activities.
• The organization must have the staff and the capacity to implement the intervention, as well as support from leadership, staff, and the community.
• The organization must be committed to sustainability or long-term implementation of the intervention.
• The organization must have access to the target population and the appropriate facilities, technology, funding, and materials to support implementation.
• The organization also must commit to collecting and reporting data on the intervention and participants for monitoring and evaluation purposes.

It is very important that the organization and agency be fully informed of what will be required to successfully implement the program.

When working with a set of interventions using the Socio-Ecological Model, a group of organizations may need to work together to implement the interventions at multiple levels.  All organizations should be assessed for their capacity to implement.

Section III: Selecting and Adapting Interventions cont'd

Fidelity of Implementation

Nearly every intervention is changed slightly to fit the target population and the circumstances surrounding implementation. The important challenge is to determine how much you can change and still have an evidence-based intervention.

Controlling the process of adaptation helps make sure that the program is implemented with fidelity.  Fidelity means the intervention is implemented in form and in delivery as closely or as faithfully as possible to the way the program was intended to be delivered.  The components of fidelity include adherence to:

1. The program curriculum, protocol or implementation guide
2. The dose or amount of program delivered (length and number of sessions)
3. The quality of program delivery
4. Participant reaction and acceptance

To ensure fidelity of implementation when adapting an intervention, you must keep the key processes and core elements intact. The key processes are the required steps that are conducted during implementation that contribute to the intervention’s effectiveness, taken from the intervention’s methods section or implementation protocol. The core elements are the required components that represent the theory and internal logic of the intervention and are the most likely to produce the intervention’s effectiveness.

Click here to learn more about core elements.

Section III: Selecting and Adapting Interventions cont'd

Methods of Ensuring Fidelity

Thought Provoker: What does your state do to ensure fidelity of implementation of your current interventions?

The core elements of the original intervention must remain the same in order to achieve results close to those achieved by the original developers of that intervention. There are several ways to make sure this happens.

First, prior to implementation, there should be a “train-the-trainer” phase, in which individuals selected to implement the intervention undergo a consistent training emphasizing that the intervention should remain as close to the original as possible. Second, all materials needed to implement the program should be provided, including standardized curriculum guides, implementation protocols, handouts, equipment or supplies, and reference materials. Finally, continuing to monitor and assess implementers’ performance over time will help to maintain the quality and fidelity of the implementation, as well as to capture any needed enhancements for successive implementations.

Section III: Selecting and Adapting Interventions cont'd

Adapting Interventions: What Can and Cannot Be Changed

There are two main ways to identify what can and cannot be modified when adapting an intervention.

1. Explore the Literature - Read about the intervention in the literature and review supporting materials such as training manuals, brochures, factsheets, handouts, videos, etc. Contact the curriculum developers about differences between your intended audience and implementing organization and the ones involved in the development of the intervention. Also, ask the developers what they consider to be the core elements that should not be changed.
2. Compare Proposed Changes to Commonly Acceptable Changes - There are some aspects of interventions that can generally be altered without issue and others that could impact effectiveness. Click here to view a chart that provides some basic information about what is generally acceptable to change, what requires more caution and care to modify, and what should never be altered.

Section III: Selecting and Adapting Interventions cont'd

Adapting Interventions for Specific Audiences

There also are general considerations regarding the target audience that should always prompt discussion about the need to adapt selected interventions. These factors include:

1. Cultural and Linguistic Appropriateness: Culture, ethnicity, and race, and degree of acculturation are important factors to think about when examining whether an intervention will be appropriate for your target population, or if it needs to be modified to be appropriate for culture and language.
2. Reading and Health Literacy Levels: Written materials and verbal communications with participants must be appropriate to their reading and health literacy levels so that the information is being communicated in a way that is easy for the target audience to understand, process, and use.
3. Sensory, Mobility, or Cognitive Impairments: When working with populations with sensory, mobility, or cognitive impairments, it is important to involve members of these groups into the planning process of selecting and adapting interventions.

Click here to download a fact sheet that describes these factors in more detail.

Section III: Selecting and Adapting Interventions cont'd

Section III Summary & Activity

Section III described the factors that must be considered when deciding what interventions to implement as part of your state plan. When selecting interventions consider:

• their appropriateness for the target population and the circumstances contributing to the problem,
• their reach, scalability, and sustainability, as well as their impact across levels of the Socio-Ecological Model,
• if you and your partner organizations have the capacity to implement the interventions; and
• whether adaptations are required for the target audience, and if those adaptations alter the core components or effectiveness of the interventions.

Now, think about how these concepts apply to your state planning process. Watch the video below, in which state health department staff from Connecticut and Minnesota discuss the process of selecting evidence-based interventions, and then answer the questions that follow.  Click here for a worksheet to record your answers.

1. What are three things the Coordinators in the video consider when determining whether an evidence-based intervention ‘fits’ their program?
2. How could collaborating across chronic disease programs increase the capacity of you and your partners to implement a particular intervention? 

Summary

Module 4 Summary

Module 4 described key concepts to guide the process of identifying and selecting the evidence-based interventions to include in your state plan.

You can download a PDF of helpful resources for more information about this topic.

Your next steps in the planning process are to create and implement your state plan.

Continue the course with Module 5: Implementation, Monitoring, and Evaluation.