0920-24FI In-Depth Assessment – Evaluability Assessment Learning C

Form Approved

OMB No. 0920-XXXX

Exp. Date xx/xx/20xx

In-Depth Assessment – Evaluability Assessment Learning Collaborative Interview Guide – The National Cardiovascular Health Program & The Innovative Cardiovascular Health Program

Note: Public reporting burden of this collection of information is estimated to average 60 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to CDC/ATSDR Reports Clearance Officer; 1600 Clifton Road NE, MS D-74, Atlanta, Georgia 30333; ATTN: PRA (0920-24XXX)

Evaluability Assessment

Learning Collaborative Interview Guide

Date of Interview
Interviewer
Notetaker
Organization Name
Organization Type
State
Organization City		Zip Code
Cooperative Agreement	☐ The National CVH Program ☐ The Innovative CVH Program
Strategy (select all that apply)	☐Strategy 1: Track and Monitor Clinical Measures ☐Strategy 2: Implement Team Based Care ☐Strategy 3: Link Community Resources and Clinical Services
Interviewee Name(s)
Interviewee Role(s) or Title(s)

Introduction

Thank you for taking the time to participate in this interview. My name is <insert name> and I am with the Deloitte evaluation team. Our team is working with the CDC Division for Heart Disease and Stroke Prevention to evaluate the <insert Cooperative Agreement>. As part of the larger evaluation, we are conducting evaluability assessment interviews to provide detailed insight into how recipients are prioritizing populations of focus impacted by the high prevalence of cardiovascular disease through learning collaboratives. Specifically, we’re interested in understanding how learning collaboratives implement models for collaboration across different types of partners to serve populations and communities affected disproportionately by CVD. We also hope to learn about the function, structure, goals, and activities of your learning collaborative.

This interview is expected to take no longer than 60 minutes. Please answer questions based on your own knowledge and experience. Remember, you are the expert and that there are no right or wrong answers. If at any time during the interview you are not clear about what we are asking, be sure to let me know. Your participation in this interview is completely voluntary. You may choose not to respond to questions at any time and it will not in any way impact the funding or technical assistance your organization receives from CDC.

Steps will be taken to protect your privacy; no information that identifies you will be shared with anyone except our project staff. All information will be kept secure and any personally identifiable information will be removed when results are aggregated for analysis.

Do you consent to this interview?

☐ Yes

☐ No

With your permission, we would like to record this discussion for transcription purposes. 

Do we have your permission to record?

​​☐​ Yes 

​​☐​ No 

 

Do you have any questions or concerns before we start the discussion? 

Background

Thank you again for agreeing to participate in this interview. For reference, for today’s interview we will be talking about <Strategy X>, which is defined as

<Strategy description from NOFO >

We will discuss the following sub-strategies under <Strategy X>:

[Interviewer Note: Only describe the relevant sub-strategies for which the recipient organization has self-nominated]

<Sub-strategy description from NOFO >

First, we would like to learn a little about you and your organization.

1. Can you tell me about your organization and the populations that it serves, specifically for cardiovascular related diseases?

Probes:

• What types of services/programs does your organization offer to support individuals who have or at high risk for high blood pressure or high cholesterol?

• How long has your organization been offering these services/implementing these programs?

• Can you describe to me the different populations (i.e., race, ethnicity, socioeconomic status, age, genders, geography etc.) that your organization typically serves?

2. What is/are your role(s) and what are your specific responsibilities within your organization?

3. Can you tell me about your role in relation to the learning collaborative (LC) that supports the work of <insert name of cooperative agreement>?

Probe:

• Are you involved with or support another LC related to a DHDSP cooperative agreement?

LC Implementation

Next, we would like to discuss your learning collaborative’s approach to achieve optimal health for populations of focus. We’re interested in learning more about how your learning collaborative operates and what it hopes to achieve.

1. From your perspective as a <LC Lead, LC Convener, LC Coach, LC Member>, can you tell me a little bit about how the learning collaborative is governed and how they operate?

Probes:

• How often are members convened and in what mode (i.e., in-person, virtual, hybrid meetings)?

• How are LC governed?

• What is the process for decision-making?

• Is there a system or process to track progress and outcomes? How is this information monitored and reported?

• Do LC members have defined roles and responsibilities?

2. From the <EPMP, LC action plan, APR, workplans, etc.> the LC aims to <summary of goals>. Is that accurate? How will the LC achieve these goals?

3. Thinking about <name of strategy> in the <name of cooperative agreement>, can you describe:

   • Strategy 1: How does the learning collaborative increase the use of EHRs and HIT to detect and mitigate health care disparities?

   • Strategy 1: What is the learning collaborative’s role in promoting quality improvement?

   • Strategy 2: How does the learning collaborative work to expand care teams to include non-physician team members to identify patient’s social services and support needs?

     • Examples: Community health workers, social workers, patient navigators, pharmacists, and other members of the care team in community settings outside of health care facilities

   • Strategy 3: How does the learning collaborative create and enhance community-clinical links to respond to social services and support needs?

4. According to <EPMP, LC action plan, APR, workplans, etc.>, the LC identified <population> as its population of focus.? Is that correct? What types of communities are you prioritizing?

Probes:

• What demographics, patient characteristics, or geographies are you prioritizing?

5. How did you identify geographies/census tracts with your population of focus?

Probes:

• How is this defined?

• How do you use GIS or other tools to do so?

6. How does the LC work to meet the needs of the population of focus?

7. [If LC Lead/Convener] When was this learning collaborative established? How long have these organizations been working together?

Probe:

• What is the history of collaboration?

8. From the <EPMP, LC action plan, APR, workplans, etc.>, we learned that the LC includes <summary of membership composition>. Can you tell us more about the membership composition of the learning collaborative? Please describe things like how many partners are involved, what types of organizations and populations are represented.

Probes:

• How long have these partners worked with the population of focus. In what capacity?

• In what ways have these partner organizations support prevention, management, and treatment of CVD?

• What are the partner organizations’ experiences with:

  • Strategy 1: Using EHRs or HIT?

  • Strategy 2: Supporting team-based care to identify, manage, and treat patients at highest risk of CVD?

  • Strategy 3: Creating and enhancing community-clinical links?

9. [If LC Lead/Convener] How has membership changed since the start of <name of cooperative agreement>?

10. [If LC Lead/Convener] How are partners identified for the learning collaborative? How are LC members recruited?

11. [If LC Lead/Convener] What has worked well for you in securing these partnerships?  

12. [If LC Lead/Convener] What has been challenging in securing partnerships? 

13. Is the current membership composition sufficient to meet the goals of the learning collaborative?

Probes:

• Who is missing from the table?

• Are there plans to grow membership? How so?

14. [If LC Lead/Convener] & [If implementing both The National and Innovative CVH Programs] How are you coordinating across learning collaboratives to minimize duplication of efforts across NOFOs?

Now that we’ve learned more about the background of the learning collaborative, we would like to learn more about the extent of implementation, successes and challenges with implementation, and factors that may support or hinder activities.

15. Tell us more about the status of your learning collaborative. What progress have you made?

Probe:

• Tell us more about milestones, key successes, other achievements.

• What are some areas where you did not make as much progress as anticipated?

16. What are your future implementation plans?

Probes:

• How will your partnerships change or grow in the next few years?

• Please describe what the LC hopes to accomplish by Year 4 (September 2026 – September 2027).

17. How will LC activities lead to system or community-level reductions in health disparities?

18. Can you tell us about the contextual factors that support or hinder learning collaborative operations?

Probes:

• Describe external factors such as complementing or competing initiatives, additional funding sources, partnerships and collaborations, state policies, political/economic climate.

• Describe internal factors such as organizational policies, leadership buy-in, internal capacity, organizational culture.

19. What would you say are the strengths of your learning collaborative?

    • What are the benefits of participating in a learning collaborative?

    • What factors do you think helped the learning collaborative be successful?

20. [If LC Lead/Convener] What are the challenges you have experienced with managing and coordinating learning collaboratives?

• Are they expected to persist? Why?

• How are these challenges addressed?

21. [If LC Member] What are the challenges you have experienced with working with the learning collaborative?

• Are they expected to persist? Why?

• How are these challenges addressed?

22. What additional support or resources are needed to strengthen or improve the learning collaborative?

LC Evaluation

We would like to understand to what extent the learning collaborative activities have been or is currently being evaluated.

1. What is the LC’s plan to monitor implementation progress and evaluate its success?

[Interview Note: Only ask remaining questions if LC has evaluation plans and if the LC representative is involved with or can speak to data collection and evaluation efforts.]

2. Who is primarily responsible for tracking and reporting data related to the LC?

Probes:

• What is your role in data collection and monitoring and evaluation efforts?

• Are other members or organizations involved in data collection? Describe their roles and responsibilities.

3. What type of data do you collect or plan to collect?

Probes:

• What types of metrics or indicators do you use to measure progress and monitor implementation?

• How is success measured?

• What outcomes do you track? What outcomes do you expect to have by September 2026 – September 2027 (Y4)?

4. How do you collect and report the data needed for monitoring and evaluation?

Probes:

• What data collection tools or instruments are used to track data (paper, electronic)?

  • [If not already provided] Request to see and get copies of data, data collection tools, or evaluation reports.

• Describe the process used to collect data.

5. What type of data related to health equity are collected and tracked?  

Probes:

• What SDOH data are collected?

• How will health disparities be measured and defined?

• What sort of methods or tools are used to measure health equity outcomes?

6. What, if any, barriers have you encountered with data collection or monitoring and evaluation activities?

Probes:

• What challenges, if any, are there with collecting SDOH data?

• What barriers, if any, do you anticipate in data collection or reporting related to outcome or impact?

• What specific strategies have been used or will be used to overcome these barriers?

• What additional support or TA do you need from [name of recipient organization]? What about from the CDC?

7. What have you learned so far about your program from your monitoring and evaluation efforts?

Probes:

• What findings can you share about implementation progress?

• What outcomes can you report at this point?

• How are data being used to make improvements?

Closing

Lastly, what questions do you have for me? Is there anything else you’d like to share?

Thank you again for participating. This concludes our discussion about learning collaboratives. If you have any additional questions, please feel free to contact the Comprehensive Evaluation Team, hdsp_nofo_eval@cdc.gov.