0930-0381 PATH_Evaluation_OMB_Supporting Statement B (SSB) final

EVALUATION OF THE PROJECTS FOR ASSISTANCE IN TRANSITION FROM HOMELESSNESS (PATH)

SUPPORTING STATEMENT

2. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

Respondent Universe and Sampling Method

Described below are the respondents and sampling methods for each data collection tool included in this request to OMB for the PATH Evaluation.

Web Surveys

The target universe for the State PATH Contact (SPC) Web Survey is the SPCs or comparable staff for all PATH grantees (i.e., the 50 States, the District of Columbia, Puerto Rico, the U.S. Virgin Islands, Guam, American Samoa, and the Commonwealth of the Northern Mariana Islands). The total expected number of respondents is 56.

Site Visits

The sample for the site visits will be purposive, in that the approach for selecting PATH grantees for the site visits includes geographic spread, identifying grantees with a good mix of providers in rural and urban counties and a spread of funding levels, and visiting one to two minimum-allocation states (i.e., receiving $300,000 PATH funds annually). As much as possible, grantees that were visited for the previous PATH evaluation will not be included. For each of the grantees selected, one to two PATH providers will be selected for site visits. The SAMHSA Office of Evaluation (OE) staff will work with the SPC to identify the most appropriate providers. In selecting providers, the team will attempt to identify providers that vary in terms of populations served and service strategies used. Signed consent forms will be obtained from consumers and community stakeholders before the start of each discussion and focus group.

Below we describe the respondent targets for each of the site visit instruments.

1. PATH Grantee Leadership Discussion Guide: The target universe is up to five relevant grantee staff (as determined by the SPC) from each of the five PATH grantees that will receive site visits. The staff may include directors and supervisors of the PATH program or the SPC. The total expected number of attendees is 25.

2. State PATH Contact (SPC) Discussion Guide: The target universe is the SPCs from the five PATH grantees that will receive site visits. The total expected number of respondents is five.

3. PATH State and Provider Stakeholders Discussion Guide: The target universe is up to five state staff either from the grantee agency or other agencies and with stakeholders of the PATH program. The staff may be involved with intermediary organizations, housing, Homeless Management System (HMIS) or Continuums of Care (CoC). At the provider level, the target universe is up to five stakeholders of the PATH provider from up to two providers from each of the five PATH grantees that will receive site visits. The stakeholders will be identified by the grantees and PATH providers. The total expected number of respondents is 75 (25 for the state and 50 for the provider).

4. PATH Service Provider Leadership Discussion Guide: The target universe is up to five leadership staff from up to two providers from each five PATH grantees that will receive site visits. The total expected number of respondents is 50.

5. PATH Provider Project Director Discussion Guide: The target universe is one project director (PD) or comparable staff from up to two providers from each five PATH grantees that will receive site visits. The total expected number of respondents is 10.

6. PATH Direct Care Provider Discussion Guide: The target universe is up to five outreach workers, case managers or direct care staff from up to two providers from each five PATH grantees that will receive site visits. The staff will be identified by the programs. The total expected number of respondents is 50.

7. PATH Consumer Focus Group Discussion Guide: The target universe is up to 10 current or former consumers from up to two providers from each five PATH grantees that will receive site visits. The total expected number of respondents is 100. PATH program staff will ask current consumers, who are available at the time of the focus group, if they would like to participate in the focus group. The PATH program staff will contact former PATH consumers, whom they are still in contact with, to determine if they would like to participate in the focus group and if they are available during the day and time the focus group is scheduled.

Table 1 shows the number of entities in the universe and total number to be sampled for the proposed data collection activities. Also, shown in Table 1 are the response rates achieved for applicable data collection activities during the last evaluation as well as the response rates expected for the proposed evaluation. It is expected that the response rates for the proposed evaluation will be equal to or greater than those obtained in the last evaluation since data collection will only occur for the PATH program and more follow-up will be conducted than was conducted in the previous evaluation. See Sections 2 and 3 for methods that will be utilized to maximize response rates.

Table 1. Universe and Proposed Sample Sizes for Proposed Data Collection Activities and Response Rates for Previous and Proposed PATH Evaluation

Data Collection Instrument	Total Number in Universe	Total Number to be sampled	Response Rate for Previous PATH Evaluation	Response Rates for Proposed PATH Evaluation
Web Surveys
SPC Web Survey	56	56	77%	77%-80%
Site Visits
Grantee Leadership Discussion Guide (State)	Unknown	25	Not Applicable	Not Applicable
SPC Discussion Guide	5	5	Not Applicable	Not Applicable
State and Provider Stakeholder Discussion Guide	Unknown	75	Not Applicable	Not Applicable
PATH Service Provider Leadership Discussion Guide	Unknown	50	Not Applicable	Not Applicable
Provider Project Director Discussion Guide	Unknown	10	Not Applicable	Not Applicable
PATH Direct Care Provider Discussion Guide (Outreach Worker/Case Manager)	Unknown	50	Not Applicable	Not Applicable
PATH Consumer Focus Group Discussion Guide	Unknown	Up to 100	Not Applicable	Not Applicable

2. Information Collection Procedures

Web Surveys

The Office of Evaluation will obtain contact information for all of the SPCs and the PATH providers from CMHS or from the following link, where it is publicly available: https://pathpdx.samhsa.gov/public?tab=stateandprovidercontacts Contact information for the intermediary organizations will be obtained from the SPCs. The same procedures noted above will be used for all three target populations. Emails will be sent to the designated respondents to inform them that data collection has started. A link to enter the Web-based survey will be included in the email, as well as information on the background, purpose, types of questions, and length of the survey (see Attachment 1). When the survey is first accessed, the consent form will appear, and the respondent will not be able proceed until the consent form is completed. If the respondent declines, a log-off message will appear; if consent is provided, the survey will begin. When respondents complete the survey a “thank you” email will be sent automatically (See Attachment 1). Non-responders will be sent three predefined, automated emails as needed, to remind them to complete their survey (see Attachment 1).

Site Visits

As noted above, the sample for the site visits will be purposive. Site visits will be a combination of in-person and virtual. Once the PATH grantees have been selected for a site visit, the OE staff will send an email to the SPC to inform them of the selection for the site visits and to provide information on the purpose, types of sessions, and length of the site visit. Once the SPC agrees to a site visit, a pre-site visit call will be arranged which may include grantee and provider staff. The staff will work with the SPC and provider staff to finalize the meeting agenda and schedule and to identify the participants for each session.

Informed consent will be obtained from all participants of the site visit discussions and the consumer focus groups (see Attachment 3). Signed consent forms will be collected before the session or focus group formally begins. The sessions may be recorded for accuracy if all interviewees consent. Notes will be taken during each session and the consumer focus groups and summarized. The recordings will be reviewed as needed to clarify the summary notes. All data will be protected by a Palo Alto Networks firewall and CrowdStrike as well as standard HHS security measures. The data will be further protected against unauthorized access using dual factor authentication (PIV card and passcode) and by restricting which SAMHSA staff can access the files. Data collected from the site visits and telephone interviews will also be stored in files that are password protected and access will be limited to individuals who need to work on them. All hard copies of forms and notes will be kept in secure, locked cabinets. Notes from the site visit discussions and consumer focus groups will be inductively analyzed with the assistance of software such as NVivo.

Focus groups with consumers (project participants) will be conducted during each site visit. The purpose of these focus groups is to understand services from a consumer perspective, including what services they received and satisfaction with services. The focus group participants will be identified by local PATH program staff. Because this is a hard-to-reach population whose participation is integral to the evaluation, focus group participants will be offered a non-cash incentive worth approximately $20. The local program staff will select the type of incentive that is appropriate for that specific community. The focus group participants will be either current or former PATH consumers. The focus groups will be open discussions structured by an open-ended discussion guide. As noted above, the notes from the focus groups will be inductively analyzed with the assistance of software such as NVivo. This will allow us to identify major themes.

2. Methods to Maximize Response Rates

Prior to beginning the PATH evaluation, all PATH Grantees’ SPCs will be sent an email from SAMHSA noting the importance of the PATH evaluation and encouraging their participation and that of their intermediaries and providers.

All web surveys and the telephone interviews will include multiple requests for participation to maximize response rates. As noted above, the recruitment process for the web surveys will include sending all potential respondents a recruitment email and weekly follow-up emails that include a link to access the survey. Respondents that start but do not complete the survey will receive a ‘partial responder' email (see Attachment 1).

2. Test of Procedures

The web survey and site visit protocols that were developed for the cross-program National Evaluation of SAMHSA Homeless Programs and that were utilized for the last PATH evaluation will be used. Those instruments and protocols have been revised to ensure that only information that is needed for the PATH evaluation is collected and to remove questions or sections that were more relevant to the other SAMHSA Homeless Programs.

2. Statistical Consultants for the PATH Evaluation

The lead for the evaluation has significant experience with qualitative and quantitative data collection and analysis. SAMHSA staff have been an important part of the evaluation process providing feedback concerning the evaluation plan. The evaluation advisors from SAMHSA are noted in Table 2.

Table 2. Advisors for the PATH Evaluation

SAMHSA Advisors
Meaghan McHugh, Ph.D., MPH Office of Evaluation Director	Director CBHSQ, SAMHSA 5600 Fishers Lane, Room 15E34, Rockville, MD 20857 Phone: 240-276-0589 Email: meaghan.mchugh@samhsa.hhs.gov
Tison Thomas, M.S.W., LMSW Division of State and Community Systems Development Director	Director CMHS, SAMHSA 5600 Fishers Lane, Room 14E22, Rockville, MD 20857 Phone: 240-276-2896 Email: tison.thomas@samhsa.hhs.gov
Dorrine Gross, MSW PATH Program Coordinator	Public Health Advisor CMHS, SAMHSA 5600 Fishers Lane, Room 14E21C, Rockville, MD 20857 Phone: 240-276-1625 Email: Dorrine.gross@samhsa.hhs.gov
Doug Slothouber, MA, MSW Division of State and Community Systems Development, State Grants Eastern Branch Chief	Branch Chief State Grants Eastern Branch 5600 Fishers Lane, Room 14E32, Rockville, MD 20857 Phone: 240-276-1453 Email: doug.slothouber@samhsa.hhs.gov

References

Charmaz, K. (2006). Constructing grounded theory. London: Sage.

U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration. (2016). FY 2016 PATH Funding Opportunity Announcement (FOA)). No. SM·16·F2 Catalogue of Federal Domestic Assistance (CFDA) No: 93.150

List of Attachments

Attachment 1: State PATH Contact Web Survey

Attachment 2: PATH Site Visit Discussion Guides

Attachment 3 : PATH Informed Consent Forms (Supplement)

Attachment 4: 2020 Triennial Process Evaluation Report of the Projects for Assistance in Transition from Homelessness Program (PATH)

Attachment 5: Introductory Script for Discussions (Supplement)

4