Final - Supporting Statement Part B MTW Asset Building Cohort

Supporting Statement for Paperwork Reduction Act Submissions

Moving to Work, Asset Building Cohort Evaluation

OMB # 2528-0345

B. Collections of Information Employing Statistical Methods

The agency should be prepared to justify its decision not to use statistical methods in any case where such methods might reduce burden or improve accuracy of results. When statistical methods are involved, the following documentation should be included with the

Supporting Statement A to the extent that it applies to the methods proposed:

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

This revision request to OMB # 2528-0345 seeks approval to include a household survey of participants in the Opt-Out Savings program. Exhibit B-2 shows the expected household survey sample size for each PHA implementing an Opt-Out Savings program, including the number of households assigned to the treatment and control groups and the targeted sample size for the survey. Exhibit B-1 provides the number of PHAs, rounds of data collection, respondents per round, and response rate for the Opt-Out Savings Account Household Survey.

Exhibit B-1: Sample Sizes and Response Rates for Requested Revision of OMB# 2528-0345 (Opt-Out Savings Account Household Survey)

	PHAs	Rounds	Respondents Per Round	Targeted Response Rate
Resident Household Surveys	9	1	1,100	60%*

*As explained below, we anticipate an 80 percent response rate from currently assisted households and a lower rate from those who have exited assistance by the time of the survey.

Exhibit B-2: Sample Size by PHA, Opt-Out Savings Programs

PHA Name	Assigned to OOS (treatment group)	OOS households sample size	Control households sample size	Number of households not in OOS
Housing Authority of the County of Santa Cruz	75	50	50	5,249
West Palm Beach Housing Authority (FL)	25	25	25	4,175
New Hampshire Housing (NH)	500	70	70	3,726
Madison Housing Authority (NJ)	60	60	60	254
Salem Housing Authority (OR)	100	100	90	3,049
South Carolina Regional Housing Authority #3 (SC)	225	80	80	1,148
Medina Metropolitan Housing Authority (OH)	70	70	70	615
Grand Rapids Housing Commission (MI)	25	50	50	3,673
Medford Housing Authority (MA)	100	50	50	1,827
Total	1,180	555	545	23,716
Notes: Our power analysis accounts for an expected 80 percent response rate for residents of subsidized housing and a 25 percent sample loss due to households exiting assistance. We assume it will be more difficult to find households after exit, so we estimate an average response rate (across those still receiving subsidy and the 25 percent of exited households) to be 60 percent. We are conducting an intent-to-treat analysis. The number of households indicated to be participating in the Opt-Out Savings Account program is based on information provided by PHAs in March and August of 2024.

2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the purpose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Revision Request

The Opt-Out Savings program household survey, the new data collection under this revision request, targets roughly equal numbers of treatment group and control group participants at each site to maximize statistical power. The sampling plan is stratified by site. Regression analyses of outcome measures will account for the stratified nature of the sample using PHA fixed effects. The sampling plan offers roughly equal sample across sites, allowing the researchers to estimate impact of the “average program” (up to the constraints of the size of the treatment sample at each site). The true constraint in this analysis is the small size of the treatment groups.

We conducted statistical power analysis using PowerUp! (Dong and Maynard 2013).¹ We assume an alpha level of 0.05 (a 5 percent probability/tolerance of falsely concluding a finding is statistically significant if there truly was no effect); we use 80 percent power (an 80 percent probability/tolerance of detecting any statistically significant impact if there truly is one); and we assume that a two-tailed hypothesis test will be conducted. Our power analysis assumes a 60 percent response rate which is fairly conservative. We find that, given this sample size, the average impact on the percent of households who have been behind in rent in the last 3 months would have to be 6 percentage points and the average impact on the percent of households who have enough savings for a $400 emergency would have to be 8.5 percentage points in order for this study to have at least an 80% chance of detecting a statistically significant impact. These minimum detectable impacts are acceptable to HUD.

Procedures with Special Populations

We do not believe we will be communicating with people from special populations.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Revision Request

Under this revision request to OMB #2528-0345, the Department seeks approval to conduct interviews with up to 1,100 participants in the Opt-Out Savings program. Interviewers will conduct the survey data collection using computer-assisted personal interviewing (CAPI) technology. Each participant selected for the interview will receive an advance letter informing them about the upcoming data collection. The letter will inform participants that an interviewer will be calling them to conduct the interview, the duration of the interview, the content of the survey, the incentive amount (see Part A for details), and who to contact with questions.

Within one week of sending the advance letter, interviewers will begin calling participants to schedule the interview. Interviewers will complete the survey by phone if possible. To maximize the response rates, interviewers will conduct in-person follow-up visits to complete the interview with those participants who did not complete by phone.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of test may be submitted for approval separately or in combination with the main collection of information.

Early drafts of the interview protocols have been reviewed by HUD personnel, Abt Global and MEF Associates staff, our Expert Panel, which includes a former HUD-assisted resident and participant in a rent reporting program, and our project consultants to ensure that the instruments are clear, flow well, and are as concise as possible. The household survey was developed using existing scales (when appropriate scales existed) to ensure that survey items are tested and suitable for this study population.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Data collection activities described in this document will be carried out directly by the staff of Abt Global and MEF Associates.

Exhibit 2: Individuals Consulted

Name	Telephone Number	Role in Study
Sam Elkin	703-838-2722	Senior Advisor
Dr. Stephen Nuñez	650-521-4532	Project Director (sub-contractor)
Prof. Susan Clampet-Lundquist	610-660-1680	Expert Panelist
Prof. J. Michael Collins	608-616-0369	Expert Consultant
Dr. Judy Geyer	617-520-2952	Principal Investigator

Inquiries regarding the study’s planned analysis should be directed to:

Dr. Judy Geyer	Abt Global, Principal Investigator	617-520-2952
Elizabeth Rudd, Ph.D.	HUD, Contracting Officer’s Representative	202-402-7607