Response to Comments (60day)

CMS Response to Public Comments Received for CMS-10861

CMS received nine comments related to CMS-10861 (OMB control number 0938-1464) for the Medicare Health Outcomes Survey (HOS) Field Test during the 60-day comment period.

• Eight commenters offered general support for CMS’s efforts to update and shorten the survey, and/or add a web mode. One commenter expressed the importance of continuing to gauge beneficiary health outcomes to hold MA plans accountable. Another commenter expressed support for changes to reduce respondent burden and increase understanding.

Response: CMS thanks the commenters for their support.

• Two commenters appreciate CMS’s proposed testing of PROMIS items as potential replacements for current functional status questions involving less inclusive VR-12 activities. One suggested further revision to make questions more appropriate, such as replacing “heavy work around the house like moving heavy furniture” with the more common activity of “carrying groceries.”

Response: CMS thanks commenters for their feedback. We recognize that some MA enrollees cannot move furniture, but many can do these activities, and assessing the full range of physical functioning improves CMS’s ability to detect changes in functioning. The PROMIS item developers believe that even people who do not regularly move heavy furniture are generally able to comprehend and assess their ability to engage in this activity.

• Several commenters requested more clarity from CMS regarding the timeline for the field test, whether the proposed new PROMIS, GAD-2 and BRFSS questions will contribute to existing measures for MY 2026 or to new measures, and the transition plan for any methodological changes. One commenter suggested HOS measures be removed from Star Ratings until improvements to the HOS are finalized.

Response: CMS will announce the timeline once field testing has been approved. We have proposed only to test alternatives for select survey items that could enhance and refine existing measures; no modifications have been proposed at this time. Any future proposals will be informed by analyses of field test data. Non-substantive changes to existing measures used in the Part C and D Star Ratings would be announced through the Advance Notice/Rate Announcement process. Substantive changes to existing or new measures for Part C and D Star Ratings would go through the pre-rulemaking review process, then be formally proposed through rulemaking.

• Three commenters expressed concern about questions being removed, specifically those used as covariates in the case-mix adjustment model and asked whether any analysis would be done on the effect of removing questions and how correlated covariates that continue to be part of the model may change.

Response: The effect of the question deletions was tested and found to be very small, with the correlations to the case-mixed performance measurement (longitudinal) results being greater than 0.99 for both PCS and MCS. In order to limit burden and make room for testing items with potentially greater value for longitudinal quality measurement, CMS opted to remove these questions.

• One commenter asked CMS to clarify the meaning of “unused sample frame” and another asked that the field test sample be pulled from both the unused sample for HOS and HOS-M.

Response: “Unused sample frame” refers to enrollees not selected for the Baseline, Follow-Up, or FIDE SNP samples in the annual administration of HOS. Because the changes to the HOS have not been suggested for application to the HOS-M, it would not be appropriate to include PACE members from the unused HOS-M sample frame in the HOS field test. However, enrollees in contracts that used HOS-M for one or more of their FIDE SNPs may be included in the field test sample frame.

• One commenter recommended the sample size be increased and asked CMS to ensure special sub-populations be included, such as those with low literacy and people who are dually eligible, to ensure that the questions and scaled responses are understood and meaningful to all. Another asked for clarification on whether plans would be asked for participant lists or whether the lists would be determined by CMS.

Response: CMS appreciates these suggestions. As indicated in Supporting Statement B, page 9, the sample size for the field test is adequate for its intended purpose. The HOS field test sample will be representative of participating contracts. The purpose of the field test is to provide information on a diverse set of MA enrollees representing all who are eligible to participate in the HOS. The dually eligible population will be included in the test, as will individuals of other representative backgrounds. CMS selects a randomized sample from the list of participating contracts and will bear all costs of administering the Field Test.

• Two commenters expressed concern about the double invitation mailings for the web mode. Another suggested second email invitations be sent only to nonrespondents instead of all those with an email address. Two commenters also asked that QR codes be added for convenience and to drive a higher response rate.

Response: CMS thanks the commenters for their feedback. Based on lessons learned from MA CAHPS, CMS will include a QR code and a website link so enrollees have options in their response. This information will go out in the prenotification letter and web mode invitations. Non-respondents will also receive a first and second mailing of the paper version of the survey.

• One commenter suggested CMS consider adjusting the minimum result threshold from 100 to the previous level of 30 in tandem with modality testing and as part of the evaluation of field test results. They noted that the increased threshold can result in contracts being excluded from the Star Ratings, impacting overall ratings when results for high performers are excluded.

Response: Beginning in Measurement Year 2022, CMS increased the minimum denominator for the HOS measures from 30 to 100. The purpose of the change, which was finalized in the Contract Year 2022 Final Rule, was to align the longitudinal HOS measures with the cross sectional HEDIS HOS measures derived from the HOS and to address plans’ concerns about representativeness of the sample and the reliability of the data used for performance analysis and Star Ratings calculations. Any changes to the denominator would be substantive and must go through the pre-rulemaking review process before being formally proposed through rulemaking.

• Two commenters asked that data be returned to plans earlier for use in quality improvement. One noted “blinded longitudinal data” do not offer actionable information to plans because they are unable to retrieve specific respondents’ answers. Another asked that baseline aggregate data be returned closer to the survey completion date.

Response: Longitudinal data are not “blinded.” Rather, Performance Measurement datasets include beneficiary-level data for each participating plan. The timeline for returning baseline aggregate data (in the fall the year following data collection) is driven by plans’ requests to receive Performance Measurement data earlier and prior to the first Plan Preview for Star Ratings (now in the summer following follow-up data collection). As previously stated, clinical data, including HRAs, are better used to screen for and address patient-level needs than survey data.

• Several commenters urged CMS to fully report all results from this field testing so that interested parties can evaluate how these changes may impact HOS. One asked how the results will be reported back to plans, and if data of disenrolled beneficiaries will be included.

Response: CMS appreciates the feedback and will share relevant results when available. Results will be communicated with plans, and communication given before changes to the HOS are proposed. Data from beneficiaries who completed a Field Test survey and subsequently disenrolled from the participating contract will be included in aggregated results.

• One commenter suggested CMS re-test the HOS instrument by beneficiary sub-group, particularly dually eligible individuals. The same commenter suggested stakeholder input to explore other languages for the survey other than English or Spanish.

Response: CMS gathered stakeholder input on HOS enhancements through a technical expert panel convening in September 2022, and this input is reflected in the field test instrument. We will test English and Spanish in their naturally occurring proportions. All future translations of the final instrument will undergo rigorous testing to ensure comprehension by intended audiences.

• One commenter expressed concern about the physical component score (PCS) and mental component score (MCS) coefficients, which sometimes lead to unexpected and unintuitive results when interpreted individually.

Response: While the interpretation of the PCS and MCS coefficients individually may not be intuitive in some cases, the coefficients do function together so that improved health statuses result in higher PCS and MCS scores.

• Three commenters sought clarification on whether telephone outreach will follow the same guidelines as current telephone outreach.

Response: Yes, the telephone portion of the HOS protocol will remain unchanged in the Field Test. Up to 5 telephone attempts per phone number will be made to nonrespondents and enrollees who return blank or incomplete surveys.

Page 3 of 3