B-2 Round 1 Cognitive Testing Findings

2025 NTIA Internet Use Survey Cognitive Testing Round 1

Conducted by CBSM at U.S. Census Bureau

Document prepared by Dave Tuttle, Rodney Terry, Jonathan Katz, Caitlyn Keeve, Marcus Berger, Betsa Otero Class, Ian Le, Jennifer Pace, and Kristen Kohm.

6/18/2025

Introduction

This document contains the results and recommendations from cognitive testing of Round 1 of the 2025 National Telecommunications and Information Administration (NTIA) Internet Use Survey; it has been reviewed for disclosure avoidance and approved under CBDRB-FY25-CBSM003-008. For questions or comments, please contact jonathan.m.katz@census.gov. In addition to the authors, we want to acknowledge Kevin Younes, Jasmine Luck, and Victoria Johnson for their help recruiting on this project.

Methods

The main goal of testing was to evaluate a series of revised items that were streamlined for clarity but also length to reduce time administered in the interview, at the request of NTIA and ADDP (Associate Directorate for Demographic Programs).

We interviewed 20 participants for Round 1 of 2025 NTIA Internet Use Survey testing. Participants came from existing Center for Behavioral Science Methods (CBSM) databases where they had either completed a screener for a prior testing project or completed a cognitive or usability interview. Participants were re-contacted for this study and asked to complete either a Qualtrics self-administered screener specific to this study prior to interviewing.

Interviews were conducted either remotely via Microsoft Teams or by telephone. Interviewers completed an interviewer-administered Qualtrics programmed instrument with which they could follow along. Participants were asked about internet use of the household and of a randomly selected household member. A section of Table 1 (below) shows the relationship between the participant and the individual they served as a proxy for, if any. Following the completion of the survey, participants were asked retrospective probes about their answers to the survey questions.

Participant characteristics

The participant characteristics are listed in the table below. We were limited in recruiting for a variety of demographic characteristics given the shortened timeline for recruitment and testing. Participants were from a variety of states across the country: Arizona, California, Illinois, Indiana, Michigan, New Jersey, New York, Ohio, Texas, Virginia, and Washington.

Table 1. Participant Characteristics
Household	Single member	2
	Related	16
	Unrelated	2
Education	Completed high school	2
	Some college, no degree	6
	Associate’s degree (AA/AS)	2
	Bachelor’s degree (BA/BS)	6
	Post-Bachelor’s degree (for example, MA, MS, Ph. D, JD)	4
Sex	Female	10
	Male	10
Race*	White	17
	Hispanic	3
	Black or African American	0
	Asian	1
	American Indian or Alaskan Native	0
	Middle Eastern or North African	2
	Native Hawaiian or other Pacific Islander	0
Age	18-34	3
	35-49	8
	50-64	5
	65 and over	4
Income	Less than $15,000	0
	$15,000 to $24,999	0
	$25,000 to $49,999	0
	$50,000 to $100,000	15
	More than $100,000	4
Interview Type	Microsoft Teams	19
	Telephone	1
Relationship to Participant for Proxy Questions (EMAIL to HOMIOT)	Self	9
	Husband/wife/spouse	1
	Unmarried partner	2
	Children	3
	Siblings	2
	Roommate(s)/Friend(s)	2
	Other	11

*A participant could select one or more races

Tested Wording	Findings	Recommendations
Production Wording LAPTOP What about a laptop? [Do you/Does anyone in this household] use a laptop computer? (If needed) A laptop is a personal computer that is portable and includes a built-in keyboard and screen. (If yes & is multi-person household) Who is that? Round 1 Wording LAPTOP [Do you/Does anyone in this household, including you,] use a laptop computer? Think about any laptop computers [you/members of your household] currently have access to and have used during the past six months. (If needed) A laptop is a personal computer that is portable and includes a built-in keyboard and screen. (If yes & is multi-person household) Who is that?	No issues detected. All participants responded with no difficulty and appeared to understand the question as intended.	No recommendation needed.
Production Wording DESKTP What about a desktop? [Do you/Does anyone in this household] use a desktop computer? (If needed) A desktop is a personal computer that must be plugged into a power outlet and is typically at a single location. (If yes & is multi-person household) Who is that? Round 1 Wording DESKTP What about a desktop? [Do you/Does anyone in this household] use a desktop computer? (If needed) A desktop is a personal computer that must be plugged into a power outlet and is typically at a single location. (If yes & is multi-person household) Who is that?	No issues detected. All participants responded with no difficulty and appeared to understand the question as intended. When probed, participants were able to define a desktop.	No recommendation needed.
TABLET Production Wording What about a tablet or e-book reader, such as an iPad or Kindle? (If needed) A tablet or e-book reader is a personal computer that is portable and has a built-in screen, but does not have a built-in physical keyboard. (If yes & is multi-person household) Who is that? Round 1 Wording What about a tablet or e-book reader, such as an iPad or Kindle? (If needed) A tablet or e-book reader is a personal computer that is portable and has a built-in screen, but does not have a built-in physical keyboard. (If yes & is multi-person household) Who is that?	Most participants understood the question as intended and did not have difficulty answering the question. However, one participant initially did not report use of a tablet because they only use it from time to time and not on a regular basis (i.e., “only a couple of times a month.”). It appears that the participant forgot the instruction to include devices even if only used occasionally. The cognitive interviewer repeated the question and the participant changed their response to “yes.” When probed, all participants understood the definition of a tablet and its differences from a laptop computer.	No recommendation needed.
WEARAB Production Wording [Do you/Does anyone in this household] use a wearable device that is connected to the Internet, such as a smart watch or fitness band? (If needed) Examples include an Apple Watch or Fitbit. (If yes & is multi-person household) Who is that? Round 1 Wording [Do you/Does anyone in this household] use a wearable device that is connected to the Internet, such as a smart watch or fitness band? (If needed) Examples include an Apple Watch or Fitbit. (If yes & is multi-person household) Who is that?	All participants understood the question as intended and responded with no difficulty. When probed, most participants were able to name wearable devices other than a smartwatch or fitness band. Other devices named include aura ring, apollo neuro, smart glasses, smart bracelets, and a Garmin watch. Medical devices were named as examples but were not reported in response to the question.	No recommendation needed.
TVBOX Production Wording [Do you/Does anyone in this household] use a smart TV, a game system, or another device that connects to the Internet and plays through a TV? Examples include an Apple TV, Xbox, or Roku that can access the Internet. (If yes & is multi-person household) Who is that? Round 1 Wording [Do you/Does anyone in this household] use a smart TV, a game system, or another device that connects to the Internet and plays through a TV? Examples include an Apple TV, Xbox, or Roku that can access the Internet. (If yes & is multi-person household) Who is that?	Most participants understood the question as intended and responded with no difficulty. However, one participant appeared to be confused. She responded “no” though she said that she has a TV that connects to the internet. This is because she does not have Amazon Prime on the TV. She appeared to think that "connected to the internet" means specifically through a paid streaming service. While appearing to be confident that her TV was a smart TV, she did not explain how it connected to the internet. Question for NTIA: If participants do not actively connect their TV to the Internet but know it is a smart TV, how should they answer this question? A second participant reported having an Apple TV. However, based on other CBSM staff’s experience knowing the participant, this participant does not actually have an Apple TV. When initially answering the question and later during probing, participants mentioned several examples: portable video camera, video camera that connects to a TV, iPad, Firestick, PS4, “Nvidia” device, Amazon Alexa, your own phone, Xumo, cable box, Nintendo Wii, Slingbox, Nintendo Switch, Google Chromecast, Google Home, Samsung.	No recommendation needed.
DEVQUA Production Wording Thinking about all the different devices we just discussed, how well do the computers and other Internet-connected devices [you use/used by this household] work overall? Please do not include home Wi-Fi routers or similar equipment. Read and select best match; if respondent is unsure, ask for an average They do not work at all They work poorly They somewhat work They work well They work perfectly Round 1 Wording How well do the computers and other Internet-connected devices [you use/used by this household] work overall? Please do not include home Wi-Fi routers or similar equipment. Read and select best match; if respondent is unsure, ask for an average They do not work at all They work poorly They somewhat work They work well They work perfectly	Most participants appeared to understand the question as intended and responded with no difficulty. However, two participants appeared to include routers in their response. They were not able to separate the performance of the devices and the quality of internet connection. Because this issue should be covered by an interviewer properly trained to listen for respondent errors, no changes to the question are needed. However, if future rounds of data collection are CAPI, the wording may need to be modified. It may be challenging for even tech savvy participants to distinguish whether an issue is because of the router or the internet service itself. Most users may only experience the effects of a problem via their internet failing to load or a device not connecting. It may not be realistic to ask users to disentangle a router issue from a problem with the internet provider. The user may only know it was the router if the user reset it and fixed the issue, or swapped out the router. Another participant was confused because he did not know the definition of a router. He said he preferred the term “router” not be mentioned in the question to avoid confusion.	Consider one of the following: Remove the sentence “please do not include home Wi-Fi routers or similar equipment” as it seems like that may still be confusing participants. Provide the definition of the term “router” if the respondent appears confused or unsure of how to answer the question or the definition of the term “router.”
INHOME TO INOTHER Probe Earlier, you answered a series of questions about [you/your household's] Internet use at different locations including at home, work, school, coffee shops, etc. Which devices were [you/your household] thinking of as you answered these questions	When answering this probe, participants overall mentioned several devices, including cell phones, laptop computers, tablets, smart TVs, and desktop computers. The most frequently mentioned devices were portable devices.	No recommendation needed.
INWORK Production Wording [Do you/Does anyone in this household] use the Internet at work? Do not include work from home. (If yes & is multi-person household) Who is that? Round 1 Wording [Do you/Does anyone in this household] use the Internet at an office or other workplace outside the home? Do not include work from home.	No issues detected. All participants appeared to understand the question as intended and answer the question with no difficulty.	No recommendation needed.
INCAFE Production Wording What about at a coffee shop or other business that offers Internet access? (If needed) [Do you/Does anyone in this household] use the Internet while at a coffee shop or other business that offers Internet access? (If yes & is multi-person household) Who is that? Round 1 Wording What about at a coffee shop or other business that offers Internet access? (If needed) [Do you/Does anyone in this household] use the Internet while at a coffee shop or other business that offers Internet access? (If yes & is multi-person household) Who is that?	Most participants appeared to understand the question as intended and answered with no difficulty. However, two participants appeared to be confused in different ways. One participant interpreted the question as only referencing internet cafes, and not all businesses that offer internet access. Another respondent believed that the question referenced only using wi-fi at coffee shop and not internet access of any kind.	Consider adding an instruction about the inclusion of the business’ wi-fi or the participants own wireless service when answering the question. What about at a coffee shop or other business that offers Internet access? (If needed) [Do you/Does anyone in this household] use the Internet while at a coffee shop or other business that offers Internet access? Please include whether using the business’s Wi-Fi internet service or your own wireless service. (If yes & is multi-person household) Who is that?
INTRAV Production Wording What about while going from place to place? Examples include using the Internet while riding public transit, or using Google Maps or Waze on your cell phone while driving. (If needed) [Do you/Does anyone in this household] use the Internet while going from one place to another? (If yes & is multi-person household) Who is that? Round 1 Wording What about while going from place to place? Examples include using the Internet while riding public transit, or using Google Maps or Waze on your cell phone while driving.  IF NEEDED: [Do you/Does anyone in this household] use the internet while going from one place to another?	All participants appeared to understand the question as intended and answered the question with no difficulty. Other than using Google Maps/Waze, participants during probing also mentioned wireless music service, handheld games, wearables, tablets, laptop computers, internet browser on cell phone, etc. One participant mistakenly mentioned Sirius car radio in response to this probe.	No recommendation needed.
MOBDAT Production Wording [Do you/Does anyone in this household] access the Internet using a data plan for a cell phone, smartphone, tablet, mobile hotspot, or other device? This type of Internet service is provided by a wireless carrier, and may be part of a package that also includes voice calls from a cell phone or smartphone. Yes No Round 1 Wording [Do you/Does anyone in this household] access the Internet using a mobile data plan for a cell phone, tablet, or other device? This type of Internet service is provided by a wireless carrier and may be part of a package that also includes voice calls from a cell phone. Yes No	Most participants understood the question as intended and answered the question with no difficulty. However, three participants appeared to be confused by the question. They seemed to confound wireless home internet with use of a mobile data plan that can be used at home or outside the home. When probed, participants mentioned several other terms for the term “cell phone.” The most frequently mentioned terms were “smartphone,” “mobile phone,” and “phone.” Less frequently mentioned terms include “mobile technology,” “cell,” and “device.”	Consider adding language to emphasize that the mobile data plan can be used outside the home and inside the home as needed. [Do you/Does anyone in this household] access the Internet using a mobile data plan for a cell phone, tablet, or other device? This type of Internet service is provided by a wireless carrier and may be part of a package that also includes voice calls from a cell phone. This service is used outside the home and inside the home as needed. Yes No
HMINT1 Production Wording In addition to [your/your household’s] mobile Internet service or data plan, we are interested in whether [you/your household] also use[s] any other type of Internet service when at home. Round 1 Wording In addition to [your/your household’s] mobile data plan, we are interested in whether [you/your household] also use[s] any other type of Internet service when at home.	Based on participants’ answers to questions about their mobile data plan, participants appeared to understand the wording change to this introduction.	No recommendation needed.
HOMTE/TEOTHR Production Wording I am going to read a list of ways that people access the Internet from their homes, other than a mobile data plan. At home, [do you/does anyone in this household] access the Internet using: Read and select all that apply High-speed Internet service installed at home, such as cable, DSL, or fiber-optic service? (If needed) This type of Internet service is often provided by a cable company or phone company. Satellite Internet service? (If needed) This type of Internet service is received through a satellite dish installed at home. Some other service? (Do not read) None What other service? Round 1 Wording I am going to read a list of ways that people access the Internet from their homes, other than a mobile data plan. At home, [do you/does anyone in this household] access the Internet using: Read and select all that apply Cable, fiber-optic, DSL, or other Internet service provided by plugging a wire into a modem or router? (If needed) This type of Internet service is often provided by a cable company or phone company. Satellite Internet service? (If needed) This type of Internet service is received through a satellite dish installed at home. Fully wireless Internet service,? (If needed) This type of Internet service is received through a device that connects to a wireless carrier’s network to provide Internet service to the home. Some other service? (Do not read) None What other service?	Many participants were able to answer the question as intended and provide the answer that best fit their situation. Most participants had either one of the first two options (i.e., a wire plugged into a modem/router, or satellite internet service). However, several participants were confused by the “fully wireless internet service” response option, either during the initial interview or during probing. They confused this term with a wire plugged into a wireless modem/router and would need to hear the definition to finally decide that they did not have fully wireless internet; or they confused the term with their mobile data plan. Further, several participants during probing reported being unfamiliar with the term. In addition, one participant reported “Yes” for “High-speed Internet service installed at home…” when in fact they had fully wireless Internet. We learned this information based on other CBSM staff’s experience knowing the participant. When probed on other forms of service, options mentioned included the use of a neighbor’s internet, a mobile plan hotspot, and dial-up internet.	Consider one of the following recommendations: Revert to the original wording from the 2023 data collection cycle Modify the “Fully Wireless Internet service option” to “Wireless home Internet service provided by a wireless carrier. (If needed) This type of Internet service is received through a device that connects to a wireless carrier’s network to provide Internet service to the home.”
USEINT Production Wording We are interested in learning about the applications and services people use on the Internet. [(If is multi-person household) We have randomly selected one member of your household for this section.] Consider how [you personally/NAME] currently use[s] the technologies we’ve been discussing, whether at home or any other location. Please focus on activities [you have/NAME has] done online at least occasionally during the past six months. Round 1 Wording We are interested in learning about the applications and services people use on the Internet. [(If is multi-person household) We have randomly selected one member of your household for this section.] Consider how [you personally/NAME] currently use[s] the technologies we’ve been discussing, whether at home or any other location, at least occasionally during the past six months.	No issues were found with the wording of this instruction, and the removal of text from the production version did not seem to affect comprehension. Several participants commented on the reference period of “the last six months” throughout the interviews, indicating that the references appear to be prominent and remain salient.	No recommendation needed.
EMAIL to HOMIOT Proxy Reporting Earlier, you answered a series of questions about ${q://QID92/ChoiceGroup/SelectedChoices}'s Internet use. These questions covered a variety of Internet use activities like checking email, watching videos, working, providing services, etc. I will show you these questions on screen.  How easy or difficult was it to answer these various Internet use questions about ${q://QID92/ChoiceGroup/SelectedChoices}? [IF NEEDED] Which questions were easy to answer?  [IF NEEDED] Can you tell me more about that? [IF NEEDED] Which questions were difficult to answer?  [IF NEEDED] Can you tell me more about that?	Nearly all participants said the questions about a sampled household member’s internet-related activities were easy to answer. They said things like, “It was easy. I pretty much know what she's doing as far as this goes.” “It was pretty easy because he doesn't use a lot of this.” “Really easy because [name] is tech savvy so he's always on social media, playing games, watching YouTube, he participates in video calls for school, he’s always texting and instant messaging and emailing. It was easy because he does all of that.” “All of them were pretty easy.” One participant indicated that the examples were helpful in clarifying the scope of the question. Only one participant said they thought the questions were not easy, and suggested asking whether the proxy respondent is familiar with the sampled person’s “wifi usage.” This type of comment is not uncommon from proxy respondents, but we think it noteworthy that the participant described internet-related activities as “wifi usage.” This may be reflective of a limited understanding of the nuances of internet and computer technology.	No recommendation needed.
SOCIAL Production Wording What about using social networks, such as Facebook, Twitter, or Instagram? (If needed) [Do you/Does NAME] use social networks? Yes No Round 1 Wording What about using social media, such as Facebook, LinkedIn, or Instagram? (If needed) [Do you/Does NAME] use social networks? Yes No	No problems were detected with this item. All participants appeared to answer it correctly. When asked whether they use other terms for “social media,” all responded that they do not, and that “social media” is the best and only term. They said things like, “No, that's the standard term that I would expect.” “No, that's a good term for me.” “No other terms, maybe ‘social media platforms.’” Participants gave many examples of social media platforms they know of: TikTok, WhatsApp, Telegram, Facebook, Instagram, Twitter/X, Snapchat, YouTube, BlueSky, Threads, Pinterest, LinkedIn, Discord, Reddit, Truth Social, and Signal.	Retain the term “social media” in place of “social networks.”
CONFER Production Wording In the past six months, [have you/has NAME] participated in video or voice calls or conferencing over the Internet, such as with FaceTime or Zoom? (If needed) [Do you/Does NAME] participate in video or voice calls or conferencing? Yes No Round 1 Wording In the past six months, [have you/has NAME] participated in video or voice calls or conferencing over the Internet, such as with FaceTime or Zoom? (If needed) [Do you/Does NAME] participate in video or voice calls or conferencing? Yes No	All participants appeared to answer this question as intended. Participants offered examples of various video conferencing platforms including Google Meet, Microsoft Teams, Skype, Facebook Messenger, WhatsApp, Zoom, and WebEx. They cited using video conferencing for education, telemedicine, and social reasons. They made comments like, “I said yes because I use Google Meet for calls, video calls, very often.” “Video calls for schooling. He's home-schooled so he meets with his teacher (through video conferencing) and he also video calls his friends on Facebook and Facetime.” “I don't think he ever Facetimed or Zoomed without me present, and it's been much more than six months now."	Consider changing to “video calls or voice calls” to make the question simpler and easier to administer. Google Trends data indicate that “video calls” is used far more commonly in the United States than “video conference.”
TELEWK Production Wording [Do you/Does NAME] use the Internet to telecommute or work while away from [your/his/her] usual workplace, such as working from home? Yes No Round 1 Wording [Do you/Does NAME] use the Internet to telework or work while away from [your/their] usual workplace, such as working from home? Yes No	Participants generally understood the question to include remote work or telework from any location other than a workplace, such as at home, in a coffee shop, at a library, when staying at someone else’s house or an Airbnb, or while traveling on a train. Some also specified that the work involves using a computer. They made comments like, "Somebody that works from home... I’m thinking somebody that has a job that they can work on their computer at or something kind of like that, kind of a job where it's, you're able to work on your laptop or something. Not a, like a restaurant job or a cook, cashier." "Places that you sporadically sometimes work from maybe like a cafeteria, maybe a library, maybe commuting, maybe somebody else's home.” One participant gave an apparently incorrect proxy response for an unrelated housemate. This participant answered yes and indicated that they were thinking of the sampled household member’s “job” as a student and home as the workplace; thus, the sampled household member is teleworking while participating in online classes away from home. We note that the participant should not have received this question because they reported that the sampled HH member does not have a job.	Assuming that the skip patterns in the production survey work correctly, no recommendation needed.
EGOVTS Production Wording What about accessing government services, such as registering to vote or renewing your driver’s license? (If needed) [Do you/Does NAME] use the Internet to access government services? Yes No Round 1 Wording What about accessing government services, such as registering to vote, renewing your driver’s license, or applying for government benefits? (If needed) [Do you/Does NAME] use the Internet to access government services? Yes No	We did not use retrospective probes with this question, and all but three participants answered it without additional comment. Two of the participants who made comments appeared to answer the question as intended. One commented that they said yes because they thought of renewing a driver’s license online. Another participant initially answered no but then wondered aloud whether the category would include FAFSA (i.e., the Free Application for Federal Student Aid) and changed the response to yes. The third participant thought of renewing her car registration online but answered no. This participant appeared not to equate “government” with a state government agency like the department of motor vehicles, and/or may have thought it meant only the federal government. When the interviewer re-read the question, the participant did not change their answer: "I'm going to say no... I mean, I don't know really because that's state, but then it probably goes to the government, I don't know." One participant who provided a proxy response for an unrelated housemate answered “don’t know.”	“…renewing your driver’s license or car registration…” (if needed) Please consider services provided by federal, state, and local government agencies.
USESVC Production Wording What about requesting services provided by other people? Examples include hailing a ride using Uber, ordering food using Door Dash or Instacart, reserving a room through Airbnb, or requesting a home repair through Angi? (If needed) [Do you/Does NAME] use the Internet to request services provided by other people? Yes No Round 1 Wording What about requesting services provided by other people? Examples include hailing a ride using Lyft or Uber, ordering food using DoorDash or Instacart, reserving a room through Airbnb, or requesting a home repair through Angi? (If needed) [Do you/Does NAME] use the Internet to request services provided by other people? Yes No	All participants appeared to answer this question easily as intended. One participant asked, "What about like for a haircut or something? Would that be the same?" The participant asked the interviewer to re-read the question and his answer remained yes.	No recommendation needed.
FINANC Production Wording [Do you/Does NAME] use the Internet for financial services such as online banking, investing, paying bills, or sending money to other people using services like CashApp, Venmo, or PayPal? (If needed) Examples include using your bank’s website or app, and paying a credit card or utility bill online. (1) Yes (2) No Round 1 Wording [Do you/Does NAME] use the Internet for financial services such as online banking, investing, paying bills, or sending money to other people using services like CashApp, Venmo, or PayPal? (If needed) Examples include using your bank’s website or app, and paying a credit card or utility bill online. Yes No	No one appeared to answer the question incorrectly. They described numerous internet-based financial activities (managing bank accounts online, paying bills, and sending money to friends, family, landlords, etc.) and applications (Cash App, Zelle, Venmo, PayPal, Apple Pay, and Greenlight). Only two participants answered no, both of whom were proxy respondents. In both cases, the proxies help the sampled person use the internet (because of advanced age and visual impairment, respectively) and they do not use the internet for financial purposes.	No recommendation needed. Note: “Cash App” is two words separated by a space.
HOMIOT Production Wording What about interacting with household equipment or appliances that are connected to the Internet, such as a smart thermostat, light bulb, or security system? (If needed) [Do you/Does NAME] use the Internet to interact with household equipment or appliances? Yes No Round 1 Wording What about interacting with household equipment or appliances that are connected to the Internet, such as a Nest thermostat, a Ring camera, or a Hue light bulb? (If needed) [Do you/Does NAME] use the Internet to interact with household equipment or appliances? Yes No	There were no apparent problems with this question. Most participants answered yes and specified a variety of smart appliances, including light bulbs, doorbell cameras, thermostats, garage door openers, refrigerators, clothes washers, vacuum cleaners, toothbrushes, and cat litterboxes. Some participants also included smart TV’s, though always as one of many smart devices, never as the only example (i.e., they did not answer yes on the basis only of having a smart TV). Several participants also included smart speakers (Alexa, Amazon Echo) among the other types of smart devices specified by the question. Some of these participants went on to say that their smart appliances and other smart devices are controlled through the smart speakers, e.g., “Everything is connected through the internet and activated with the Alexa.” All participants were familiar with Nest and Ring devices, and most had heard of Hue light bulbs. One participant commented on the use of brand names as “weird” and suggested just using “smart” to describe various types of devices: “I would just say ‘smart bulb’… It's weird to use brand names instead of just saying ‘smart’… I think it’s broader to just call them smart thermostat, smart camera, smart light bulb than to give specifics."	Use “smart appliances” and drop “household equipment.” Change “Hue” to “Philips” as the latter is the manufacturer of Hue smart bulbs and is a familiar household brand.
MEDINT Production Wording If multi-person household) Our remaining questions are about your household as a whole. (All) I’m going to ask a couple of questions about how [you have/your household has] used the Internet for health-related activities during the past six months. Round 1 Wording (If multi-person household) Our remaining questions are about your household as a whole. (All) I’m going to ask a couple of questions about how [you have/your household has] used the Internet for health-related activities during the past six months.	All participants indicated they found the health-related questions easy to answer and nearly all indicated they were thinking about their entire households, not just themselves. Several respondents, all female, indicated that they themselves handle all health-related matters for the entire household. They made comments like, "Pretty easy because anything medical, I'm the one who does it for the both of us." “Easy because I thought about who does it in the household and it's me.” “Both, myself and my household, but usually it's just me who does my whole household.” Similarly, a proxy for an unrelated housemate, who is an older person and does not use computer devices on his own, clarified his own knowledge of the housemate’s health-related internet activity: “I had to help him with Medicare and his, all that health care. So I don't think he's ever logged into those without me." Another participant responding as a proxy for an unrelated housemate answered “don’t know” to the question about looking for medical information.	No recommendation needed.
MEDREC Production and Round 1 Wording [Do you/Does anyone in this household, including you,] access health records or health insurance records online? Yes No	No problems were detected with the wording of this question. One participant who is a college student living at home initially answered no but changed their answer because they started thinking about a parent who is a doctor and who uses “a lot of services for work and research.”	No recommendation needed.
MEDMON Production Wording [Do you/Does anyone in this household] use an electronic health monitoring service that collects and sends data to your doctor or health care provider through the Internet? Examples include connected devices that monitor vital statistics, blood glucose levels, or blood pressure. (1) Yes (2) No Round 1 Wording [Do you/Does anyone in this household] use an electronic health monitoring service that collects and sends data to your doctor or health care provider through the Internet? Examples include connected devices that monitor vital statistics, blood glucose levels, or blood pressure. (1)Yes (2)No	The concept of health monitoring devices was salient for all participants, including those who do not have such devices or know anyone with them. The most common examples given were blood glucose and heart monitors, and some also mentioned other devices like sleep monitors and smart scales. All participants attended to a critical part of the question, “sends data to a health care provider through the internet.” For example: “That would be like a wearable device, where all the information goes to like a Fitbit or glucose monitor, and that information goes to a doctor.” "Like a glucose monitor or something like that, that's connected to your doctor." “I was thinking about like high blood pressure monitors that collect the data and send it to the doctor, or if you have diabetes the device can collect data. We don't have any of that, that sends it automatically to the doctor.” "Neither one of us wears any type of monitor… Any… data or test results, I'm the one that handles all that, and I just send that directly through my doctor's portal or admin, like directly." Nearly all participants said the question was easy to answer. Only one person expressed confusion, and only temporarily. This participant was thinking of the Apple Watch because it collects health data, but decided it was not included because it does not send data to health care providers by default. Another participant who answered no as a proxy for an unrelated housemate indicated that they found the question to be intrusive and a violation of confidentiality: "If I didn’t know them and they did have one, yeah, I mean it would be kind of intrusive… [If] [name of housemate] has a medical device and that's kind of breaking confidentiality of HIPPA, so I don't like it.”	We recommend removing some extraneous text to make the question simpler and easier to administer: [Do you/Does anyone in this household] use a device that collects and sends health data to a health care provider through the Internet, such as a heart monitor or blood glucose monitor? Do not include smart watches or other devices unless they send data to health care providers.

1 The relationship to the participant was their unmarried partner’s child.

9