Supporting Statement Aspects of Postgraduate Researcher Experience

OMB Control #0693-0033

Expiration Date: 09/30/2025

NIST Generic Clearance for Program Evaluation Data Collections

Aspects of Postgraduate Researcher Experience Scale NIST Survey

FOUR STANDARD SURVEY QUESTIONS

1. Explain who will be surveyed and why the group is appropriate to survey.

Background

High attrition rates in science, technology, engineering, and mathematics disciplines are an ongoing problem. Most studies to date have focused on undergraduate and graduate student attrition, with a particular emphasis on students’ attributes, undergraduate preparation, and mentoring relationships. Systematic studies looking into the experiences of Postgraduate Researchers are few and far between. One example is a recent post-COVID survey organized by Nature. The responses to Nature’s survey uncovered a list of issues that Postdoctoral Researchers face: “high pressure, long hours, relatively low wages compared with those for analogous research positions outside academia, and pervasive job insecurity”. Another example is a study that looks into challenges that female postdocs face in academia (R. Ysseldyk et al. Front. Psychol. 10, 1297, 2019). The study revealed that a sense of belonging and the overall work environment are critical for female postdocs to overcome the decision to leave academia.

A recent pilot assessment conducted at the National Institute of Standards and Technology (NIST) offers additional insight into the challenges and support structures for postdoctoral and early-career researchers. Using the Aspects of Postdoctoral Researcher Experience Scale (APRES) survey, the study conducted within one of the six Organizational Units at NIST found that while individualized mentorship and research alignment were generally strong—especially through one-on-one interactions with advisors—other forms of support were less consistent. Respondents reported gaps in formal training on professional skills such as leadership, mentoring, and time management, as well as limited opportunities for structured networking or community building across divisions. Many participants expressed concern about the lack of inclusive social activities and dedicated spaces to foster a sense of belonging, echoing broader findings in the literature about the importance of community and environment. Although most respondents felt financially secure, with health benefits and research funding in place, about half still experienced uncertainty about completing their training without financial strain. Together, these results suggest that while mentorship at NIST provides a solid foundation, more systematic institutional structures are needed to support the long-term career development and retention of postdoctoral researchers (J. P. Zwolak and R. P. Dalka. NIST SP 1337, 2025).

The goal of the proposed study is to assess the experiences of Postgraduate Researchers at NIST. This study will be used to inform what changes might be necessary to ensure that the time Postgraduate Researchers spend at NIST prepares them well for the next steps on their career path.

Target population

This survey will be administered to all Postgraduate Researchers, including Federal and non-Federal employees. Non-Federal employees may include guest researchers, contractors, and other early-career researchers working at or for NIST. It is important to survey all Postgraduate Researchers to ensure all viewpoints regarding experiences at NIST are considered and to capture an unbiased sample. For the purpose of this work, Postgraduate Researchers are defined as early-career researchers no more than 7 years from receiving their final degree. This includes NRC postdocs, federal term employees, and guest researchers (US citizens and foreign nationals) supported via the PREP program or other agreement with external institutions.

2. Explain how the survey was developed, including consultation with interested parties, pre-testing, and responses to suggestions for improvement.

The original “Aspects of Student Experience Scale” (ASES) survey was developed by researchers at Western Michigan University to study what factors that foster a supportive environment for graduate students. The questions included in ASES were developed based on prior literature and the American Physical Society Bridge Program (APSBP) recommendations, revised based on APSBP feedback, and subjected to psychometric evaluation. It has been demonstrated that ASES meets the standard criteria for divergent validity, discriminant validity, and internal consistency. The Aspects of Postgraduate Researcher Experience Scale (APRES) Survey is a revised version of the ASES survey, modified to be relevant to Postgraduate Researchers. The revisions include replacing references to the “department” with “OU” or “division”, as appropriate, replacing “graduate student” with “Postgraduate Researcher”, “courses” with “professional development”, etc. The resulting APRES was discussed with researchers having prior experience working with ASES. We have made every attempt to ensure the resulting APRES survey is as anonymous as possible, while still ensuring we are able to capture data that will enable us to tease out where we may have concerns about the Postgraduate Researchers' experiences at NIST. The analysis plan is described in question 4.

3. Explain how the survey will be conducted, how customers will be sampled if fewer than all customers will be surveyed, expected response rate, and actions your agency plans to take to improve the response rate.

The survey will be administered annually and electronically using Google Forms (or Qualtrics). The survey will be anonymous (i.e., no identifiable information, such as email address or NIST aliases, will be collected), and only aggregated data will be analyzed. An email invitation, provided below, will be sent to all Postgraduate Scholars. Given NIST’s Postgraduate Researchers total staff is estimated to be about 500 people, the potential burden for this survey is estimated at: 500 respondents * 15 (minutes) / 60 (minutes in an hour) = 125 burden hours. The expected response rate is approximately 40%. We intend to send follow-up reminders and encourage participation at staff meetings and town hall events.

Proposed survey invitation email:

Dear Postdoctoral and Early-career Researchers,

We invite you to consider participating in a survey about your experiences as an Early-career Postgraduate Researcher at NIST. The purpose of this survey is to assess how Early-career Postgraduate Researchers find themselves at NIST and to identify areas where their experiences need improvement. The results of this survey will be shared with NIST leadership and will be used to inform training programs and policies.

For the purpose of this work, Postgraduate Researchers are defined as early-career researchers no more than 7 years from receiving their final degree. This includes NRC postdocs, federal term employees, and guest researchers (US citizens and foreign nationals) supported via the PREP program or other agreement with external institutions.

No identifying information about you will be collected. Survey data will be summarized, and aggregated results will be included in presentations and publications which may be shared with NIST management, staff, and the public. The raw, individual data will be accessed only by the team administering this survey, which is led by Justyna Zwolak.

The survey is completely voluntary and should take you about 15-20 minutes to complete. If you have any questions, please contact Justyna Zwolak, Ph.D., at jpzwolak@nist.gov or 301.975.0527.

Your opinion is very important!

Thank you,

Justyna Zwolak

————————————————————————

Applied and Computational Mathematics Division

National Institute of Standards and Technology

100 Bureau Drive, Stop 8910

Gaithersburg, MD 20899-8910 USA

Phone: (301) 975-0527

Email:  jpzwolak@nist.gov

————————————————————————

4. Describe how the results of the survey will be analyzed and used to generalize the results to the entire customer population.

The survey results will be aggregated and analyzed to identify common themes in responses. These themes may include Postgraduate Researchers’ social and scholarly exploration support, mentoring and research experience, professional and academic development, and financial stability. The proposed analysis plan is listed below. We will be looking for areas where we need to improve Postgraduate Researchers' experiences at NIST. Finally, the results will be shared in an annual report to NIST leadership, staff, and the public. The results will be posted to NIST internal and external websites.

• Preliminary analysis

  • Quantitative: Summary results for each question

    • For multiple-choice questions: count per response

    • For scaled questions: mean response

• Secondary analysis

  • Analysis using the Network analysis approach to Likert-style surveys (NALS) of the full response data.

  • NALS analysis for response grouped by

    • Type of appointment (Federal employee and non-Federal employee)

    • Tenure at NIST

    • Site

    • Prior experience

    • Sex

    • Ethnicity (we may group ethnicity into minority and non-minority if the response rate does not allow a more detailed analysis)

  • Exploratory analysis: comparison between groups

• Future analyses will include comparing these results over time as well as between OU as we implement the survey annually.