Measures are survey instruments, items, or scales that operationalize constructs of interest (e.g., behaviors, attitudes, or skills). In other words, measures transform behaviors, beliefs, or skills into numbers and categories (Mathison, 2005). When studying behavioral or educational interventions, like those collectively investigated by National Research Mentoring Network (NRMN) research teams, survey measures play a central role in evaluating the efficacy of individual interventions and their impact on participants’ attitudes, skills, and future behaviors.

Developing an approach for identifying, implementing, and evaluating common measures that all research teams within a research community agree to is one of the most challenging tasks for a coordination center, yet it is one of the most critical aspects towards realizing collective impact (Kania & Kramer, 2011; Panjwani et al., 2023; Stachowiak et al., 2020). When multiple research teams use common measures in a coordinated fashion, they move beyond the investigation of single interventions and can assess the efficacy of programming or approaches collectively. The use of common measures benefits individual research studies as well as research consortia, as it increases the power beyond individual research studies and provides a common framework of prioritized measures with validity evidence (Sagrestano et al., 2018; Sancheznieto et al., 2025).

One role of a research coordination center—like the NRMN Coordination Center—is to coordinate the selection and use of common measures for use across research studies. This includes the early establishment of constructs of interest across research teams and the selection of a small number of measures that all research teams agree to use. Once research teams begin data collection, having them change their selected measures and study design becomes difficult, unwieldy, and could invalidate their study controls and constraints, especially where they are collecting data longitudinally. For this reason, it is essential that selection occurs early in the project, ideally before individual research teams begin data collection.

To navigate this process successfully, care needs to be taken to build trust among research teams by honoring the work put into their original plans and processes. Coordination centers must carefully manage differing research interests, goals, and personalities across teams to ensure consensus on, consistent use of, and effective data sharing from common measures. By framing the identification of common measures as a collaborative process, research teams are empowered to contribute meaningfully, allowing them to propose, question, and ultimately reject measures that will not align with their study’s needs. To determine the most appropriate measures, Preskill and colleagues urge collective impact partners to consider what they seek to learn and the extent to which each method can offer high quality data. Once common measures are established, it is important for coordination centers to ensure their implementation is straightforward and transparent (Preskill et al., 2014). In addition, the Institutional Review Board (IRB) processes should align to some degree, and allow the coordination center to access, curate, store, and disseminate the data collected from common measures during later project stages.

In the case of NRMN, the coordination center and the 11 independent research studies were funded at the same time in 2019. When the NRMN Coordination Center initiated discussions with the NRMN research teams about establishing common measures, it was critical to acknowledge that each research team already had independent plans for survey design and measure selection. This chapter will cover how we were able to successfully work with all 11 research teams to decide on a set of common measures in the short time available before data collection began.

While the notice of awards from our funder noted the expectation for NRMN research teams to engage in collective work, the NRMN Coordination Center did not have the authority to enforce compliance with use of specific measures. We acknowledged the challenges research teams faced in designing a funded research study only to later be asked to collaborate with a coordination center and other research teams. We also noted the varying experiences, attitudes, and comfort levels research teams had with the common measure process, including adjusting their selected measures and sharing data. We understood these factors were also influenced by their varying levels of familiarity and trust with members of the NRMN Coordination Center team. Because of this, we prioritized transparent, regular communication and made efforts to cultivate relationships with each research team to build trust from the beginning. This included underscoring our respect for each team’s independent research project, research design, data collection plans, and measures during our initial meetings with the principal investigators and team leaders for each research team (see Chapter 3).

We began discussions around common measures by asking research teams to share their research plans so we could identify relatively easy areas for synergy and collaboration. While not all teams were comfortable sharing their plans with us, we were able to identify common measures from plans shared by roughly half of the 11 studies. By starting these discussions early, in the first three months of the grant, we were able to communicate a list of common measures with research teams before their research plans solidified and they began data collection. However, this also meant that some research teams would need to adapt their research plans or measures of choice. Although compromises were made to realize the collective impact of the NRMN research community, we acknowledged the effort required to integrate these changes and expressed our appreciation for each team’s willingness to trust us throughout the process. We also reminded research teams that the data collected using the common measures would remain theirs throughout the process and the shared common data would only be curated and made available for use by all research teams in the final year of the grant. We also established a process to ensure that research team leaders would be informed of all planned uses of the shared common data, that shared common data would be used only with their explicit approval, and that all NRMN research community members would be acknowledged in any publications using the shared common data.

To ensure the greatest potential impact, we knew the work around establishing common measures should be done within the first few months of a program. This gives research teams a chance to adjust measures if needed and include them in their first rounds of data collection. While this timeline may not always be feasible, it is worth the effort when possible.

First, the NRMN Coordination Center established a timeline for finalizing common measures before the first round of data collection. We asked each research team when they would start recruiting participants and collecting data. Recruitment started as early as month 1 and as late as month 10 in the first year. Data collection was expected to begin by month 2 for one research team, with 9 of the 11 research teams collecting data by month 6.

Second, we developed a short but comprehensive list of community-level indicators that could be measured across research studies (Kania and Kramer, 2011). To develop this list, we had to identify the measures each research team planned to use. This allowed us to discern which measures were common across research studies and ultimately select a small set of common measures for the NRMN research community. Luckily, many research studies shared similar goals, as they all were investigating some form of mentorship intervention and were interested in its impact on career progression.

Next, we generated a list of constructs we understood the 11 research teams to be assessing. We supplemented this list with measures from Phase I of the Diversity Program Consortium (DPC), the original NRMN research study grant proposals (if available), and communication with research teams.

Then we launched a survey to identify what constructs each research team planned to assess and how critical it was for them to use a specific measure to assess each construct. To obtain this information, our survey asked research teams to review each construct, confirm whether they planned to assess that construct, and if so, indicate whether the common measure provided was appropriate or whether they wished to provide an alternative (see Appendix 5: Template for Common Measures Survey). Example constructs not including demographic and background measures included:

• Science identity
• Mentoring competency
• Research self-efficacy
• Mentorship quality
• Developmental networks
• Student stress
• Mentoring self-efficacy
• Joy in work
• Grantsmanship self-efficacy

Results from the survey provided a clear picture of where there was already consensus on measures, where there was willingness to compromise, and where there was concern about changing a given measure from what a research team had originally planned (see example of results in Figure 4). While most research teams had preferences about the measures they had originally selected, they were open to using other measures so long as they had been tested and had some evidence of validity (Reeves & Marbach-Ad, 2016). When strong evidence was lacking in the literature, but a strong rationale could be made, research teams were willing to commit to using suggested measures. In the few cases where it was not clear which measure to use (for example, satisfaction with quality of mentorship, see Figure 4), we organized working groups to facilitate discussion and finalize which common measures would work best for the NRMN research community. We also provided guidance that research teams incorporate additional measures beyond the selected common measures for a particular construct, if doing so better served their research goals.

Figure 4. Example survey results for sample measures to address two constructs. Research teams were asked to note whether or not they were measuring the same construct as the measure provided, and whether or not they had their own measure to suggest. Response options included: Not relevant - our study is not addressing this construct, so the common measures chosen for it are not relevant to our study. May address, open - our study may decide to address this construct, but we are willing to use a common measure others choose. Important, will work - this construct is important to our study and the suggested common measure will work for us. Critical, alt - this construct is critical to our study and the suggested common measure will not work for us; we do have an alternative to suggest. Critical, no alt - this construct is critical to our study and the suggested common measure will not work for us; we do not have an alternative to suggest.

In the spirit of transparency, we shared the aggregated data from the survey responses with the NRMN research community, along with a proposed plan for a final set of common measures. We invited research teams to meet with us to discuss their questions and concerns. This time was especially important for us to build trust with research teams who were being asked to adjust their original plans. For example, some teams discussed the possibility of including two measures for a construct – one they had originally selected and one being used across research teams. Other discussions focused on the NRMN Coordination Center providing assurance that the common data would not be used without the research team’s permission.

Figure 5. Common measure usage across career stages. Visual representation similar to those on our NRMN Research Community Google Site showing planned usage of common measures across the 11 research teams.

Towards the end of the grant, our list of common measures migrated to a public data portal housed at https://nrmndata.sites.wisc.edu/nrmn-phase-ii-common-measures/. These libraries (both internal and external) include information such as:

Coordination centers benefit from establishing processes that support data sharing and enable its use in future research. We developed a process that aggregated data collected by each NRMN research team using common measures into a single dataset to be shared with the NRMN Coordination Center (see chapter 5). This process included providing research teams with templated language for their (IRB) protocol that allowed data sharing and storage and provided participants with the option to consent to the use of their data in future research (see Appendix 6: Template for IRB Protocol Language and Appendix 7: Example of Internal Data Sharing Plan).

To seamlessly coordinate this process, we communicated early with research team leaders and their research managers about the need to coordinate with the NRMN Coordination Center when writing and revising their IRB protocols (see Chapter 5). We made ourselves available to answer questions as research teams navigated this process with their local IRB offices. We provided clarity and transparency about what data research teams would need to share (e.g., data collected using common measures), how data would be used, and how data would be shared in the future. We also shared descriptions of potential future collaborations using the common data to generate excitement and foster buy-in for the collective effort we were initiating. We asked each research team to share their final IRB protocols and data sharing agreements to ensure they included language permitting future use. Additionally, we worked with one research team whose participants had not consented to future use of their data, in order to review the consent processes that would be needed moving forward.

Conclusion

Acknowledgements

We acknowledge Dr. Jenna Rogers for her work in establishing processes and leading efforts around common measures and data sharing early in the NRMN Coordination Center project. We acknowledge Emma Dums for her contributions to the common measures work of the NRMN Coordination Center and contributions to this chapter.