The following case studies give examples of recent projects that have utilized a variety of M3 features with a wide range of complexity.
Survey of Doctorate Recipients (SDR)
Survey of Doctorate Recipients (SDR), a longitudinal study, involves keeping track of Ph.D. recipients and their careers over many years through a multimode survey (web, mail and CATI) that is conducted every two years. The National Science Foundation and National Center for Science and Engineering Statistics, as the clients for this project, wish to learn demographic and career history information about those who have earned a research doctorate degree in a science, engineering or health field from a U.S. academic institution. SDR tracks responses of those degree holders from the point of graduation until age 76. The sample data is refreshed every two years to include newly minted degree earners.
- Tracing
- Inbound Email Management
- Multi-Pin Authentication
- Ticketing
- Help Desk
- Collection of operational paradata
In 2021, SDR utilized several newly built M3 features. Multi-PIN authentication created a unique PIN for each email or mailing, so participants would use that PIN to log into the survey website. M3 tracked those responses so staff could easily learn which method of communication garnered the best response. Tracing features gave staff a way to log all research done to locate correct addresses or email addresses for participants who may have moved or changed jobs over the last two years. Study staff were able to learn which online sources provided the best information as well as increase response rates for those missing participants. Additionally, M3 added automated scoring to each mailing address, email address, and phone number to better determine the best contact types for each case. The collection and storage of operational paradata was utilized for reporting/monitoring as well as exported for use in adaptive design models.
“In the 2019 SDR cycle, we had to manage all email manually through Outlook and create each EROC and ticket. M3 really reduced the time and burden of doing that. We were able to communicate with other Westat staff about a case using the comments in an M3 ticket, without violating PII.” -Help Desk Staff
The Help Desk functions allowed staff to log EROCs (Electronic Record of Communication) in M3 to track every point of contact with participants, with customized reasons for each phone call or email. Participants needing a new PIN to complete a survey could be helped from within M3, or they could be transferred to a CATI interviewer to complete a telephone survey. Inbound email management automatically sorted emails into a variety of staff selected folders, creating EROCs and tickets if relevant. This was a tremendous time saving for help desk staff, allowing them additional time to resolve issues, escalate bounced email to tracing staff, and assist participants.
National Survey of Older Americans Act Participants (NSOAAP)
Over many years, Westat has conducted the National Surveys of Older Americans Act Participants (NSOAAP). The 2022 NSOAAP marks the 16th iteration of the study for the Administration of Community Living! NSOAAP was designed to study people who receive services funded by ACL Administration on Aging's Older Americans Act (OAA). The OAA promotes the well-being of older individuals by providing services and programs – such as meal delivery, transportation, case management, and homemaker services – designed to help them live independently in their homes and communities. The purpose of NSOAAP is to obtain performance outcome information that demonstrates the effect of services and illustrates client-reported quality of service. Data collectors conduct extended service-specific telephone interviews with the clients of the services, including caregiver clients.
- CATI Integration
- Participant Data Management
- EROCs
The NSOAAP project utilizes M3 to facilitate inbound calls. Cases are loaded into a CATI system, Blaise Mainline, which is accessed by approximately 35 data collectors. Identical data is also loaded into M3. Study participants (clients to be interviewed) call in to request to speak with a data collector to either complete the interview, schedule a future appointment, provide updated information, and/or refuse. As data collectors answer phone calls, they look up the participant in M3 to find their contact information, screening information, and participant ID. They log EROCs (Electronic Records of Contact) and then enter the participant ID into the Blaise CATI system to begin the formal interview process. An integration between Blaise and M3 allows nightly transfer of case outcomes, updated contact information, and other survey data from Blaise into M3.
This integration, among others, meets the business goal of standardizing inbound lookup. When M3 is used in conjunction with CATI systems, data collectors only need to learn one study management system (SMS) and have their accounts and permissions managed in a consistent way across multiple projects. This standardization reduces administrative costs and development time needed.
“Prior to using M3, every group used a different SMS to manage studies. So CATI staff had to be trained on each one. Using M3 made it standard.”
Infant Feeding Practices III (IFPS-III)
Infant Feeding Practices III (IFPS-III), a longitudinal study for the CDC, tracks feeding and health of children from birth through age two. Pregnant women were recruited through social media ads to complete a web-based screener survey between August 2021 – May 2022. If they passed the screener, they were eligible to complete the enrollment process, which included a phone call to verify the participant’s identity. 4,200 participants were enrolled and began to receive invitations to complete periodic surveys, starting from prenatal through 1 month, 2 months, 3 months of age, and so on until age 2. For each successfully completed survey, participants could choose to receive their incentive via check or Amazon gift card. 70% of this audience selects Amazon. The study will continue through 2024.
- Incentives(Amazon and check)
This study utilizes a separate internal study management system (SMS), with web surveys. A web participant portal allows participants to view incentives and switch preferences from Amazon to check before each survey. At launch, while waiting for M3 integration to be completed, the study initially bought a batch of Amazon codes and stored them in a spreadsheet for access by the systems, and as a result would occasionally run out of codes. They then implemented M3 to use its Amazon integration. Now, after survey completion, the program calls M3 to generate a code and display it to the participant. The new process has worked smoothly, with only one participant question about Amazon codes so far.
“Amazon incentives are very cost effective, with no added fees and maintenance. It works for this particular population well.”
Fragile Families, 2020-2022
The Fragile Families & Child Wellbeing Study, conducted by Princeton University and Columbia University, includes a sample of nearly 5000 children born between 1998 and 2000. The study consists of a core survey with the focal child/young adult (YA) and a primary caregiver (PCG). Mothers were interviewed shortly after birth and fathers were interviewed at the hospital or by phone. Follow-up interviews were conducted when children were approximately ages 1, 3, 5, 9, 15, and 22. Additional study cycles collected data on a subset of parents and children via in-person clinic visits.
The study for the 22-year-old cohort launched in 2020. Study staff tracked down YA and PCG participants to engage them in web interviews, home visits, and clinic visits. M3’s database handled not only data about YA participants, but also tracked relationships with PCG respondents. The communications protocol was centralized in M3, utilizing hardcopy mailings, email, and text messages. Communications, such as survey invitations and reminders, were sent anytime new contact information was identified via internet tracing activities. Incentives were distributed via debit cards sent to participants after survey completion. EROC functionality was available to interviewers, tracers, and home office help desk staff, to enable a full picture of the history of each case.
- Participant contact information updates and QC
- Incentives
- Communications protocol: mail, email and text
- Web surveys
- EROCs
- Tracing
- Scheduling clinic appointments
- Clinic workflow
A new feature, M3 scheduling, allowed staff to schedule appointments for participants at in-person health clinics across the country. Schedules were visible in list format, as well as calendar format, with simple rescheduling options available. M3 workflows were designed to log every step of a clinic visit from check-in to check-out. Participant contact updates allowed participants to update their own contact information and the contact information of their household members.
“The core features were really helpful. Being able to expand data tables, rearrange columns, and export data is powerful.”