LISA CAPUTO SANDY
University of Colorado Anschutz Medical Campus

Pragmatic clinical trials (PCT) aim to inform healthcare decisions and treatment effectiveness based on data from real-world settings and populations. This generally requires that data be collected from a large and representative population, which in turn necessitates minimizing costs and burdens for patients and clinicians that can be associated with traditional clinical research, such as elaborate written informed consent processes. Opt out consent practices are sometimes used as a means to promote patients’ autonomous preferences regarding research participation while reducing both patient and researcher burden. Despite this, little is known about the characteristics of patients who choose to opt-out of research and their reasons for doing so. Therefore, we gathered such information in a large PCT seeking to improve medication adherence via evidence-based text messages employing nudge theory.

This trial included patients from the VA Eastern Colorado Health Care System and Denver Health and Hospital Authority aged 18 – 89 years of age diagnosed with least one cardiovascular condition and prescribed at least one medication to treat the condition(s). Patients that did not speak English or Spanish, did not have a cell phone or home address on file, or did not reside within the State of Colorado were excluded.

Eligible patients identified through Electronic Health Records were sent informational material about the study and were provided an opportunity to opt out. Those who opted out were asked to complete a voluntary survey regarding their reasons for choosing not to participate. Demographic data and reasons for opting out were compared with patients enrolling in the study using chi-squared tests.

Of 9,444 patients eligible for the trial, 895 (9.5%) patients opted out. Of those who opted out, 451 (50.1%) returned the opt out survey. Patients who opted out were more likely to be older, male, white, and non-Hispanic than those that did not (p<0.001 for all). Nearly half (46.6%) of respondents reported concerns about time as a reason to opt out of the study. Perceptions of not needing the intervention (19.5%) and discomfort with technology (18.6%) were the next most common responses. Conclusions: In this PCT focused on medication adherence improvement, less than 10% of patients opted out of participation, with significant differences in age, race, gender, and ethnicity between patients that opted out and those that did not. Future trials may wish to provide additional information to address the most commonly cited reasons by patients for opting out of a low-risk trial. The impact additional patient characteristics may have on opting out will be analyzed in April and May.

JENNA RENO, PhD
University of Colorado Anschutz Medical Campus, School of Medicine

Stakeholder engagement is increasingly expected by funders and valued by researchers in clinical and translational science, yet many researchers lack access to expert consultation or training in selecting appropriate stakeholder engagement methods. Scalable infrastructure could support improvements in stakeholder-engaged research, and self-directed, web-based interactive tools are emerging solutions across clinical and translational research. We undertook an iterative process of design, development, and testing of an interactive web-based tool (henceforth “webtool”) to guide researchers in learning about, selecting and using a variety of methods for stakeholder-engaged research for their grant writing, protocol planning, implementation, and evidence dissemination.

Design Thinking methods were used to engage stakeholders, including investigators from the Anschutz Medical campus and Colorado Clinical and Translational Sciences Institute communities, who are interested in using stakeholder engagement methods for research.

The design and development of the engagement methods webtool was guided by user-centered design processes. We followed the Design Thinking stages described by Ideo.org: Empathize, Define, Ideate, Prototype, and Test. Design Thinking stages are iterative in nature – such that progress from one stage to another often returns to prior stages with new insights. We conducted an environmental scan and literature review, along with investigator interviews, surveys, and engagement expert facilitated group discussion. We formally reviewed and catalogued 29 distinct engagement methods. To develop the webtool we used an iterative design process that included a contextual inquiry approach (low fidelity prototype user testing) and a ‘Think Aloud’ approach (high fidelity webtool prototype user testing) to produce webtool V1.0.

As prioritized during user testing, the Stakeholder Engagement Navigator webtool both educates and guides investigators in selecting an engagement method based on key criteria. Insights from the Empathize and Define stages included: Researchers understand that stakeholder engagement is valuable and want to include it in their research design and implementation; however, researchers are not familiar with stakeholder engagement methods. Based on the environmental scan of comparable tools from the Empathize stage, we developed a modification of an interactive “bubble” feature that displayed results. The V1.0 Navigator webtool filters methods first by purpose of engagement (noted by 62% of users as the highest priority criteria), then by budget, time per stakeholder interaction, and total interactions.

The Stakeholder Engagement Navigator webtool is a user-centered, interactive webtool suitable for use by researchers seeking guidance on appropriate stakeholder engagement methods for clinical and translational research projects.

LINDA SALGIN
San Ysidro Health, University of California San Diego, San Diego State University

Meaningful engagement of stakeholders is at the heart of successful program development and implementation. Community and Scientific Advisory Boards (CSABs) have been frequently used to engage diverse sets of stakeholders to inform research projects. Traditionally, CSABs meet in person, however, in light of the COVID-19 pandemic, many CSABs have moved into the virtual realm, raising questions about the quality of engagement and data collection processes. Our objective is to describe our approach and preliminary findings to adapting ethnographic methods to assess stakeholder engagement in virtual CSABs.

CSAB meetings were hosted via an online video conferencing platform. A total of 33 stakeholders across two CSABs participated in 16 sessions. Seven undergraduate students and two masters-level research staff were trained as direct observers.

Documentation forms were developed to assess the following in each meeting: attendees, time spent speaking and language (English or Spanish), modality used (computer, phone, or both), and types of stakeholder interactions (e.g., interruptions, sharing or requesting information). Documenters participated in a two-hour interactive training led by three implementation scientists along with ongoing debrief meetings after each CSAB for quality assurance and process refinement. Each CSAB meeting lasted two hours and was facilitated by the Global Action Research Center, a social change organization. Documenters were assigned to observe specific CSAB sub-groups and used a combination of live and recorded meetings to complete their documentation forms.

Debriefing sessions led to a number of identified challenges and subsequent methodological refinements. The primary challenges were: ability to accurately document content and technical issues, with recording information about the content of the discussions being the most challenging. The virtual format of the meetings limited ability to document body language and behavioral nuances and lack of ability to record all breakout rooms. Pre-assigning documenters to focus on specific CSAB sub-groups along with the ability to record CSAB meetings for repeated review made documentation more feasible. Data derived from the preliminary content analysis of the documentation forms indicated that the majority (60-70%) of interruptions or comments were related to technical issues.

As research continues to expand use of virtual platforms, we highlight key lessons learned to adapt ethnographic methods to facilitate community engagement through virtual CSAB contexts. Assessing stakeholder engagement virtually allowed for the collection of rich ethnographic data but these adapted methods presented unique obstacles. Prior to embarking on a virtual ethnographic journey, we recommend ongoing trainings including debriefing sessions, and thorough investigation into the functions of virtual platforms before selection.