CTSA

Using Instructional Design Principles to Engage Citizen Scientists as Clinical Research Partners

Background: Citizen Scientists help bridge the gap between researchers and community members. These stakeholders provide input on many aspects of clinical research studies, from general consensus feedback such as selecting a logo for a fitness app, to providing suggestions on how to recruit patients more effectively. Citizen Scientists bring fresh perspective to research studies and offer insights typically overlooked by research staff. Depending on their prior experiences within the health care system or in their professional lives, Citizen Scientists may initially not understand fundamental elements of the research process. This knowledge gap can create a communication barrier between the stakeholder and researchers, and limit actionable feedback for implementation.

Methods: Working with experts in educational research and instructional design, a curriculum was created to train new community stakeholders. The curriculum consists of seven modules with different themes relevant to clinical research. The modules each contain didactic lessons with video tutorials presented by subject matter experts. The curriculum was created using pedagogical best practices such as use of an instructional design model to guide the process, interactive assessments with color-coded feedback, learning objectives, and multimedia content delivery. The Citizen Scientists worked with the research team to help create the materials, from reviewing assessment language to starring in the videos. The curriculum was pilot tested with IRB approval among Citizen Scientists between July 2017 and January 2018 using the Canvas learning management system.

Results: Citizen Scientists were divided into two groups: established members and new members. The cumulative score for both groups across all modules was 81%, ranging from 76% for the clinical and translational science module to 98% for the stakeholder engagement module. Notable differences were observed in the cultural competency module, where younger participants scored higher than older participants (90% vs 77%), and in the orientation to citizen science module, where established Citizen Scientists scored higher than new members (93% vs 81%). Participants also completed an evaluation that asked questions about comprehension, video quality, and assessment clarity. Most (75%) participants felt it was easy to understand the information in the videos. The majority of participants (62.50%) felt the assessment items matched the learning objectives and 75% felt the assessments were comprehensive. The evaluations asked for unstructured comments as well, and participants used this to offer concrete feedback for future iterations of the curriculum.

Conclusion: After completing the course, Citizen Scientists received sustained engagement certification and are now using this knowledge on new research projects. The Citizen Scientist curriculum was a well-received approach to educating community stakeholders in aspects of clinical research. When community stakeholders are knowledgeable about research terminology, their ability to collaborate with researchers, and provide a community perspective while remaining cognizant of how research works, is enhanced. Having a more in-depth understanding of the research process fosters collaborative efforts and improved communication between the stakeholder and researchers. Through an understanding of how clinical research works, community stakeholders are better able to offer critical feedback to help advance health care.

SciTS Presentation: Using Instructional Design Principles to Engage Citizen Scientists as Clinical Research Partners

An Integrated Approach to Promoting Team Science Principles Across a CTSA

The application of the principles of team science have been an important part of the CTSAs almost from the inception of the program. Implementation of this mandate has been largely left to the discretion of individual centers.  As we developed our approach to integrating team science into our CCTST we identified the need for a comprehensive model included multi-level training, team consultations, rewards for successful collaboration and creation of high-functioning teams, mechanisms to overcome institutional barriers to collaboration, and a shift in the institutional research culture that currently accepts dysfunctional and sub-optimal teams as the norm.

We have been building a cross-institutional, comprehensive approach to transdisciplinary research to create a sustainable impact on the quality of research teams, collaboration and the culture of our CTSA partners. Our approach includes initiatives in the following areas:

Administration: The director of the Center for Improvement Science (CIS) sits on the CCTST Executive Committee. He also consults directly with the University of Cincinnati Sr. VP for Research and our health systems. This high-level administrative network allows the CIS to understand broad-based institutional needs and propose cross-institutional solutions to common problems.

Education: The CIS provides on-going, multi-level education in the form of workshops (from team science basics to specialized topics), a graduate-level course in Team Science and Collaboration, on-demand education (one-hour events, grand rounds, workshops, and community workshops). These are broadly advertised to CCTST members (over 4,000) and through institutional training announcements. We typically have over 15 events per years with approximately 500 participants. We are currently co-sponsoring workshops with the university’s Staff Success Center.

Consultation: The CIS provides consultations for individual investigators, teams, and administrative units. Time periods vary from a single session to on-going relationships that include team assessments, customized training, and “touch base” meetings. We are averaging over 12 consultations per year.

The Collaboration Network: The CIS operates a weekly collaboration session Web-Ex.  The purpose is to help investigators connect with potential collaborators and to learn more about colleagues and resources across the CCTST. We have monthly “member spotlight” sessions and quarterly “special topic” sessions that are well advertised across all CCTST institutions and the community. Our most recent quarterly meeting on Aging included over 50 people from the CCTST and community partners.

Integration Committee: This is a group of top investigators and core directors from the CCTST.  We invite new and mid-level investigators to present their current research and challenges. The purpose of the meeting is to offer career advice, connect investigators with collaborators and resources, and to provide regular follow-up contact to support their needs. This has been in place for over five years. We currently follow over 75 investigators.

Monitoring and Evaluation: We have been gathering data on individual investigators, teams, and funding programs to assess progress in promoting a collaborative research culture, educating and supporting investigators, and increasing the effective use of team science principles. As part of this presentation, we will provide the latest evaluation data that measure progress and effectiveness of our efforts.

SciTS Presentation: An Integrated Approach to Promoting Team Science Principles Across A CTSA

Implementing a Continuous Quality Improvement Intervention in a Clinical and Translational Research Network

Purpose: To determine strengths and areas for improvement for the Great Plains IDeA Clinical and Translational Research (GP CTR) network and develop insight into these needs through key informant interviews of Network Steering Committee members over two years.

Background:  Clinical and translational research networks have been funded and developed to provide infrastructure to support breakthroughs in disease treatment, prevention, and health promotion. There is a growing body of literature on the effectiveness of scientific teams across and within these networks, but limited information on the use of systems thinking tools for continuous quality improvement (CQI). Measures such as the Internal Coalition Effectiveness© (ICE), a validated instrument designed to aid community coalitions with identifying organizational strengths and areas for improvement, can be used to collect longitudinal data for reflection and action planning.

Methods: This project examined the implementation of a mixed methods CQI framework designed to identify strengths and areas for improvement across domains measured by the ICE: Shared Social Vision, Efficient Practices, Knowledge and Training, Relationships, Participation, and Activities, across the GP CTR network. The ICE instrument was adapted to measure the internal governance of the network, and then used to develop an interview protocol.  A convergent parallel mixed methods design was used to: (1) Administer the ICE to GP CTR Steering Committee members and grant-funded faculty/staff (n=36); (2) Conduct semi-structured interviews with GP CTR steering committee members (n=6). ICE survey data and interview data were analyzed and compared. We hypothesized the proposed framework is an effective method to identify opportunities for changes that will facilitate the translation of research evidence to improve population health outcomes.

Results: A mix of facilitators and barriers were identified from survey and interview responses. ICE survey respondents rated network effectiveness positively in all domains (mean= 5.30). The highest rated construct was Knowledge and Training and the lowest rated construct was Participation. Key informant interview responses validated the survey results regarding the positive responses across ICE domains. A new code emerged as unique “Accomplishments” were highlighted, including comments around professional development for pilot and scholar awardees, the quality of the annual GP CTR scientific meeting, and improvements in quality and number of applications across network-specific funding calls. Participation emerged as a predominant category with respondents indicating a need for continued focus on transparency, engagement in decision making, and sustained communication. Additional recommendations included continuing to facilitate networking opportunities and strengthening support mechanisms across GP CTR institutions.

Conclusions: A mixed methods CQI framework may be a useful method for transdisciplinary research networks to identify structures and processes that facilitate or inhibit the implementation of transdisciplinary research activities. We supported our initial hypothesis and identified feedback garnered from the ICE survey and the key informant interviews that can be used to inform changes in governance to promote advancements in translational science across the GP CTR network. Clinical and translational research network members enjoy the knowledge sharing and training opportunities that come with network membership and are looking for opportunities to contribute to the development and achievement of the network’s vision.

SciTS Presentation: Implementing a Continuous Quality Improvement Intervention in a Clinical and Transitional Research Network: Monitoring Governance to Enhance Stewardship

Background: The functional perspective of groups highlights the importance of understanding the relationship between internal factors that emanate from within a group (e.g. diversity and composition); external circumstances that are intractable (e.g. workload and time pressures); and the quality of interactions between team members during a given task. This premise has not yet been explored in the context of patient-reviews in multidisciplinary tumor boards (MTBs). We have therefore set out to gain better understanding of how external circumstances and internal factors relate to the quality of interactions in the disciplinary diverse group of health professionals that constitute a MTB.

Methods: Breast, colorectal and gynecological tumor boards from across 3 academic hospitals in the UK were video recorded over 12-weekly meetings, encompassing 822 patient-reviews over 55h of MTB real-time work. Quality of interactions for each of the 822 patient-reviews was assessed using a validated instrument, namely, Bales’ Interaction Process Analysis, that captures frequency of task-oriented and socio-emotional interactions during a team task. We also measured the following items:

• Group size (number of core team members present at any one patient-review),
• Disciplinary diversity (number of core disciplines present at any one patient-review),
• Disciplinary distribution (number of core members present per core discipline), and
• Gender balance (more males, more females, equal number of males and females)
• Time and workload pressures (ratio based on the time left to review patients on the MTB agenda divided by the number of patients left to be reviewed).

The relationship between the variables was assessed using partial correlation analysis controlling for team/tumor type and the complexity of the patient reviewed (using a validated tool, MeDiC).

Participants were senior specialist physicians and nurses across the 3 MTBs (N=41 team members in total) including;

• 6 radiologists with on average 12 years’ experience,
• 5 histopathologists with on average 11 years’ experience,
• 12 surgeons with on average 8 years’ experience,
• 6 oncologists with on average 6 years’ experience, and
• 12 cancer specialist nurses with on average 7 years’ experience

Results: Disciplinary diversity and group size were significantly associated with reduced task oriented interactions such as sharing of patient related information (r=-.08, N=833, p<.05; r=-.14, N=388, p<.05; respectively), and increased seeking of such information from the team (r=.13, N=833, p<.05; r=.08, N=833, p<.05; respectively). Group size and more male team members present during patient-reviews were also associated with increased seeking of clinical opinions into care planning (r=.18, N=833, p<.05; r=.21, N=833, p<.05; respectively), while the opposite pattern was evident with more females present (r=-.16, N=833, p<.05).

Negative socio-emotional reactions such as antagonism were heightened with disciplinary diversity (r=.08, N=833, p<.05) and tension also increased with team size (r=.11, N=833, p<.05; r=.249, N=833, p<.05, respectively). Patient-reviews with more male team members present were associated with significantly more tension (r=.01, N=833, p<.05) and fewer disagreements (r=-.11), while the opposite relationship was evident when more females were present i.e. more disagreements (r=.10, N=833, p<.05) and less tension (r=-.11, N=833, p<.05).

Time-workload pressure was associated with reduced task-oriented interactions in particular providing fewer specialist opinions during patient-reviews (r=-.15, N=833, p<.05), and reduced positive socio-emotional interactions between team members such as solidarity (r=-.14, N=833, p<.05) and agreeableness (r=-.08, N=833, p<.05).

Discussion: Disciplinary diversity, team composition and time-workload pressures are important elements to take into consideration when evaluating MTBs since we found that they affected the quality of team interactions in multiple different ways. It is arguable that smaller size teams with only core disciplines present and streamlining workload to reduce time-workload pressure on MTBs may be strategies to apply in order to ensure better team functioning and service quality in these clinically important specialist teams.