Five Main Approaches to Assessing Interdisciplinarity

Bethany Laursen 

Abstract: Many science teams are interdisciplinary, combining expertise from several bodies of knowledge. It these cases, it is crucial to assess interdisciplinarity as part of an overall assessment of the team and its outcomes. But, assessing interdisciplinarity remains onerous and mysterious. Interdisciplinarity is practiced in myriad ways, which makes it hard to know which criteria are important for assessing it in any given case. Further, as we have discovered in related research, assessors struggle to link criteria to appropriate measures, methods, standards, and, where needed, evaluative judgments. Too often, the lack of consensus and clarity around assessment can mean that interdisciplinary efforts are never assessed at all, leaving practical decisions under-informed and expectations untested.This presentation extends findings from a companion study presented at SciTS 2020 that systematically reviewed the ID assessment field from 2000-19 to understand why interdisciplinary assessment challenges persist. Engaging with the same extensive dataset here but in a new way, we employ a cluster analysis that condenses 1,006 published assessment designs into five main assessment approaches. Together, these five approaches collectively represent the first twenty years of the ID assessment field. We present each approach and offer guidance for using each one, including potential settings in which the approach could be most useful and ways it could be modified to overcome challenges and increase rigor. We conclude with reflections on what is not yet prominent in the literature and note opportunities for innovation. Overall, our empirical synthesis provides a more accessible, big-picture understanding of the vast and rapidly expanding field of interdisciplinary assessment. It serves as a brief, robust guide to the field as it has been researched and practiced in the first twenty years of the 21st century. The synthesis makes the field more approachable for practitioners while also sparking new questions for assessment researchers. An interactive online visualization helps users explore these five assessment approaches, which we term 'pathway profiles.' The underlying dataset is available in a repository. We look forward to audience discussion about which of these five approaches they have used and how, what assessment practices have not made it into the literature, and what opportunities for research and innovation they see among and between the five main approaches presented here.

Mitigating Challenges of Collaborative Scientific Teams through Ethnography

Eduardo Piqueiras

Abstract: Background: Teams embody novel scientific opportunities, but collaborative research can also create interpersonal, epistemological, and institutional challenges that can slow progress and present unique difficulties in achieving the goals driving a team-based approach to research. This study employs ethnographic methods to further discussions in the team science literature about how barriers, tensions, and challenges arise and affect the scientific conduct of active teams. Objective: We argue that by employing ethnographic methods and epistemologies, team science scholars can advance research on collaborative science, and practicing teams can locate a self-evaluative method to reflect on their performance and relationships. To do this, we engage ethnography as both methodology and epistemology to reveal how barriers to teamwork appearing in various contexts and at multiple scales may hinder team science's collaborative potential.Methods: We conducted a multimethod ethnographic study of a multi-sited, interdisciplinary research team through participant observation of weekly meetings (in-person and virtual), document analysis of proposal and funding documentation, and day-to-day management of logistics of the project were all included. A focus group and follow-up semi-structured interviews with nine team members, along with a post-interview anonymous online survey, investigated the roles, relationships, and tensions that emerged within an interdisciplinary, federally-funded study.  Results: Qualitative analysis indicated that barriers faced in the process of team science are often multi-scalar and converge at three primary levels: (1) academic culture, (2) institutional structures, and (3) interpersonal dynamics. We explore how team members manage, normalize, and overcome challenges within these distinct but overlapping contexts.  We conclude that the in-depth, nuanced, and contextual analysis of the ethnographic approach can help both researchers of team science and active teams learn how to improve the study and performance of scientific collaboration. An ethnographic approach has the advantage of providing a holistic and more complete picture of team science implementation as it is experienced by real-world teams.Conclusion: Our findings suggest that ethnographic studies of team science can uncover and mitigate the particular barriers facing collaborative research teams with an in-depth understanding of the contextual factors that affect active teams.

How does Social Network Structure Predict Team Readiness? A Case Study of Three Interdisciplinary and Interinstitutional Research Teams

Anne Mook

Abstract: As research projects become more ambitious and complex, large interdisciplinary teams are required to understand and improve modern-day challenges. We use sociocentric network and Likert-scale data from three large interdisciplinary and interinstitutional research teams to assess to what extent social network structure predicts team readiness. To further our understanding of successful team dynamics, we use principal component analysis on a twelve-item Likert-scale matrix to validate the underlying three dimensions of team readiness, namely excitement, trust, and contribution. To assess how network structure (density, closeness, betweenness, and centralization) affects the three dimensions of team readiness (excitement, trust, and contribution) we use exponential random graphs models (EGRMs). We also include demographic measures (gender, race, and seniority) to assess whether team diversity has an impact on team readiness. We find that density and diversity are strong predictors of all dimensions of team readiness. This finding implies that future teams should pay special attention to building dense social connections and recruiting team members with a variety of demographic backgrounds to form an effective team.