An Evaluation of the Trauma and Learning Policy Initiative’s (TLPI)
Inquiry-Based Process: Year Three

Suggested Citation: Atallah, D. G., Koslouski, J. B., Perkins, K. N., Marsico, C., & Porche, M. V. (2019). An Evaluation of Trauma and Learning Policy Initiative’s (TLPI) Inquiry-Based Process: Year Three. Boston, MA: Boston University, Wheelock College of Education and Human Development.

(The Abstract and the Executive Summary were taken directly from the “Year Three” Evaluation Report.)

Abstract

This evaluation investigated the impact of the Trauma and Learning Policy Initiative’s (TLPI) Inquiry-Based Process on three participating public schools. TLPI’s Inquiry-Based Process is a whole school effort to create trauma-sensitive school environments. We aimed to (1) analyze participant educators’ reported cultural and organizational change at the school and teacher levels from Year 3 of schools’ implementation of TLPI’s Process, and (2) to use these results to complement understandings generated from Year 1 and 2 outcomes which were set forth in an earlier report by the American Institutes for Research. TLPI’s theory of change is that a deepening understanding of the impact of trauma on learning, and participation in an Inquiry-Based Process of educator empowerment to address school-based priorities, will lead to shifts in thinking and shifts in practice that can become embedded and part of the way the school is run; that is, part of the culture of the school. Thus, the research aim was to glean from participant reports whether and how changes became embedded in the schools’ cultures.

Using an adapted Situational Analysis qualitative research design, we found that leadership and staff reported cultural and organizational shifts in their schools that clustered into four emergent themes:

1. facilitating empowerment and collaboration
2. integrating whole-child approaches
3. affirming cultural identity and promoting a sense of belonging
4. re-envisioning discipline toward relational accountability

Overall, this evaluation provides evidence for profound impacts that schools’ engagement with TLPI’s Inquiry-Based Process, with the requisite level of commitment and focused effort, can have for leadership, staff, students, and families. Lasting changes reported by educators were multi-leveled, and included shifts in both thinking and practice resulting in changes that positively impact school culture. Educators’ reports evidence a critical transformation where they no longer approached instruction of their students as primarily an intellectual endeavor, but rather saw their students as whole beings and aimed to transform how school community members related to one another. Within educator reports we observed the emergence of a re-humanizing relationality, which could be akin to building new social capital in school communities. This study suggests that, while this transformation may take time and effort to cultivate, the outcomes it generates may be more sustainable than other education reform approaches.

Executive Summary

This report describes findings from an evaluation of the Trauma and Learning Policy Initiative (TLPI)’s Inquiry-Based Process. Findings are derived from data previously collected from three schools (with pseudonyms School A, School B, and School C) by TLPI staff members and American Institutes for Research (AIR) investigators. These data included in-depth interview and focus group transcripts from audio-recorded conversations with school staff collected at the beginning and end of the third year of implementation of TLPI’s Inquiry-Based Process.

The current evaluation was completed by PI Dr. Devin Atallah and Co-PI Dr. Michelle Porche (faculty at BU Wheelock College of Education and Human Development), who together organized and led a Data Analysis Team (DAT) with three BU students: Jessica Koslouski, doctoral student of Applied Human Development; Kesha Perkins, undergraduate psychology student; and Christine Marsico, doctoral student of Counseling Psychology. This five-member DAT completed the current evaluation, which is a secondary data analysis project using innovative qualitative methods (Situational Analysis) capable of evaluating complex and contextually-embedded processes, such as shifts in thinking and shifts in practices towards increased trauma-sensitivity within the three participating schools. Situational analysis is a method that provides substantial advantages over existing approaches to qualitative analysis. A key component of this method is the development of a diagram that synthesizes a series of maps reflecting data coding, to show relations between themes. This is in contrast to the typical list of codes organized into themes. This is important for the evaluation of TLPI to address the research questions and reflects how we interpret the change process based on the data.

To illustrate the empirically-based findings from our analysis we describe the multi-leveled transformations and cultural shifts within the three participating schools through the figure below. We hope that this illustration of our interpretation of our findings also deepens understandings of TLPI’s Inquiry-Based Process more broadly. Similar to figures, or models, that represent statistical results, we are depicting the relationships between themes from the qualitative coding of the data. We will discuss how this illustration summarizes the qualitative findings, reflecting how TLPI’s Inquiry-Based Process became embedded in schools leading to changes in their cultures (see Figure 1 below). The figure illustrates complex and multileveled processes of cultural changes in School A, School B, and School C, as found in the data, using TLPI’s Inquiry-Based Process. The figure conveys three levels of change that were facilitated by Shifts in Thinking and Shifts in Practice among educators. Additionally, the data suggests that the shifts in thinking and practice were dependent on, reciprocally supported, and reinforced by strengthened relationships, trust, and sense of community. The salience of the emergence of this strong relationality in schools on the process of promoting trauma-sensitivity is represented by a vertical arrow on the left-hand side of the figure. We use this figure to illustrate how we interpreted the process of change, as supported by interview and focus group data. We are limited, in that the data is comprised of self-report of participants’ actions and recall of process, rather than prospective observation and testing of specific strategies for change.

Figure 1. Synthesis of Transformations and Cultural Shifts Reported by Educators

In the current report, the three levels of the figure above describe a deepening progression that emerged from educators’ reports about their work to build a trauma-sensitive school using TLPI’s Inquiry-Based Process. In our analysis of the qualitative data, our findings take the shape of a triangle, which best represents how we interpret both the frequency of types of codes and structure of change. We find greater reports of foundational actions, and fewer of specified examples of culture shift, with what we identify as bridging actions in-between that act as mechanisms of change.

1. Foundation: When critical initial groundwork was being laid out, often including more formalized and surface-level processes and practices, and where steering committees and sounding boards played a stronger role in supporting action planning and initiating inquiry-based roadmaps;
2. Bridging: When mindsets and practices were being “tried on”, and deeper-level work was beginning to unfold in a school, with continual critical conversations, strengthening of collective reflection among faculty and staff, and ongoing support from steering committee and sounding boards;
3. Culture Shift: When more nuanced and holistic approaches were embodied in the mindsets and activities of school faculty, staff, and students, which depended less on formal structures, and instead, were embedded in strong relational bonds and systems internalized within the school.

Additional key characteristics of change revealed in our results include the Shifts in Thinking and Shifts in Practice dimensions. These two dimensions, as identified from the data, are represented as the two vertical sides of the triangle, which illustrate the schools’ progression toward an ever-deepening cycle of trauma-sensitive thinking and practice, as follows:

Shifts in Thinking: The development of mindsets, awareness, knowledge, and values with ongoing reflection that guided culture changes in schools towards trauma-sensitivity. These shifts in thinking were catalyzed by staff’s deepening, shared understanding of trauma’s impact on learning, behavior and relationships and the need for whole-school approaches.

Shifts in Practice: The continual conversations, critical reflections, and creative implementations of actions, structures, and supportive systems in schools that facilitated culture changes towards trauma-sensitivity.

Furthermore, the figure above describes cultural and organizational changes within the schools as they occurred across the three levels (Foundation, Bridging, and Culture Shift) and across the two dimensions (Shifts in Thinking and Shifts in Practice). Yet also importantly, results are organized along four categories (that are all interrelated constructs), which describe the Emergent Themes of trauma-sensitivity, and are grounded on our study team’s interpretations of the statements and detailed accounts of research participants:

1. Facilitating Empowerment and Collaboration: This theme is grounded on the intersection of the development of quality relationships in schools and trauma-sensitive collaboration. First, the data reveals that some school faculty and staff recognized the benefits of safe and supportive environments and were willing to stimulate motivation within those who were not yet onboard. These evolving mindsets were accompanied by the work of the Steering Committee, dialogues about teaching mindsets and practices, and brainstorming of action plans. As the schools executed these action plans, faculty and staff readily validated each other’s knowledge-sets and collaborated on trauma-sensitive practices throughout the building. Educators gradually became empowered trauma-sensitive leaders and drivers of ongoing change, as they initiated Whole Child practices and community and family engagement.
2. Integrating Whole-Child Approaches: In this theme, school faculty and staff began to acknowledge how student social-emotional needs and academic success go hand in hand. Thus, schools allocated time for trauma-focused professional development, where they learned and had discussions with one another about the effects of direct and vicarious trauma on both students and adults across the school community. With this developing knowledge, faculty and staff were enabled to respond to students with evolving empathy and the intent to listen/think first, before acting. As these relationships developed, adults recognized the need to reflect on not only their own practices and mindsets, but also students’ complex environments and experiences. Therefore, schools brainstormed solutions for student success based on Whole Child principles and aimed towards keeping students in classrooms and making their school communities more inclusive.
3. Affirming Cultural Identity and Promoting a Sense of Belonging: Culturally-affirming and trauma-sensitive practices intersected within this theme to produce supportive school environments. Educators may have been aware of the need to build cultural awareness and humility, yet using professional development time for building knowledge and skillsets of these topics was essential. Schools worked to transform these insights into practices that embrace diversity and inspire difficult dialogues across cultural differences. During this process, educators consistently reflected on their perspectives, actions, curricula, and environments to work towards affirming the identities of students by the school community. Educators began to comprehend and develop practices that reflected their understanding that one of the key meanings of trauma-sensitivity is: deeply understanding their students’ contexts. Finally, schools began to comprehend that to understand their students’ contexts, connections with students’ families and broader community partnerships needed to be strengthened. Therefore, schools began to promote familial and community dialogues and interactions to attempt to foster relationships where meaningful conversations and connectivity could be cultivated.
4. Re-envisioning Discipline towards Relational Accountability: The concluding theme presents the transformation of disciplinary practices within the school buildings. In this theme, educators questioned the purpose of retributive techniques and disciplinary mindsets that focus on punishing and separating students in response to infractions and disruptions. Furthermore, schools collectively explored how these mainstream educational disciplinary mindsets and practices affect student well-being and success. As a result, schools sought alternative disciplinary solutions that respond to students’ social-emotional needs, allowed for self-reflection, and that focused on restoring relationships. Faculty and staff worked towards holding themselves more accountable to their students and worked to keep them in the classroom. Faculty also aimed to help students develop more accountable relationships with each other and the school community as a whole. Furthermore, adults thought about circumstances behind student behavior and offered supports to manage this. As schools moved from punitive measures towards more restorative practices, students’ behavioral missteps were reframed as opportunities for learning.

For the full “Year Three” Evaluation Report, click here.