Wisconsin Residential Provider Turns Evidence Use Skills into Action
By Dr. Jeffrey Krukar, Licensed Psychologist/Certified School Psychologist, Genesee Lake School
In our last post, we offered an example of how members of the Wisconsin Association of Family & Children’s Agencies used the EDGE curriculum to advance best practice performance measurement among a group of private child welfare service providers. In this post, we focus on the work of one of those organizations. Here, guest contributor Dr. Jeffrey Krukar describes how he is applying lessons learned in EDGE to improve the way his agency, Genesee Lake School, uses evidence to drive the process of improvement.
In her description of WAFCA’s work, Emily Coddington wrote, “WAFCA members are strong believers in the concept that once you know better, you must do better.” GLS’s efforts post-participation in EDGE exemplify this commitment. Inspired to do better, GLS leaders have invested time, resources, and staff to ensure their performance measures are valid, reliable, and free of bias. They are also working to educate their colleagues, taking steps to diffuse evidence use throughout the organization. Their story shows that individual providers, regardless of their size, have the capacity for this work, and that small steps can build the foundation for elevating evidence-based decision-making agency wide.
Genesee Lake School is a private child and adolescent residential provider in Wisconsin. We enrolled in the EDGE program because we believed it would strengthen our capacity to use evidence to deliver high quality and effective services. We are very appreciative that we were able to take advantage of this learning opportunity. EDGE gave us critical skills for agency management, and we have begun to translate those skills into improved evidence use at the leadership level.
While there are many important lessons in EDGE, several resonated in particular. First, the course offered an extremely user-friendly format for understanding foundational principles of CQI and performance measurement in child welfare. EDGE helped us better understand the distinction between research evidence and other organizational information, as well as the importance of using evidence to guide our improvement efforts. EDGE also encouraged us to incorporate evidence into our theory of change, forcing us to think through essential questions such as, “How do we really know this is a problem that needs improvement?”, “How do we really know this intervention will improve the problem?” and “How will we really know whether the problem improves?” Finally, and critically, EDGE compelled us to be more accurate in our thinking, methods, and language when it comes to crafting performance measures. We learned about the importance of identifying the correct risk set and how failing to do so can lead to a biased picture of performance. We also learned how to establish a performance window and how to use longitudinal analyses to examine student change over time.
Given what we learned from EDGE, we were inspired as agency leaders to do some things differently when it comes to how we generate, disseminate, and use evidence about performance. Prior to the course, we had already launched an agency improvement plan to reduce the use of physical intervention (PI). After completing EDGE, we decided to revisit our plan and make some changes aligned with what we learned. For example, we completed EDGE’s Plan-Do-Study-Act and Theory of Change (PDSA/TOC) templates for our PI reduction plan. We supplemented the plan with specific evidence from the social science literature, and developed entry cohort measures so we could examine the likelihood of PI across the organization and change over time. Through EDGE, we have also taken a closer look internally at how we collect, organize, and analyze data, and have had internal discussions about potential changes we can make to improve our capacity for data analysis in the future.
At the present time, through additional collaboration with Dr. Lily Alpert, we are working to develop a quarterly physical intervention summary. The summary will provide GLS leadership with two measures related to PI. One is a monthly snapshot of incidents per student served. GLS leaders are accustomed to this type of measure but, as we learned in EDGE, it is fairly limited in terms of its ability to illuminate change over time or inform strategic action. Therefore, the second indicator is an entry cohort measure that will allow us to examine the likelihood of PI and any change associated with our interventions. Although this measure provides a more representative view of PI, agency leaders will need education on how to interpret it and why it offers more programmatic insight. To that end, the summary will include narrative explanations of major EDGE principles and apply them directly to the measures in the report. In this way, we will use the summary as in instrument for teaching as well as for reporting.
Of course, EDGE principles can be applied to many aspects of our organization. We are also exploring how we can use the PDSA/TOC processes in other areas including permanency, and potentially for employee issues in the workplace such as injuries and retention. In short, as a result of EDGE, seeds are being planted across the organization that will bring us into alignment with true evidence-based practice.
Human service agencies are very busy places where very well-intentioned people work at a fast pace. EDGE encourages all of us to slow down (at least for a little while) and carefully think through the questions we have about our performance, answer them correctly, examine the evidence, and proceed accordingly. It is this kind of deliberate and accountable work that will enable us to provide the best services we can to those we serve in the time we have.