The Role of Trade Organizations in Advancing Best Practices in Performance Measurement
By: Emily Coddington, Associate Director, Wisconsin Association of Family & Children’s Agencies
In a previous post, we described how the Data Center’s EDGE training program improved research evidence use capability among child welfare administrators. A number of those agencies are members of the Wisconsin Association of Family & Children’s Agencies (WAFCA), a trade organization that represents child and family serving agencies across the state. In this guest post, WAFCA’s Associate Director, Emily Coddington, talks about how, after participating in EDGE, member agencies leveraged WAFCA’s support to put what they learned into action.
We draw attention to WAFCA’s work to highlight the role professional organizations can play in supporting their member agencies to advance best practice measurement. The case study below is very much a story about private organizations achieving together what they might not have been able to achieve on their own. In this case, a committed group of agency leaders, supported by a like-minded membership organization, gave Wisconsin providers the resources they needed to improve their practice of evidence-based decision-making.
WAFCA is a statewide association that represents nearly fifty child and family serving agencies and advocates for the more than 200,000 individuals and families those agencies impact each year. The services and supports WAFCA provides are designed to help member agencies live into their values, advance their missions, and deliver services in an ethical, sustainable, and equitable manner. WAFCA also engages in partnerships that promote a skilled and knowledgeable workforce and the availability of high-quality services.
As part of WAFCA’s best practice agenda, we encourage and support agency leaders in measuring outcomes and strategizing to achieve the best outcomes possible for those in their care. In 2016, WAFCA created the Impact Data Benchmarking Project with the goal of developing an outcome database. The vision was to gather information about who agencies were serving; presenting problems and the challenges facing those receiving services; interventions provided to those served; and the effectiveness of the interventions provided. The intention was to collect and analyze relevant data and disseminate lessons learned.
With these priorities at the forefront, WAFCA quickly recognized the value of the EDGE program. EDGE responded to member agencies’ desire to measure their performance and feed that evidence back into a self-correcting, ongoing process of improvement. It also addressed members’ interest in demonstrating their impact to state and county partners, donors, agency boards, and the staff working hard to make a difference. And so, WAFCA supported the EDGE program from the beginning. We believed strongly that the training would improve agencies’ evidence use knowledge and skills, and in turn, help our members adopt practices that promote the well‐being of children and families.
Over the course of their participation EDGE, members of the Impact Data group identified how the principles and techniques they were learning could advance these goals. Specifically, EDGE made them realize that there was room for improvement in the way the group was working with data to generate information—namely, they needed to switch to a more representative approach to measurement, analyzing agency performance based on entry cohorts instead of exit cohorts. WAFCA staff also reviewed the curriculum and understood how this shift could substantially improve agencies’ ability to demonstrate their impact.
WAFCA members are strong believers in the concept that once you know better, you must do better. During Impact Data meetings, members who had gone through EDGE were ready to put their skills into practice. They started thinking differently about how to frame their pressing performance questions and expressed interest in making the shift from exit cohorts to entry cohorts, in alignment with what they had learned. All members recognized that they could continue gathering the same information they had been over the years, but that they would need to modify the way those data were organized and analyzed if they wanted their performance measures to help them tell their story.
Group members committed to creating a longitudinal spell file—the type of file required to organize client data in terms of entry cohorts—and a volunteer stepped forward to assemble and analyze the new database. The group agreed to start with just a few fundamental questions related to permanency and duration in care as a first step in building a foundation for future analyses that generate performance evidence. To date, the participating members have updated the spell file three times, refining their criteria and coding schemes along the way. We have used that file to produce preliminary outcomes. Moreover, we are in the process of compiling those findings in document for WAFCA and its members to use in discussion with core stakeholders. We see this work as just the beginning. From here, the group will work to develop their skills and answer other questions they have, so they can build narratives around their effectiveness and opportunities for improvement. These agency leaders have a strong desire to use data to make evidence-informed practice improvements, which speaks not only to their own professional motivation, but also to the success of the EDGE project.