Framing analytic questions in the context of continuous quality improvement

The best data analyses start with well-formed analytic questions. Therefore, before you set out to study your system’s administrative data—whether you are doing so with the Data Center’s web tool, your state’s complete longitudinal file, or any other dataset—it is important to spend time at the front end making sure that you’ve identified exactly what it is you want to know. What is your question? Which group of children will you need to examine in order to answer it? And what type of data will get you the answer that you are looking for?

Longitudinal analyses of foster care data center around two main components—the likelihood of events and the speed with which they happen. In other words, good longitudinal inquiries are always variations on the same two-pronged theme: What happens to children who enter foster care (the likelihood of events), and how long does it take for children to have those experiences (the speed with which they happen)?

Using these two basic questions as a model enables us to generate many questions pertaining to a range of programmatic child welfare issues. These can include inquiries into permanency outcomes for children in care, the effectiveness of casework interventions, and expenditures related to caring for children in state custody—all of which play into a system’s process of continuous quality improvement.  Here are just a few examples:

• How long will children placed in foster care stay in care?  
  

• Will children have one placement during their time in care, or will they have multiple placements?
  

• What will the end result be for the children we place in care? Will they be returned to the care and custody of their parents? Will they be discharged to relatives? Will they find permanent homes through the adoption process? Will the children placed in care find permanent homes at all, or will the state ultimately be responsible for their transition to adulthood?
  

• When children are discharged to permanent homes, how many of them will wind up coming back into care again, and when?

At the same time as we ask ourselves which issues are most pressing in our specific systems, we must also consider the fact that children’s trajectories through foster care differ based on certain child and spell-related characteristics. Answering the above questions for all children in care is a necessary first step; but the fact that different kinds of children have different experiences requires us to dig deeper. Breaking the population into subgroups and analyzing a certain question for each is called stratification. Working off the inquiries posed above, here are some examples:

• Do children re-entering foster care spend as much time in care as children entering for the first time? (stratified by first entry/re-entry)
  

• Do children who are initially placed in congregate care settings experience more placement moves than children who are initially placed in family settings? (stratified by first placement type)
  

• How long does it take for African-American children to get adopted? Does it take the same amount of time for white children? (stratified by racial/ethnic background)
  

• Are children who enter foster care as infants as likely to be reunified as children who enter care at older ages? (stratified by age at entry)
  

• Are children who spend short amounts of time in foster care as likely to re-enter care within one year as children who spend long amounts of time in care? (stratified by length of stay)

Finally, the knowledge we gain from these inquiries begs questions related to policy, practice, fiscal planning, and the cycle of continuous quality improvement (CQI):

• What are we doing to get children to permanency as soon as possible? Since foster care is among the most expensive interventions for children experiencing maltreatment, how much will our foster care program cost?
  

• What are we doing to stabilize children in family settings?
  

• What are the barriers to permanency for different types of children in foster care and how can they be overcome?
  

• What are we doing to prepare our older youth who are at risk of aging out of care? If we reinvested some of the money we would have spent on foster care on strengthening the relationship between children and their parents, could we increase the likelihood of reunification and decrease the likelihood of a non-permanent exit?
  

• What are we doing to prevent re-entries into care?

All of these analytic steps inform us of the current state of affairs, which issues need tackling, and which groups of children need targeted attention. Thus, the CQI process begins. Based on information from these diagnostic inquiries, we proceed in the CQI cycle with the following three presumptions:

1. There is a gap between current performance (baseline) and what’s possible (the goal).
2. Both the baseline and the goal may differ by subpopulations.
3. Given that there is a gap, it will take time for the gap to close. Innovation can only influence that which has yet to happen.

The innovations, of course, are our initiatives—policy, practice, and fiscal changes designed to improve outcomes for children and their families. Child and Family Team Meetings, Permanency Roundtables, foster parent recruitment and retention efforts, treatment foster care, performance-based contracting—and the list goes on. We select these initiatives based on what we believe will make change, implement them with the highest level of fidelity as possible, and then track their effectiveness over time. Here, we need longitudinal analyses again, because the only way to measure the effectiveness of our initiatives is to examine the experiences of children who are exposed to them and compare those children to previous cohorts of children who were not. The specific methods involved in this type of analysis will be covered in subsequent Recipes.

In summary:

• Longitudinal foster care data inquiries are based on two main components: What happens to children who enter foster care (the likelihood of events), and how long does it take for children to experience those events (the speed with which they happen)?

• Different types of children in foster care experience different trajectories; therefore, it is important to stratify data inquiries by child- and spell-related characteristics.

• The knowledge gained through longitudinal data analyses has implications for foster care policy, practice, and fiscal planning; therefore, such analyses are essential in the context of continuous quality improvement.

• The CQI process involves implementing initiatives designed to improve outcomes for children in care and evaluating those initiatives using longitudinal methods.

As you work your way through the Recipes to come, you’ll find that all of them are based on these principles. One of the main goals of Recipes is to help you integrate these principles into your own work; to install them as the foundation of the CQI process in your own jurisdiction, and to learn how to conduct analyses that keep your CQI process moving in the right direction.