Recipe: Censored data—How do I know if my findings are stable?

One important thing to bear in mind about entry cohort analyses is that when we follow a cohort of children forward, it takes time for each member’s outcomes to unfold. For example, say we want to know the length of stay of children who entered foster care in 2011. We can calculate that figure for the 2011 entrants who have already exited care, but there will be other 2011 entrants for whom we can’t calculate that figure because as of the censor date–the date as of which the archive was most recently updated–they were still in foster care. These children could have exited the day after the censor date or they may not exit for another two years; we just don’t know yet.

When outcomes for children cannot be determined because they were still in care as of the censor date, we say that their spells (and thus, their data) are censored. Censored data are constantly factoring into longitudinal analyses, so it is important to know how to identify and interpret them.

In this Recipe I’ll show you how to produce a helpful report that shows the time it takes for children to achieve various permanent exits and how to identify and interpret the censored data that it contains.

Question: Has children’s time to permanency changed over time? How do I know whether or not I need to account for censored data in interpreting my findings?

To answer this question, we’re going to use the Profile Report function—one of the newer features of the web tool. Profile Reports contain canned, preset inquiries into trends that administrators usually want to be able to observe over time. Running a Profile Report triggers a live query of the most updated Data Archive, and produces results for the most recent 8 years.

Profile Reports run queries off of your state’s most recently updated file. Therefore, depending on the schedule with which your state submits data to the Data Center, the censor dates of individual member states’ files will differ. In the case of the county that I used in the example below, the state’s censor date was 6/30/2011.

1. Log into the FCDA Multistate site (Profile Reports only exist on the Multistate site) and click Profile Report on the left-hand sidebar.
2. Under Select report category, select Permanency.
3. Under Select report type, select Table 12: Cumulative Number and Percent Discharged to Permanent Exits, All Placement Ages. Then click Run Report.

A new tab will open containing your report. Below is an excerpt from the results for the county that I used:

This excerpt of Table 12 shows the number of children that entered foster care for the first time in each of the last eight years. Immediately to the right, we see the number of children in each entry cohort that exited to reunification within six months, one year, etc. And to the far right, the percentage of children exiting within those timeframes is calculated by dividing the number of exiters (numerator) by the total number of children in the entry cohort (the denominator).

How do I know when I have to account for censored data? When we cannot calculate the outcome of interest for all of the children in an entry cohort, we know that we must account for censored spells—children who may one day achieve the outcome of interest, but as of the censor date, haven’t. Using the Table 12 Profile Report, we can see how this phenomenon plays out:

• If a cell contains a figure and it is not shaded yellow it means that, as of the censor date, all of the children in the entry cohort (the row) had the opportunity to achieve the outcome of interest (the column). In these instances, we don’t have to worry about censored spells (i.e., children who may still, one day, achieve the outcome of interest) and, as a result, the findings are stable. For example, the “within 2 years” column notes that 100 of the 225 children who entered care in 2008 reunified within 2 years. This figure is stable or fixed because the last possible day that a child entering care in 2008 could have exited within two years was 12/31/2010 (six months prior to the censor date). Even if additional children from the 2008 cohort reunify in the future, if they were still in care on 12/31/2010, they will necessarily have lengths of stay longer than two years; in other words, they cannot add to the 100 children who reunified within two years.

• If a cell is shaded yellow it means that, as of the censor date, some, but not all of the children in the entry cohort have had the opportunity to achieve the outcome of interest. In these instances, we do have to worry about censored spells because there are still children in the entry cohort who may, one day, achieve the outcome of interest. For example, “the within 2 years” column is shaded yellow for children entering care in 2009. That’s because children who entered foster care in 2009 had the chance to be in care for a maximum of two and a half years (1/1/2009 to 6/30/2011 = 30 months) and a minimum of a year and a half (12/31/2009 to 6/30/2011 = 18 months). The 110 reunified children shown above is an unstable figure because, as time goes on, it has the potential to increase. More 2009 entrants will likely reunify between 7/1/2011 and 12/31/2011 (in the six months following the censor date). When they do, they will add to the number of children “reunified within two years.” After the 12/31/2011 database update, the yellow shading in that cell will disappear; at that point, the figure will be fixed because it will no longer be possible for any 2009 entrants still in care to reunify within 2 years.

•  If a cell is empty ( — ) it means that, as of the censor date, none of the children in the entry cohort have had the chance to achieve the outcome of interest. For example, the “within 2 years” column is blank for children entering care in 2010. That’s because if a child entered care in 2010, the longest he or she could have possibly been in foster care as of 6/30/2011 is only a year and a half (1/1/2010 to 6/30/2011 = 18 months). This empty cell is also unstable because it will also increase as time goes on. After the 12/31/2011 database update—once it becomes possible for a 2010 entrant to be in care for two years—the currently empty “within 2 years” cell will start being populated with children who reunify after that period of time. That cell will be shaded yellow until the 12/31/2012 update, at which point it will no longer be possible for a 2010 entrant still in care to reunify within two years.

In summary:

• Obtaining complete results of entry cohort analyses takes some time because we must wait to find out the outcome for every member of the cohort.

• In order to know whether censored data are factoring into your entry cohort analyses, note the entry year under consideration and ask yourself if all, some, or none of the children in the cohort have had the opportunity to achieve the outcome of interest.