WHAT DO THESE NUMBERS TELL US?

If there is one lesson to be learned about dropout statistics it should be this – there is no single measurement that gives all communities the answer to why students leave school before graduating. Reliance on a single statistic not only forecloses exploration into other areas of inquiry, it may lead dedicated activists to avidly pursue strategies that are simply not helpful. What we provide here is a number that communities can use to begin discussing what it takes to keep their young people in school. We invite you to dive right into the discussion by understanding some of the elements that make this a complex debate about measuring student advancement and graduation.

The best way to get started is to take a look at what’s behind the numbers. Here are some factors that influence the calculating of graduation rates and why they are important.

The Finish Line Calculator does not cover every school district in every state – why? A large number of schools in the CCD data are missing enrollment data for particular years, without this data we cannot calculate a graduation rate. Schools missing data were removed from consideration. Also removed from consideration were schools that clearly did not grant diplomas such as some alternative schools and special education programs.

Failure to remove these types of schools can result in an artificially high discrepancy between a high school’s senior year enrollment and the number of graduates in the subsequent year. It should be emphasized that it is close to impossible to determine from the data itself whether a school grants a diploma or not. While the CCD has designations for four different types of schools (Regular, Special Education, Vocational, Other/Alternative – the variable is labeled “TYPE” in the CCD codebook) this is ultimately not a useful way of distinguishing between types of schools. Some school systems will count all students as enrolled at a central middle or high school and then send groups of students to vocational, special education, or alternative schools. The result is large enrollment numbers for the school system and low to no enrollment for the other school types.

When considering school districts think about the size of the district. Many districts have over 30 or 40 schools covering elementary, middle, and high school for children with a wide variety of needs. We found that the composition of the schools in a district can change rapidly over the course of a decade. New schools will be created, others will disappear, some school systems consolidate, others disaggregate. In most instances it is difficult to discern whether students have been added or lost to redistricting, bussing, or class size requirements as a result of these movements.

What does it mean to say that a certain number of students are enrolled in a particular grade in a particular year? Enrollment data reported to the CCD is collected in October of each year. This measure is deemed fairly accurate as it would be timed in a way that hopefully captures the arrival of new students and the departure of those students who might not otherwise attend an entire school year. At the same time, any students who join or leave the school following the October date for measuring enrollment will not be reflected in the data. Typically this is a minor threat to the validity of this measure but the threat will vary depending on the size of the school and change in enrollment.

Anyone examining enrollment data for schools nationwide will immediately note a bottleneck of students in the 9th grade. In other words, it is fairly typical for enrollment to remain steadily until 9th grade when it spikes significantly upward and then gradually declines until graduation. This bottleneck represents the problem of transitioning from middle to high school for many students, it also can represent a culmination of unfulfilled requirements from previous years that condemn some students to repeat grades before they can be promoted.

The 9th grade bottleneck creates a problem for anyone attempting to create a warning indicator for schools based on enrollment data. Differences in enrollment from one year to the next can demonstrate gaps left by students dropping-out, measured across all grades these gaps will give a strong indication of a student’s overall chance of graduating. An increase from 8th to 9th grade (as with a bottleneck) will often counter the gap experienced between other years and artificially inflate the reported probability of graduation. One of the solutions to this problem is to create a measure that starts in the 9th grade (as with the Urban Institute’s Cumulative Promotion Index), rather than the 8th grade (as with the Learning to Finish measure), and thereby skips the problem created by the bottleneck. This approach, unfortunately, avoids the pressing question of how to integrate 8th grade enrollment data with high school enrollment data in a way that meaningfully expresses a graduation or dropout rate.

Enrollment rates were used as the primary type of data for the Finish Line calculator because they were easy to obtain, easy to use, and formed a compromise between what we felt were some draconian measures of graduation rates and others that were overly generous. A more accurate assessment of the graduation rate can be achieved using the same formula but incorporating transfers into, and out of, the school system. Unfortunately, the CCD does not include this information but concerned citizens can calculate this more accurate measure by asking their local school district for transfer data and incorporating that with CCD enrollment data. Two rates that incorporate transfer data are the Revised Cohort Rate used by the National Governor’s Association and the Timely Progress Rate.

It cannot be emphasized enough that a simple count of enrollment captures a wide variety of factors that are not readily apparent. Think about the broad array of students being served by the school system. Socio-economic factors such as poverty are predictive of high graduation rates but are not expressed by enrollment data and demographic tendencies of groups like African-American or Latinos are similarly not captured by this measure. Vocational and tech students, students returning to school after a leave of several years, advanced placement students, students who have repeated multiple grades, students for whom English is a second language, students whose parents are migrant laborers, students with learning disabilities, students with individualized learning plans, students spending a year in public school after several years in a private school … all are hidden somewhere under an umbrella term of “enrollment”. The point is that while enrollment statistics give a good sense of how many students might graduate and students in which grades are most at risk it says little about which students suffer the most and the difference in challenges faced by schools with otherwise similar graduation rates.

The CCD provides a count of graduates in each school district that is used as the final term in the Finish Line calculator. While this graduation data may seem essential it can be terribly misleading as it has no direct connection to the student count. According to CCD staff this measure may include juniors graduating early and post grads graduating late, they felt it was unreliable enough that it was not included in the CCD’s own dropout rate measurement (called the “Averaged Freshman Graduation Rate”.)

For our purposes in constructing the Finish Line calculator a term was needed to reflect the loss of students between the beginning of senior year and the awarding of diplomas. Every school has a gap between these two terms, the question for us was how to create a term that adequately expressed this senior year drop off. We strongly considered the option of calculating the average national rate of completion and then using this rate as a standard term for all schools. Such a measure would penalize some schools slightly and reward others slightly but in every case it would offer some representation of a phenomenon that we know exists. In the end we decided that the addition of a somewhat elevated count of diplomas would not severely damage the accuracy of a measure intended to be used mostly as an entrée to a larger discussion in the community.

Given what we know about the high school completer data from the CCD, though, users should be aware that this term will likely yield a slightly higher result from the Finish Line calculator.

We considered two formulas when developing the Finish Line calculator, both of them operate on the same principle by tracking the difference in enrollment from one class year to the next. The CPI measure starts in the 9th grade, the Learning to Finish measure starts in the 8th grade. Both formulas presented unique challenges but perhaps the greatest challenge was correctly matching middle schools with 8th graders, or “feeder schools”, with high schools that had grades 9th-12th. In many instances it was possible to make a match and calculate the Learning to Finish measure. There were fewer problems encountered (and thus more school districts covered) by using the CPI measure. Ultimately we chose to use the CPI method but we regret not having the level of detail that would make calculating the Learning to Finish measure possible for a broader range of school districts nationwide.

After having applied enrollment data to these formulas for countless communities nationwide we arrived at some important conclusions about the formula itself. Namely, that this is a formula that works extremely well with populations that are relatively stable and static but is less useful for communities in transition. In practice this means that the formula works well for many urban and rural communities but less well with suburban communities.

Of course, as with any calculation, the more observations you have the more likely you are to achieve a representative outcome. Our formula operates no differently. School districts with thousands of students enrolled in each grade will have a more accurate estimate provided by these formulas than districts that have a few hundred students. It should be clear from this reason alone why we chose to calculate graduation rates at the level of school district rather than on a school-by-school basis.

Finally, because these formulas are sensitive (by design) to fluctuations in enrollment numbers they will react to events external to the school. Natural disasters like Hurricane Katrina obviously had an immediate impact on the school systems in not just New Orleans but throughout Louisiana, Mississippi, and the surrounding states. Economic development can play a role as well, though, with economically successful areas potentially attracting an influx of new parents and economically unsuccessful areas experiencing the flight of parents in search of jobs. As a result, graduation rates should be considered within the context of the trends and recent events experienced by that school district’s community.