Eight Reasons Why Scoring Schools Doesn’t Work

Pennsylvania has just released its new School Performance Profiles, or SPP. As I’ve said before, that acronym probably ought to stand for Stupid Public Policy. These profiles are essentially scores assigned to schools based on the results of student testing and replace the previous Adequate Yearly Progress (AYP) rankings. [See “From AYP to SPP”] It’s very trendy right now among corporate-style reformers to grade schools like this. But the whole idea should receive an “F” and here’s why:

1.  The stakes are too high. Assigning scores to schools adds the “high stakes” to high-stakes-testing. When student test data is being used to determine resource allocation and to shape public perceptions of schools, the system creates a perverse incentive for adults to cheat. [See “A Plague of Cheating”] Recall Florida’s state superintendent, Tony Bennett, who was forced to resign this summer after reporters discovered that he had helped increase the grades of several schools. In one case, he increased a C to an A grade for a charter school run by and named after one of his campaign donors. [Politico, 8-1-13]

And don’t forget the Atlanta superintendent who was indicted this spring along with 34 others, including teachers and principals, for widespread cheating on the state’s standardized state tests. Investigators found 178 Atlanta educators had worked to change student answers, among other things, to increase the district’s performance. Eighty-two people have already confessed and the superintendent now faces up to 45 years in jail. [Washington Post, 3-30-13] This year we have confirmed cases of test score manipulations in at least 37 states plus the District of Columbia. [FairTest, 3-27-13]

Of course adult cheating is just one consequence of high-stakes-testing. Teachers are being demoralized by this system. Pittsburgh superintendent Dr. Linda Lane reports that when teachers received the results of the high-stakes-testing that formed the SPP scores “some were in tears.” [Post-Gazette, 10-5-13] But students suffer the most. The over-emphasis on testing results in lost class time, a school year spent on test preparation, the narrowing of curriculum, and the perpetuation of abusive practices that undermine actual learning. [See our piece “Testing Madness,” which was just republished in the Washington Post.]

2.  Scores actually reflect bad state policy making. The SPP scores are largely based on PSSA and Keystone test results, which are down for many students as the result of state decisions. Dr. Lane suggested the drop in Pittsburgh test scores resulted from, among other things, budget cuts, the elimination of modified testing for special education students, and the new Common Core standards, which are being taught in the classroom but not measured on the tests. [Post-Gazette, 10-5-13] With increased class sizes, school closings, and the loss of hundreds of educators again this year in Pittsburgh alone, our student test scores say more about poor state educational policy making than about actual teaching or learning.

3.  A single number is insufficient. The Pennsylvania Department of Education calls its new SPP system “comprehensive” and boasts it “brings together multiple academic indicators that are proven to provide a full overview of academic growth and achievement in our public schools.” [PDE, 10-4-13] I don’t know what evidence there is that these indicators “prove” academic growth in schools, but the idea of using multiple academic indicators sounds like a good idea. Too bad, then, that 26 of the 31 indicators listed for each school are actually based on high-stakes-test scores. [PA School Performance Profiles] While factors such as attendance and promotion rates are now being considered, these SPP scores are little more than a re-packaging of high-stakes-testing. Test scores don’t tell us much about what is actually happening at a school: the after-school mentoring program that parents started, the new playground or garden built and paid for by the local community, or all of the programs teachers volunteer to make happen, from directing the school chorus and plays, to coaching sports teams and the math club, mentoring student government, and collaborating with local artists. Where are those things on the profile?

What’s more, nearly everyone is fixated on the single “academic score” calculation – the grade – assigned to each school. The PDE can claim all it wants that these are robust profiles, but the media in every corner of the state has already demonstrated the way in which these profiles will be reported as single scores. For instance, the Post-Gazette reported, “Of those [Pittsburgh schools] that have academic scores, the highest is 82.6 at Pittsburgh Liberty K-5.” [Post-Gazette, 10-5-13; see also Post-Gazette 10-5-13 graphic.] Yay, Liberty! But honestly, what does that mean? The SPP scores effectively rank and sort schools.

4.  These systems are prone to error. The state has already bungled the release of SPP data. More than 600 schools (out of 3,000) do not have complete scores because of problems with relying on students to correctly fill in bubbles on the tests indicating if they were “end of course” exams. Rather than hold up the promised roll-out of the new profiles, the PA Department of Education instead released only partial data on Friday, leading to more confusion. For instance, no Pittsburgh high school or any school containing eighth grade currently has a score. The state is also delaying the release of the 2013 PSSA and Keystone results. West Mifflin Area Superintendent Daniel Castagna summed it up, saying, “This is a mess, an absolute mess.” [Post-Gazette, 10-5-13]

5.  Scores actually just measure poverty. It’s great that my former elementary school (Eisenhower in Upper St. Clair) got the highest score reported so far in the county, with a whopping 97.9. Not surprisingly, there’s my middle school, Fort Couch, at 96.8. And all of Mt. Lebanon’s reported schools so far are over 90. But did we need all these tests and this elaborate new system to tell us that upper-middle-class kids in predominantly white suburbs are doing better than those in the struggling Duquesne school district, which weighed in at 49.3, the lowest in the county? What standardized test scores are really good at showing is family income. For an excellent visualization of the correlation between test scores and poverty, take a look at last year’s SAT:

6.  Scores don’t measure what matters. The Pittsburgh school district has conducted research on its own graduates and concluded that, “the most important predictors of post-secondary education success are grade point average and attendance, not state test scores.” [Post-Gazette, 10-5-13] If that’s the case, why are we spending so much time giving these high-stakes-tests to our students? Why are we giving 21, even 30, standardized tests each year to our kids? [PPS Assessment Calendar] Why aren’t we focusing on providing a rich, engaging curriculum with music, art, languages, and activities so that students want to be in school? We can’t even discuss these important questions because the School Performance Profile system forces districts to continue playing the game of ever-more-testing in the name of accountability. But if we really care about what matters – such as actual student learning or college success – policy makers must move away from systems that simply reinforce testing by assigning grades to our schools.

7.  Scoring schools wastes valuable resources. The SPP system cost us taxpayers $2.7 million to develop over the past three years. [Post-Gazette, 10-5-13] That’s $2.7 million at the exact same time that Governor Corbett and our legislature were telling us we did not have money to pay for our public schools. And it will cost an estimated $838,000 every year to maintain. That’s a lot of drumsticks for the Westinghouse Bulldogs Marching Band or library books for Pittsburgh Manchester K-8. Beyond the ridiculous price tag, grading our schools costs valuable staff time and wastes the attention of the public, media, and policy makers by forcing them to focus on the wrong thing.

8.  School scores don’t help students. SPP scores don’t give students what they really need: adequate, equitable, and sustainable state funding for their public schools. Public policies that support, rather than vilify, their teachers. Quality early childhood education. Pre-natal care. Healthcare. The stability of their community school remaining open. Smaller class sizes. It’s would be funny, if it weren’t so cruel, to hear the PA Department of Education proudly explaining that under the new SPP system, the lowest scoring Title I schools (those that serve a large proportion of low-income students) are now eligible for “access to intervention and support services.” [PDE, 10-4-13] How about access to their laid-off teachers and state funding they desperately need?

Even worse, the highest performing Title I schools will now be rewarded by becoming “eligible to compete for collaboration and/or innovation grants.” Are you kidding me? This is right out of the Race-to-the-Top playbook, making schools compete for the resources they desperately need. Races and grant programs by definition have winners and losers. No student at a Title I school deserves to be a loser in this game invented by policy makers. Our kids don’t need “technical assistance,” they need state legislators to restore the budget cuts and reinstate a modern, fair funding formula. These SPP scores are only going to hurt our poorest students and communities of color more.

From AYP to SPP

Move over Adequate Yearly Progress (AYP). Time to make room for the School Performance Profile (SPP). Pennsylvania has just been granted its waiver to the federal No Child Left Behind (NCLB) Act, which had required that all students in the country be proficient in reading and math by next year. But don’t start celebrating just yet.

When it became apparent that AYP goals were pie in the sky, states started lining up to get waivers. It took Pennsylvania longer than most others to get in line (41 others were ahead of us). But that’s only because then-Secretary of Education Ron Tomalis was concerned that a waiver would force the state to change the way it performs standardized testing and that any new federal laws could make the state change its testing again. Once it became clear there would be no new federal legislation, Pennsylvania applied for the waiver, which comes with plenty of strings attached. [See “Waiver no Favor”]

What we will get now is a School Performance Profile, based on student test participation rates, graduation and attendance rates, and two measures of progress towards closing achievement gaps. [Post-Gazette, 8-21-13] That means that three out of the four “annual measurable objectives” in the profile will be based on student testing. In other words, SPP threatens to simply replace the old high-stakes-test-crazy system with a new one. Or as our Shippensburg grassroots colleague Susan Spicka aptly put it: “It is absurd that our state government is focusing so much time, energy and money ensuring that all children have an equal opportunity to be evaluated when it is clear that all children do not have an equal opportunity to learn.” [The Sentinel, 8-22-13]

And that’s exactly the problem. This new SPP system will label schools without providing any real help for struggling students. If a school receives federal Title I money (based on its proportion of poor students), it will be labeled “priority,” “focus,” or “reward.” All other schools will get a profile score. It’s not clear if that score will be a number or letter grade (A – F), which is very trendy right now among corporate-style-reformers who support vouchers, charter-expansion, school closure, and other privatization efforts. Either way, the bottom line is these rating systems do not appear to work and are definitely subject to cheating.

In an analysis of Indiana’s school grading system, sociologist Matthew Di Carlo found an extremely high correlation between poverty and a school’s grade. Almost 85% of low-poverty schools earned and A or B, and almost none got a D or F, while over half of the high-poverty schools earned Ds and Fs. A full 80% of the schools labeled as failing with an F were in the highest poverty quartile. Dr. Di Carlo concluded, “This is not at all surprising. It is baked into this system (and you’ll see roughly the same thing in other states).” [Albert Shanker Institute, 11-1-12]

This summer we learned how these school grading systems encourage adults to cheat. Have you heard of “Campbell’s Law”? Dr. Donald Campbell was a social scientist who famously theorized, “The more any quantitative social indicator (or even some qualitative indicator) is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”

That is exactly what happens with high-stakes-testing and its application to school grading. Earlier this month Florida’s state superintendent, Tony Bennett, was forced to resign after reporters discovered that he had changed accountability measures to lift the grades of several schools back in Indiana when he was the elected public school leader of that state. In one case, he helped change a C to an A grade for a charter school run by and named after one of his campaign donors. [Politico, 8-1-13] See the distortion and corruption of Campbell’s Law?

And even when school grading systems aren’t corrupted, they still have the effect of reinforcing inequality. Next door in New Jersey, forty-five parent groups and civil rights groups petitioned U.S. Secretary Duncan last year to stop the waiver process that was imposing the priority-focus-reward labeling system in that state. Those groups found that the system targeted the poorest schools with the highest percentage of African American and Latino students with mandated interventions, including possible closure or conversion to charter schools. The coalition noted that the 122 “reward” schools identified by the state, which were all to receive financial bonuses, were all located in the wealthiest districts. They concluded, “The blatant economic and racial inequity built into this classification system harks back to the days when such segregation and inequity were policy objectives for our State.” [Save Our Schools NJ, 10-15-12]

It appears that Pennsylvania may sidestep this issue by using two separate systems here: applying the SPP score to all schools, and the priority-focus-reward labels only to Title I schools. But without actual assistance for struggling students and districts, all of these scores and labels are meaningless. In fact, they are harmful, since they tend to have major consequences – from stigmatizing schools (who wants to send their kids to an “F-rated” school?), to further narrowing the curriculum (since the high-stakes-tests cover only the basics, subjects like art, music, and world languages get fewer and fewer resources), to the threat of closure.

It’s almost a cruel joke to hear Gov. Corbett announce, “This waiver allows Pennsylvania to focus on improving schools by directing resources to areas that help students academically succeed.” [Post-Gazette, 8-21-13] Oh really? What resources would those be? Our schools are still missing $2.3 BILLION in cuts since 2011. [See “Budget Failure”] This year, when the legislature approved a miniscule increase over last year’s budget, it allowed select legislators to hand out extra funds to 21 of their favorite school districts – with no basis at all in directing resources to helping real students in need. [Lancaster Online, 7-21-13]

Under the new SPP labeling system, Gov. Corbett’s administration promises interventions, “focusing on having principals who are strong leaders; ensuring effective teachers; providing additional time for student learning and teacher collaboration; and strengthening instructional programs.” The state will also “provide academic recovery liaisons to help priority schools.” [Post-Gazette, 8-21-13] Hello, what? Where is the money to hire back our teachers, school counselors, nurses, and librarians? How about some funding for our after-school tutoring programs we had to cut? And early childhood education? Maybe SPP should stand for Stupid Public Policy.

If we are serious about offering interventions and supports to struggling schools, we have to be talking about wrap-around services that address trenchant disparities: we need community healthcare in the buildings, childcare, adult job and literacy training, family crisis services. We need to make sure communities have a good public school to send their children to and that we aren’t creating school deserts. We need authentic parent engagement. And we need to make schools enriching, welcoming places that students want to be in with full art programs and a wide range of activities.

I worry that SPP will just replace AYP: with more high-stakes-testing, more labeling-and-punishing schools, more blaming teachers, and still no results for our kids.