The Department of Education’s Misleading Academic Score System

In an apparent effort to put the best face possible on failing schools in the state, the Pennsylvania Department of Education (DoEd) has concocted a complex scoring scheme that is intended to blunt the impact of the reality of the poor performance at many of the state’s public schools. This is done by giving dubious credit for some measures and irrationally weighting some factors, all of which are aimed at making the academic performance at failing schools look better than it actually is.

 

A comparison of two schools in Allegheny County will illustrate the absurdity of the DoEd’s rating scheme. For the latest school year, 2013-2014, Lincoln Elementary in Mt. Lebanon received a DoEd academic score of 85.2. In the same year Kelly Elementary in Wilkinsburg achieved a score of 62.9.  One would assume from these scores that even though Lincoln has a better score, Kelly is only 22 points (25 percent) below Lincoln.  Not good but not terrible.

 

Here is the problem. The results on the PSSA (state assessment) tests reveal a vastly different picture from that portrayed by the DoEd’s Academic Scores. At Lincoln, 93.3 percent of students tested proficient or advanced in math: at Kelly 38.9 percent were proficient or higher. 72.5 percent of Lincoln students scored at the advanced level on the math exam, only 9.4 percent scored at the advanced level at Kelly. The reading results were similar; 92.8 percent either proficient or advanced with 61.2 percent at the advanced level at Lincoln. At Kelly, only 29.4 percent were either proficient or advanced with only 3.9 percent advanced. Finally, on what is called a crucial indicator for elementary education and a predictor of future accomplishment of students, 95.8 percent of Lincoln third graders were proficient or advanced in reading; only 34.7 percent of third graders at Kelly attained proficiency or better.

 

So how does this enormous difference in test results for the two schools get converted into the mere 22 point Academic Score difference reported by DoEd?

 

There are three very large elements in this distorted picture. First, actual test results on four PSSA tests, including science and writing plus the score for third grade reading, account for a maximum of only 40 points in the final total—7.5 points for the four PSSA tests for all grades together and 10 points for third grade reading. Kelly earned 17.3 points of these but only 8.6 of those from math and reading (out of a possible 25 points in those two testing categories), the foundational framework for other learning. Meanwhile, Lincoln earned 37.7 points, 23.6 of those from the 25 possible in math and reading. Note that earned points are calculated by multiplying the possible points per test by the percentage of students scoring proficient and advanced on the PSSA exams.

 

Second, 40 possible points toward the final score are based on the percentage of students “meeting annual academic growth expectations.” This seems to be a meaningless measure in terms of actual progress.  Nonetheless, meeting expectations on each of the four PSSA tests is assigned a weight of 10 points so if 100 percent of students meet expectations on each test, the school gets 40 points.

 

Lincoln students received 29.3 points in the “meeting growth expectations” category, while Kelly students received 31.3 points.  Interestingly, the lowest point count any school can get in this category is 20 because the percentage of students meeting academic growth expectations on each test is set by arbitrary rule so that it cannot fall below 50 percent.

 

A big part of the distortion in final score totals arises from the fact that actual test results, i.e., the percentage of students scoring proficient or above, are weighted at 7.5 points for each PSSA test while  “meeting academic growth expectations” are weighted at 10 percent for each of the four tests. Thus, a school where 100 percent of students score at the proficient level on math will get 7.5 points toward the final Academic Score. If 30 percent are at the proficient level in math, the school will earn 2.25 points toward the Academic Score. This is repeated for the other three PSSA tests. A school could theoretically receive 30 points total from the results on the four tests. The third grade reading weight is set at 10 percent, so a school could get anywhere between zero and 40 points for actual achievement on PSSA exams.

 

At the same time, the “meeting growth expectations” component assigns weights of 10 percent for the four PSSA tests. However, there is an arbitrary minimum guarantee of 50 percent “meeting growth expectations” whether that level is actually reached or not. Therefore, all schools are guaranteed 20 points toward their Academic Score through this component even if not a single student showed any academic improvement.  A school will earn 20 to 40 points in this category depending on how many students above 50 percent meet expectations.

 

In short, the category of “meeting growth expectations” is given higher weights per test than actual performance on the tests.  Further complicating the issue is just how meaningful is the concept of “meeting growth expectations”?  Say a student in third grade falls well short of proficiency in math and in fourth grade the same student is still not proficient. So how do we know whether there has been any academic growth?  If year after year the percentage of students scoring proficient in math is around 35 percent in each grade, would it not be reasonable to assume that, for the most part, it must be the same group of students scoring proficient as they move through higher grades?  Giving points for “meeting expectations” for students who never reach grade level competence is useless and completely misleading.

 

So how do we get 70 or 80 percent “meeting growth expectations” at Kelly Elementary? Or said another way, if children cannot do third grade math, why would one expect they would be equipped to do fourth grade math?

 

And even more astounding, how can the number of students at Lincoln “meeting growth expectations” in math be only 53 percent when 93.3 percent of the students score at the proficient or advanced level?  Indeed, it is even more puzzling considering that 72.3 percent of the students score at the advanced level. This suggests that the procedure for measuring “meeting growth expectations” is irrational, to be kind.

 

The third distortion in the state’s academic rating scheme:  A possible 10 points are awarded based on promotion rate (5) and attendance rate (5). With Kelly’s promotion rate at 97.37 percent, the school earned 4.9 points and with attendance at 92 percent, worth 4.6 points, for a total of 9.5 toward the final score. Lincoln received 9.8 points for the two measures.

 

With promotion rates in most schools at or near 100 percent, and attendance in elementary schools almost always 90 percent or better, all schools are essentially being given 9.5 to 10 points even if students learn almost nothing.

 

And how can promotion rates be close to 100 percent at Kelly when 65 to 70 percent of students are scoring below proficient in math and reading?

 

Adding to this 9.5 to 10 points the guaranteed 20 points from the “meeting expectations” measure gives schools a total of almost 30 points for merely existing.  On the other hand, the scheme does give a modest possible four points for excellence—one point for each PSSA test times the percentage scoring advanced on each test.  Lincoln earned 2.5 additional points on this measure. Kelly earned zero. One would have thought that having 70 percent of students scoring advanced in math would be worth more than 0.7 points toward the Academic Score. Sadly, that is not happening.

 

Without question there is a lot wrong with this rating scheme. It is misleading in the extreme because of its arbitrariness and its underweighting of actual, measured strong academic achievement.  But probably most deleterious is the false sense of academic achievement it gives to schools that score a 70 when 30 points are basically freebies and makes them appear to be not too far behind schools with truly excellent academic performance and receive a score of 85 on the rating scheme.

 

And it also takes away from schools that are sterling performers when the “meeting expectations” measure is determined in a way that punishes high achieving schools.

 

In sum, the Department of Education Academic Scoring system for Pennsylvania schools is a travesty.  One has to wonder how much it has cost in man hours, employee wages and consultant fees to set it up and to run it.

Pension Obligations Are Taxing Property Owners

Pension problems facing school districts have come home to roost.  As we wrote in a recent blog: “Unless there is agreement on pension reform legislation…most school districts in Pennsylvania face ruinous increases in pension funding.”  To handle this increase, districts will have to raise taxes, lay off personnel, or both.  And while the Commonwealth, through Act 1 of 2006, restricts a district’s ability to raise property tax rates, in Allegheny County eleven school districts, 25 percent of the total, have petitioned for an exception to this law meaning they now have permission to increase property tax rates above the Department of Education’s prescribed limit.

 

 

Briefly, Act 1 of 2006 charges the Department of Education (PDE) with setting an inflation index each year that serves as a cap on each school district’s allowable millage increase.  Anything above this cap has to be granted an exception or go before the voters in a referendum.  There are only three allowable reasons for an exception-school construction (grandfathered debt), special education expenditures and pension obligations.  In 2012 the index was calculated to be 1.7 percent for the upcoming school year (2013-14).  Any increases over 1.7 percent, as stated above, require either a voter referendum or an exception from the PDE. 

 

As we had written about in a previous Policy Brief (Volume 12, Number 8), the fact that this is a reassessment year (or the year the reassessments are implemented) in Allegheny County, muddies the water a little bit.  As we stated then, “The act contained a section on property tax limits on reassessment and noted ‘notwithstanding any other provision of law’ that a school board ‘shall…reduce its tax rate, if necessary, for the purpose of having the percentage increase in taxes levied for that year…be less than or equal to the index for the preceding year.’ That does not mean revenue neutral, but within the index determining how much school taxes can increase under the statute.”  Thus school districts in Allegheny County have to roll back their millages to comply with Act 1 by using the index for the 2012-2013 school year (calculated in 2011 which was also 1.7 percent).  If they need to increase beyond the index, they must go before voters or apply for an exception. 

 

For the upcoming school year, 186 school districts across Pennsylvania (of 500) adopted preliminary budgets indicating that they were going to seek millage increases.  171 are seeking exceptions as their desired increase, based on their preliminary budget, exceeds the index while the remaining 15 are holding increases to within the index.  The PDE released a “Report on Referendum Exceptions for School Year 2013-2014” which provides details of the requests.  Eleven Allegheny County school districts made such a request:  Avonworth, Bethel Park, Brentwood, Clairton, Keystone Oaks, Mt. Lebanon, North Allegheny, Pine-Richland, Riverview, South Fayette, and West Allegheny. 

 

All eleven districts cited pension obligations as reasons for petitioning for the exception with three-North Allegheny, Riverview, and West Allegheny-also citing special education expenditures.  In fact statewide, 169 of 171 school districts (98.8 percent) seeking exceptions did so based on the grounds of pension obligations.  Seventy five also asked for an exception for special education expenses (44 percent) and eleven (6.4 percent) for school construction (grandfathered debt).  Clearly pension obligations, the contribution rate for the retirement system was raised from 12.36 percent to 16.93 percent, are straining school district budgets and many are looking to the taxpayers to pick up the tab. 

 

For the eleven Allegheny County school districts, a total of approximately $5.36 million additional tax revenue was being sought to pay for pension obligations.  The requests ranged from $115,200 (Clairton) to $1.38 million (North Allegheny) with an average of just over $487,000.  The PDE approved nearly all of the requests (Brentwood and Clairton were approved but for less than they requested) for an eleven district total approved exception of $5.23 million.  The PDE also has given approval for millage hikes to cover the amounts approved for each school district.  The range covers a low of 0.1987 mills (Keystone Oaks) to a high of 0.3811 (Riverview).  Clairton, which has a separate rate for buildings and land, was approved for an increase of 1.3384 mills on the buildings and 3.9359 on land. 

 

That is not to say that these districts will use their exceptions, but they have permission to do so-after all these calculations were based on preliminary budgets.  In many cases final budgets are still being worked out and may not be approved until the State budget is set at the end of June.  And of course the other districts not filing for an exception can still raise millages, they just have to do so within the parameters of Act 1. 

 

Pension obligations are straining district budgets and there are basically only two options to deal with shortfalls-eliminate programs/staff positions or pass the higher spending onto the backs of taxpayers.  Taxpayers have certainly borne the rising spending burden for a long time but now are pushing back.  How many teachers would have to be laid off to close the budget gaps caused by the increase to pension contributions?  North Allegheny’s shortfall of $1.38 million suggests that dozens of teachers would be on the block. 

 

It’s time for the state to address the issue by allowing some common sense solutions such as allowing districts to lay off teachers for economic reasons and eliminating the teachers’ right to strike.  Likewise teachers need to realize what is at stake and come to the table willing to accept needed reforms of the pension plans to save teaching jobs.