|
|
News : Catalyst Crib Sheets
Changes to 'value-added' measure may have lowered some districts' scores
This article orginally appeared in the August, 2010 issue of Catalyst. Click
here to see that issue's table of contents.
September 1, 2010: The adjustments are meant to make data more useful for teachers, principals.
Changes made to an important formula for measuring academic performance were reflected on the Ohio Department of Education's annual report card, released last Friday. Now educators around the state are trying to figure out just what the results mean.
Last year’s “value-added” scores surprised to the department's data crunchers, as nearly 90 percent of school districts met or exceeded the state’s value-added standards. The value-added data is supposed to measure how much a student learns in a year; the difference between what a student knew at the end of one year to the end of the next is the "value added." To have 90 percent of districts meeting the state standard for that measure is unusually high.
“Last year, we were scratching our heads and saying, ‘is this for real?’” says Matthew Cohen, executive director for policy and accountability at the department.
Fast forward to the 2009-2010 state report card, released Friday: districts settled back into scores closer to what was achieved in 2007-2008, the first year the data was used on the state report card. About 63 percent of districts met or exceeded the measure this year.
During the past year, the state made some changes in the formula it uses to calculate value-added, mathematically smoothing out unlikely differences between grades. It’s possible that schools just had a great year in 2008-2009, Cohen says, but he thinks the new formula gives a truer measure of value-added performance.
“We’re pretty confident we got it right now,” he says.
Across the country, the use of value-added data is gaining more interest and attracting controversy. Critics warn that it’s difficult to measure teachers’ success on student test scores.
Emmy Partin of the Thomas B. Fordham Institute, which closely follows the report card data, says that while the changes were aimed at the grade level, they could be the cause of the drop in many districts' overall value-added scores. Five of the eight urban districts saw their scores fall this year. Last year, seven of the urban districts had above-expected gains; this year, only two did.
Ellen McWilliams, assistant superintendent for the Akron Public Schools, doesn’t want to write off the decrease to a change in the formula. The district, which fell from above-expected performance to below, owns its performance, McWilliams said, and Akron is looking into what could have gone wrong. The district plans to focus more on math this year, which McWilliams says is one of its weak spots.
Jerry Pecko, the new superintendent for the Toledo Public Schools, noted that the district made incremental gains in many areas. “But am I satisfied? The answer is no.”
Pecko viewed the change in the value-added measure as just one of the areas he wants to improve in the district, and said he planned to delve into the grade-level value-added results.
Ohio currently uses value-added data to determine whether schools and districts are challenging students. Saying that a school “met” value-added means that, overall, student scores stayed at the same level from year to year. In other words, a student who was found to be proficient on state tests in 4th grade would be expected to be proficient in 5th grade, as well. If students improve their test scores, effectively making more than one year’s progress, a school or district is “above.” Less than a year’s progress, and they’re “below.” Two years of above-expected gains can earn a district a higher rating on the state report card; three years below can knock them down a grade.
The calculations are complex, and by using multiple years of data before rewarding or penalizing a district, Ohio is able to control for some statistical shifts.
But in past years, the state still saw fluctuations between some grades and subjects. A report on the Ohio Department of Education’s website outlined the problem--varying standards from grade to grade led to an unlikely range of results. With a student population as large as Ohio’s, the researchers would expect only incremental changes from year to year. But if the tests vary in their degree of difficulty at each grade level, a student who passed with flying colors one year may get a less-than-stellar score the next. This can make it difficult to gauge whether a student learned a year’s worth of material from grade to grade.
This is the issue Ohio tried to address this year, adjusting the formula used to calculate value-added so it better represents true gains and losses. Over time, department spokeswoman Julie Daubenmire says this will make the measure more useful at the building level. As principals and teachers have a better understanding of what’s going on in each classroom, they’ll be able to use the data to make changes in areas like curriculum to address real weaknesses.
This article orginally appeared in the August, 2010 issue of Catalyst. Click
here to see that issue's table of contents.
Search Related Stories
Last year’s “value-added” scores surprised to the department's data crunchers, as nearly 90 percent of school districts met or exceeded the state’s value-added standards. The value-added data is supposed to measure how much a student learns in a year; the difference between what a student knew at the end of one year to the end of the next is the "value added." To have 90 percent of districts meeting the state standard for that measure is unusually high.
“Last year, we were scratching our heads and saying, ‘is this for real?’” says Matthew Cohen, executive director for policy and accountability at the department.
Fast forward to the 2009-2010 state report card, released Friday: districts settled back into scores closer to what was achieved in 2007-2008, the first year the data was used on the state report card. About 63 percent of districts met or exceeded the measure this year.
During the past year, the state made some changes in the formula it uses to calculate value-added, mathematically smoothing out unlikely differences between grades. It’s possible that schools just had a great year in 2008-2009, Cohen says, but he thinks the new formula gives a truer measure of value-added performance.
“We’re pretty confident we got it right now,” he says.
Across the country, the use of value-added data is gaining more interest and attracting controversy. Critics warn that it’s difficult to measure teachers’ success on student test scores.
Emmy Partin of the Thomas B. Fordham Institute, which closely follows the report card data, says that while the changes were aimed at the grade level, they could be the cause of the drop in many districts' overall value-added scores. Five of the eight urban districts saw their scores fall this year. Last year, seven of the urban districts had above-expected gains; this year, only two did.
Ellen McWilliams, assistant superintendent for the Akron Public Schools, doesn’t want to write off the decrease to a change in the formula. The district, which fell from above-expected performance to below, owns its performance, McWilliams said, and Akron is looking into what could have gone wrong. The district plans to focus more on math this year, which McWilliams says is one of its weak spots.
Jerry Pecko, the new superintendent for the Toledo Public Schools, noted that the district made incremental gains in many areas. “But am I satisfied? The answer is no.”
Pecko viewed the change in the value-added measure as just one of the areas he wants to improve in the district, and said he planned to delve into the grade-level value-added results.
Ohio currently uses value-added data to determine whether schools and districts are challenging students. Saying that a school “met” value-added means that, overall, student scores stayed at the same level from year to year. In other words, a student who was found to be proficient on state tests in 4th grade would be expected to be proficient in 5th grade, as well. If students improve their test scores, effectively making more than one year’s progress, a school or district is “above.” Less than a year’s progress, and they’re “below.” Two years of above-expected gains can earn a district a higher rating on the state report card; three years below can knock them down a grade.
The calculations are complex, and by using multiple years of data before rewarding or penalizing a district, Ohio is able to control for some statistical shifts.
But in past years, the state still saw fluctuations between some grades and subjects. A report on the Ohio Department of Education’s website outlined the problem--varying standards from grade to grade led to an unlikely range of results. With a student population as large as Ohio’s, the researchers would expect only incremental changes from year to year. But if the tests vary in their degree of difficulty at each grade level, a student who passed with flying colors one year may get a less-than-stellar score the next. This can make it difficult to gauge whether a student learned a year’s worth of material from grade to grade.
This is the issue Ohio tried to address this year, adjusting the formula used to calculate value-added so it better represents true gains and losses. Over time, department spokeswoman Julie Daubenmire says this will make the measure more useful at the building level. As principals and teachers have a better understanding of what’s going on in each classroom, they’ll be able to use the data to make changes in areas like curriculum to address real weaknesses.
Catalyst Ohio is an editorially independent newsmagazine published by the Community Renewal Society
Our sister publications: Catalyst-Chicago | The Chicago Reporter
Privacy | Usage