Dave’s Photo Blog

The results of the 2008 MCAS tests for the Thompson School have been reviewed with parents several times. At the start of the school year, principal Sherri Donovan discussed them in an orientation meeting. There have also been letters sent home to the parents from both principal Donovan and Interim Superintendent Kathleen Bodie. However, the information in those letters has been slightly different. Like many parents, I’ve looked up the data independently. Here it is.

Graph Methodology

When graphing this data, I did several things very deliberately to avoid any misinterpretation of the trends of the lines. Specifically:

• The complete range of values is represented by the Y-axis (vertical axis). If I had only shown part of the range, the slope of the lines would increase. This would make the trends look worse than they are.
• There is no data for MCAS math testing prior to 2006 because that was the year the 3rd grade math test was phased into the overall MCAS testing program.
• The scoring for English Language Arts (ELA) changed from a weighted system in 2002 to the current scale of 1 to 100 in 2003. Thus, scores prior to 2003 are not shown.
• The overall size of the charts is identical. This means that if you draw a rectangle around the plot area, the horizontal and vertical dimensions are exactly the same size.

Interpreting the Results

One cannot jump to a conclusion about the meaning of these numbers without a full and deep study of the factors that contribute to the overall score and rank. Numbers are so often used to misrepresent complex situations. This is a reasonably complex situation. Here are some factors that have a significant bearing on the MCAS results:

• The sample size is extremely small. Remember that we are talking about roughly 40 third grade students in the Thompson School. They are being compared against all the third grade classes in Massachusetts whos membership ranges from just a dozen or so to almost 300 students. And those number change every year as the classes advance through the schools.
• Other schools’ performance can impact the rank for the Thompson School. This makes direct comparisons of the charts very difficult and helps explain situations like the math performance from 2006 to 2007. The score was identical both years at 68%, but the rank dropped from 147 to 225.
• Changing demographics in the classes can have a big impact on the scores and rank. For example, if a group of underperforming students were to move into a particular school district during the school year, they could cause a significant drop in the MCAS rankings. In that scenario, it would be very wrong to point blame at the teachers, curriculum or other factors that are considered to be controllable. Similarly, each year a different set of students is tested and compared against the prior years class.

Thompson School 3rd Grade MCAS Scores, 2003 – 2008

Summary of Results

There are three trends of interest in the above chart and I’ll do my best to summarize them here

2005 decline in ELA score

The drop in 2005 should be viewed in the context of that class as a cohort. Indeed, if you follow that class through the 4th grade scores in 2006 and the 5th grade scores in 2007, you will find steady, year over year improvement. This is probably one of the reasons that a single year drop in scores does not result in a requirement for an anticipated yearly progress (AYP) program. It should be expect that there will be years that are lower – the membership of the grade is completely different from year to year. It would be unfair to expect that every 3rd grade class is composed of better students year over year. I don’t believe that this inflection is any cause for concern.

2007 and 2008 decline in ELA score

This trend is a bit more troubling and should be addressed by the school. Summarizing the point above, a single year drop does not define a trend, but a year over year drop of the magnitude we see here does. It is crucial to understand why that trend has occurred and to address those root causes. Until I have more information, I am unwilling to render an opinion. I would also point out that the students in those two classes have moved on to 4th and 5th grade where there hasn’t been an indication of decline in scores. To me, that implies that the curriculum is sound as students advance up grade levels. However, it is important to understand why this trend has happened as it could indicate that there are problems with students’ performance in the lower (pre-third grade) levels. Those grades are not tested in the current MCAS testing program.

2008 decline in math score

Again, this decline is not a trend, it is a single year drop that could occur for a variety of reasons. As such, I would be interested in principal Donovan’s perspective on this drop. She has not discussed this result, probably for the reasons I’ve stated here. It may well be immaterial.

Thompson School 3rd Grade Rank Statewide, 2003 – 2008

Sources: Boston.com reporting from the Massachusetts Department of Education: 2003, 2004, 2005, 2006, 2007, 2008.