Thompson School 2008 MCAS Results
The results of the 2008 MCAS tests for the Thompson School have been reviewed with parents several times. At the start of the school year, principal Sherri Donovan discussed them in an orientation meeting. There have also been letters sent home to the parents from both principal Donovan and Interim Superintendent Kathleen Bodie. However, the information in those letters has been slightly different. Like many parents, I’ve looked up the data independently. Here it is.
Graph Methodology
When graphing this data, I did several things very deliberately to avoid any misinterpretation of the trends of the lines. Specifically:
- The complete range of values is represented by the Y-axis (vertical axis). If I had only shown part of the range, the slope of the lines would increase. This would make the trends look worse than they are.
- There is no data for MCAS math testing prior to 2006 because that was the year the 3rd grade math test was phased into the overall MCAS testing program.
- The scoring for English Language Arts (ELA) changed from a weighted system in 2002 to the current scale of 1 to 100 in 2003. Thus, scores prior to 2003 are not shown.
- The overall size of the charts is identical. This means that if you draw a rectangle around the plot area, the horizontal and vertical dimensions are exactly the same size.
Interpreting the Results
One cannot jump to a conclusion about the meaning of these numbers without a full and deep study of the factors that contribute to the overall score and rank. Numbers are so often used to misrepresent complex situations. This is a reasonably complex situation. Here are some factors that have a significant bearing on the MCAS results:
- The sample size is extremely small. Remember that we are talking about roughly 40 third grade students in the Thompson School. They are being compared against all the third grade classes in Massachusetts whos membership ranges from just a dozen or so to almost 300 students. And those number change every year as the classes advance through the schools.
- Other schools’ performance can impact the rank for the Thompson School. This makes direct comparisons of the charts very difficult and helps explain situations like the math performance from 2006 to 2007. The score was identical both years at 68%, but the rank dropped from 147 to 225.
- Changing demographics in the classes can have a big impact on the scores and rank. For example, if a group of underperforming students were to move into a particular school district during the school year, they could cause a significant drop in the MCAS rankings. In that scenario, it would be very wrong to point blame at the teachers, curriculum or other factors that are considered to be controllable. Similarly, each year a different set of students is tested and compared against the prior years class.
Thompson School 3rd Grade MCAS Scores, 2003 - 2008
- Thompson School MCAS Scores, 3rd Grade
Summary of Results
There are three trends of interest in the above chart and I’ll do my best to summarize them here
2005 decline in ELA score
The drop in 2005 should be viewed in the context of that class as a cohort. Indeed, if you follow that class through the 4th grade scores in 2006 and the 5th grade scores in 2007, you will find steady, year over year improvement. This is probably one of the reasons that a single year drop in scores does not result in a requirement for an anticipated yearly progress (AYP) program. It should be expect that there will be years that are lower - the membership of the grade is completely different from year to year. It would be unfair to expect that every 3rd grade class is composed of better students year over year. I don’t believe that this inflection is any cause for concern.
2007 and 2008 decline in ELA score
This trend is a bit more troubling and should be addressed by the school. Summarizing the point above, a single year drop does not define a trend, but a year over year drop of the magnitude we see here does. It is crucial to understand why that trend has occurred and to address those root causes. Until I have more information, I am unwilling to render an opinion. I would also point out that the students in those two classes have moved on to 4th and 5th grade where there hasn’t been an indication of decline in scores. To me, that implies that the curriculum is sound as students advance up grade levels. However, it is important to understand why this trend has happened as it could indicate that there are problems with students’ performance in the lower (pre-third grade) levels. Those grades are not tested in the current MCAS testing program.
2008 decline in math score
Again, this decline is not a trend, it is a single year drop that could occur for a variety of reasons. As such, I would be interested in principal Donovan’s perspective on this drop. She has not discussed this result, probably for the reasons I’ve stated here. It may well be immaterial.
Thompson School 3rd Grade Rank Statewide, 2003 - 2008
- Thompson School MCAS Rank, 3rd Grade
Summary of Results
Basically, I think this measurement is so compromised that it is almost worthless in the context of the situation at the Thompson School. The ranking of the grade against just over 1,000 other third grade classes is bound to be problematic. Think of it this way - if there are 300 schools with a score of, say, 70% on ELA and the Thompson School drops from 71% to 69%, the school’s rank will drop at least 300 points. In that case, the rank is meaningless. Nobody wants to see their school’s rank drop, but without understanding the distribution of scores from all of the schools, you can’t draw any conclusions about the change in rank of a particular class against the entire population year over year.
Conclusions
I commend Principal Donovan and Interim Superintendent Bodie for communicating with parents about the issue and offering an in-person meeting to discuss these results. The two-year drop in ELA scores for the Thompson School deserves closer attention. It could represent a trend that has an impact on the early-learning experiences of children in the school. However, it is unfair to blame the faculty, staff, students or anyone else until we have a better understanding of why the trend has occurred. We’re only heard bits and pieces of that explanation to date and although I’m not yet comfortable that I understand it, I’m looking forward to Monday’s meeting for a deeper discussion on the topic.
The MCAS results are particularly prone to misinterpretation. My advice for those parents that are concerned over declines like these is to stay calm, look deeply at the underlying data and remember that the comparisons made in both scoring and ranking are complex relationships.
The MCAS rankings are not helpful to the overall understanding of most schools’ performance. They are most meaningfully employed by those in the top and bottom 10% as a source of pride or anguish. For the rest of the schools (the middle 80%), they are almost useless.
Sources: Boston.com reporting from the Massachusetts Department of Education: 2003, 2004, 2005, 2006, 2007, 2008.