Having read “TEA teacher evaluation plan flawed” by Gary G. Godsey, I did some research to discover what the state of Texas had gotten itself into this time. Basically, local school districts will have the opportunity to evaluate teachers in a new system that includes 20% of the score for student achievement on state tests. That achievement will be determined by a Value Added Model (VAM) when appropriate, which the state of Texas noted would only apply for 25% of its teachers. The primary issues center around whether it is reasonable to rate teachers on student test scores from the high stakes STAAR exams and whether the VAM is actually an effective or fair measure of student growth. I am including some thoughts below.
1. Many teachers do not approve of the STAAR exams as effective or fair measurements for student learning, so how could you expect support for using those exam results to evaluate teacher performance?
a. As a math teacher, I understand that when students can’t read, they perform poorly on the math tests, which is not necessarily a reflection of their math ability. If the tests are not measuring students’ learning accurately, how can that be used to evaluate a teacher’s effectiveness?
2. If all teachers are not evaluated on the same system, there is already a reason to abandon the system. I can argue both sides of this idea. Why should I be evaluated with the VAM when 75% of my peers are not? Why can’t I be evaluated with a VAM?
3. The quote, “run them through complex mathematical equations,” tells me a few things. Most importantly that the designers of the model do not want those involved with the applications of that model to know what is going on.
a. Teachers will not understand how they are being evaluated. Common teachers, as well as most adults, are challenged by higher mathematics, let alone mysterious complexities in equations.
b. Administrators will not understand how they are going to be evaluating teachers, either.
c. The state appears to want mystery to be the guiding principal for revealing ambiguous information that will not be justified and can be statistically skewed to report anything they want. If they want to say teachers are doing great according to VAM, they will be able to. Likewise, they can report the opposite. They will be able to do this because “complex mathematical equations” means pretty much no one will have a clue as to the intricacies of the calculations.
d. I have seen “complex mathematical equations” that the state has used for other things, such as TAKS testing, even though they do not make these easily accessible or allow for evaluation of their equations. My understanding of many VAMs is that they refuse to allow others to see their models and indicate that there are multiple models available for various situations. It is also highly questionable whether competing VAMs would indicate the same performance reviews in the same situations.
4. On the VAM side, teachers should stop complaining about that “one kid” who will have a bad day and do poorly on the state exam. The VAMs I have read about supposedly take an average of many aspects and would not be severely skewed by one child. Thus, the argument becomes, “my entire class had a bad day.” Then, teachers will sound a little silly.
5. Also on the VAM side, if it were possible to see that Mr. X not only had horrible results for state testing with this year’s class, but every class over the last five years, this information should be valuable. On top of that, what if the other aspects of his evaluations show him to be a poor teacher, and attempts to improve him through professional development have achieved nothing. At that point, aren’t we doing more of a disservice to not only the last 5-6 years of students but future multitudes of students as well by not removing this teacher from his assignment? A better education for all students is a valid argument for teacher evaluation, and complaints about VAM in this regard would be less impactful.
6. Because VAM is generally based on previous test scores, more scores equates to a better model. Therefore, if 3rd grade is your first testing year, 3rd grade teachers should NOT be evaluated with VAM. 4th grade teachers are probably in the same boat as they only have one year to build a model from. What is a sufficient number of previous test results before VAM is actually valid? [These points emphasize the claim that only 25% of teachers will be evaluated with the VAM.]
a. A corollary to the previous point would be when a math student enters high school. Test results from previous years do not necessarily lead towards the “new” mathematical study of Algebra. Furthermore, Geometry is completely different than Algebra. Great Algebra students can perform poorly in Geometry and vice versa. These are just examples. Simply put, the VAM may not make much sense as a straightforward continuation of studies to be tested in the next grade. Additionally, since Algebra is the only tested subject in Texas high schools, the vast majority of math teachers could not be evaluated with this system.
b. Similarly, how do you create a VAM result for U.S. History when the kids have not been evaluated in social studies for at least two years and never for U.S. History content?
7. Similarly, teachers need to have accumulated data on which to compare themselves. So, first year teachers should not be evaluated with a VAM. Like the previous point, how long will a teacher need to have been teaching before the VAM is valid?
8. If the VAM system relies on randomized student placement in teachers’ classrooms, there is a problem with the model as this is rarely the case in any school system.
9. I have read that VAMs consider a massive amount of data that can vary from situation to situation. Additionally, they fill in missing gaps with best guesses. Anyone with any knowledge of statistical analysis knows that the more variables involved in an equation, the more challenging it is to determine what the results mean and which variable is causing the results in each situation. I am not saying it is impossible, but a large number of variables in an equation about raw materials in business, for example, would be more likely to produce a usable result than the variables involved with the tremendously complex realities of human beings.
Having read up on VAMs and seen the impressive controversial debates, I am concerned about teachers receiving poor evaluations or being fired based on highly variable, convoluted results. With only one-fourth of teachers even qualifying for VAM evaluation and the other 75% being evaluated with district-by-district and course-by-course varying measures, I would recommend that Texas does not implement the new teacher evaluation system. I envision lawsuit after lawsuit (already happening in Houston ISD). I know that they will implement it, and the federal government will continue to push for it, but I thought I would get my two cents worth out there.
The following are the best reads I found. mostly opposed to VAMs [incidentally, the information I read in favor of VAMs came primarily from the creators of the VAMs themselves and nowhere else.]:
VAMs and the “Dummies in Charge”
EVAAS’s SAS Inc.: “The Frackers of the Educational World”