With testing season wrapped up and students out of school for summer, questions have been posed about who exactly is scoring the tests. Over 40,000 scorers will be tasked with grading the tests of over 100 million students. How does Pearson intend to attract quality personnel to ensure proper scoring? The Washington Post detailed the process earlier this week:
Standardized test scoring, the kind done by human beings when written answers are required, is a mysterious, mostly hidden activity. The best scorers assess free-response answers on high school Advanced Placement tests. I have seen them in action at summer scoring sessions across the country.
The AP scorers are experienced teachers and professors. A somewhat different bunch has just embarked on the nation’s largest human scoring experiment ever, and I am not sure how it will turn out. The companies handling free-response questions for the Common Core-based tests — the education reform of the decade — are hiring and training far more people than have ever done such work, and many of them have little relevant experience.
Catherine Gewertz of Education Week gathered the numbers. At least 42,000 people will be grading 109 million student responses in the 28 states plus the District that are part of two large federally funded test-creating consortiums. That is nearly four times the previous record: 13,000 readers who scored 17 million AP responses last year. It is 26 times the 1,600 graders who scored 1.7 million SAT writing-test responses in 2014. And it doesn’t include many states creating their Common Core-based tests in other ways.
Hiring rules differ from state to state. Most seem to require that scorers have bachelor’s degrees, but not necessarily in the subjects they are grading.
Can anyone else see this ending in disaster? Let’s go on:
The Common Core graders will be paid differently in each state, roughly in the range of $12 to $15 per hour. Maryland and the District have Common Core exams; Virginia does not.
I’m going to go out on a limb here and guess that $12 to $15 an hour will not be enough to attract quality personnel. If they can’t attract quality graders, test scoring will be inconsistent and inaccurate.
At an Ohio Common Core scoring center run by one of the testing companies, Pearson, Gewertz saw a third-grade math question: Inspect solutions by two fictional students, say which is right and explain why. Pearson graders were expected to evaluate 50 to 80 third-grade math answers an hour.
As has happened with the old state tests, it is likely that some graders will say they are not being given enough time to do a good job. One expert told Gewertz the testing companies will have difficulty finding enough experienced scorers to supervise the newbies.
Many Common Core style questions are complex word problems which force students to explain how the arrived at their solution. When graders who may not be familiar with the subject matter that they are scoring are told to complete one question per minute… it’s not too hard to figure out that this is going to be a fabulous disaster. With public perception quickly turning against the Common Core, the scoring of the test could become yet another black eye for the educational initiative.