16-year-old: Whew, I am glad that’s finally over! How did I do?
Examiner: It’s hard to say. You’ll have to wait and see.
16-year-old: What do you mean? What is there to wait on?
Examiner: We have to see how everyone else in Oklahoma taking this same examination does.
16-year-old: I don’t get it. What do you mean by “everyone else”? Did I pass the test or not?
Examiner: It’s not that easy. The state approved new standards for driver’s education last year. Since your examination was based on these new standards, no one can be sure whether or not you earned enough points to pass.
16-year-old: I’m really confused. I did everything you asked me to do. I stayed under the speed limit, used my mirrors and turn signals correctly, and followed all of the traffic signs and signals. I even nailed the parallel parking.
Examiner: I agree. But until we get the results of the standard setting committee, I don’t know if you passed. I can tell you that you earned a raw score of 45 out of 60.
16-year-old: 45 out of 60? That’s a 75%, right?
Examiner: Maybe or maybe not. The examination also included several test items that were being field tested. These are items that the state is trying out but they’re not sure they are valid. They can’t determine this until everyone takes the examination. Therefore, some of these items I assessed today may not count for your final score. I don’t even know which ones.
16-year-old: You’re freaking me out. I don’t understand any of that gibberish at all. How did I lose 15 points anyway?
Examiner: I can’t tell you. I’m not allowed to discuss any aspect of the test with you.
16-year-old: Huh? How am I supposed to know what I did wrong?
Examiner: Your parents will get a final testing report in October. It will give you your performance level and tell you what general areas you did well in and which ones you did not. Your detailed student score report will also provide you with a final scaled score and normative information. But you won’t be able to see individual items that you didn’t do well on. As an examiner, the state doesn’t even allow me to see the final items that will be tested. Someone up top thinks I might cheat and share the information with future driver education students.
16-year-old: Did you say October?!! That’s six months from now! That’s insane!
Examiner: All I can tell you is standard-setting takes time. We don’t even have the committee together that will evaluate the test items yet.
16-year-old: A committee? Can’t you just tell me if I passed or not? I don’t want to wait six months to get my license.
Examiner: Sorry, but that’s how the process works. The state will bring together about a dozen Oklahoma driver education instructors to make expert judgments about the content and skills that a driver should know and be able to do in order to be reported in a specific performance level (e.g., Advanced, Proficient, Limited Knowledge, or Unsatisfactory). This committee will then provide cut score recommendations to the Office of Oklahoma Driver Educational Quality and Accountability (OEDQA) for final review and consideration of cut scores. The OEDQA makes final determinations based on both policy and committee recommendations. This information will be sent to the DMV and they will produce the final reports. You will then know if you passed.
16-year-old: That is the MOST ridiculous thing I’ve ever heard. Why would that take six months?
Examiner: Well, I’m not sure about that. It just does. I know the committee has to analyze the depth of knowledge and difficulty of each driving skill, then compare this with how students as a whole performed on the task, before deciding on what level students have to score to earn a passing score. Skills like parking downhill on a curb might count more than stopping at a stop sign. It’s very confusing, but really smart people are on the committee. It’s important they get it right because these scores are used to evaluate the quality of your teachers and school.
16-year-old: Why would my school get a grade?
Examiner: Well, if you cannot pass a driving test, that’s obviously a poor reflection of the education you have received at school.
16-year-old: Who makes this stuff up? Anyhow, what if I get my final report in six months and find out I failed.
Examiner: In that case, you will need to take a driver’s license remediation course and then retake the examination next April.
16-year-old: What the hell! Are you saying it could be a year or more before I get my driver’s license?
Examiner: Don’t shoot the messenger, kid. It’s the way the system works.
If you think the scenario described above is completely preposterous, welcome to the world of annual standardized testing.
The notion of waiting six months to receive the final results of a driver’s test – or ANY other test – probably seems incomprehensible.
Can you imagine your doctor telling you she would let you know the results of your back MRI in six months? Would a prospective attorney be willing to wait half a year to get his bar exam results?
Yet, in just the latest edition of testing follies, Oklahoma schools have been informed that the release of results of this year’s assessments will be delayed until October. The only exception to this will be the 3rd-grade reading test, which will only be delayed until late July!
Why a six-month delay? I’ll just let the state department explain that:
WHY WILL THE OKLAHOMA SCHOOL TESTING PROGRAM (OSTP) REPORTS BE DELAYED THIS YEAR?
In 2016, Oklahoma implemented new Oklahoma Academic Standards in Reading and Math; new science standards were approved in 2014 (§70-11-103.6). As required by §70-11-103.6a.C, Oklahoma developed statewide assessments aligned to the new Oklahoma Academic Standards. These changes require standard-setting for the new assessments. Standard-setting is the process that allows Oklahoma educators to make expert judgments about the content that a student should know and be able to do in order to be reported in a specific performance level (e.g., Advanced, Proficient, Limited Knowledge, or Unsatisfactory). Committees of Oklahoma educators provide cut score recommendations to the Office of Oklahoma Educational Quality and Accountability (OEQA) for final review and consideration of cut scores. The OEQA makes final determinations based upon both policy and committee recommendations. A call for applicants to serve on the standard-setting committees will be communicated in March 2017. Ultimately, the preparation for standard setting and the sheer number of tests (18) undergoing standard-setting will delay final OSTP reporting until October 2017.
These results will then be used to calculate new A-F report card grades for schools, ostensibly sometime in November or December.
Schools will not only be accountable for this year’s results, they will have fewer than 100 days to analyze these results before the next round of state testing comes again.
The proposed A-F revisions will hold schools responsible for every student’s academic growth from 2017 to 2018, even though the schools will not know how their students performed this year until three months of next year’s school year are complete.
This will make the 2017 assessments virtually useless.
In larger schools, the time necessary to disaggregate these reports by site, department, and teacher is substantial. Even after that process is complete, there will be little time to fully analyze student scores to determine gaps in curriculum or deficits in teacher instruction. There will no time for teachers to collaborate to align curriculum to areas of weakness or standards poorly covered. Schools will have to jump through hoops to ensure students scoring below proficient are provided with extra remediation and intervention. It’s not fair. It’s not reasonable. It’s just nuts!
Another year, another year of testing insanity! Will it ever end?
For now, I guess we’ll have to rely on our kits to get us through.