ucfdiver
Contributor
Write it as much as you'd like. It still doesn't change the fact that without a standard there's no accountability. If you see an instructor letting a student kneel in the sand to tie in a reel, you can't report that because, as long as the instructor finds it satisfactory, it's within standards.I wonder how many times I will have to write this.
In performance assessment, it is very difficult to write very specific standards that clearly define acceptable performance. With some standards it is not that hard, but with others it is nearly impossible. If you look at the written performance standards for open questions on advanced placement exams, you will see wording that looks very vague and open to interpretation, yet the graders who score those exams (2 graders required for each student) will give the same score on a 9 point scale more than 90% of the time. Give them a 2 point scale (pass/fail), and it would be nearly 100%.
In assessing student performances, the assessor compares what is being seen with a mental image of what is acceptable. The key to consistent inter-rater reliability in scoring is ensuring that all assessors have the same mental image of each scoring level. That is done through training, and it takes surprisingly little time to achieve that level of scoring reliability. Trainees see repeated performances, score them themselves, and learn how the experts scored them and why. A person who has gone through a thorough training program is supposed to have seen enough models of what is acceptable and what is not acceptable that he or she will look at a new performance and be able to score it in a manner consistent with the best experts. It works.
If someone has gone through a solid training program and is scoring students far outside the norm, then it is almost certainly a calculated decision. The instructor knows it is wrong and is doing it anyway. If someone is intentionally scoring incorrectly, it does not matter what the standards say. That was my point earlier when I mentioned the instructor who certified my niece as an OW diver with only one short pool session and one shallow OW dive. That instructor obviously knew the standards did not allow that, but he was willing to violate them anyway.
I would also expand on your statement that 90% of the time grading is similar. While it might be similar, that doesn't mean it's correct. I would suggest reviewing the research around normalization of deviation as I believe it's relevant here. I worked for a major airline for 3 years in the training agency, and it was always one of the biggest fears.
The normalization of deviance in healthcare delivery
Take, for example college freshmen. I almost never saw anyone drink and drive. Fast forward 3 years when they've had one beer and driven home, two beers and driven home, then start to see more and more friends drive home after 5-6 beers over a few hours. I'd bet that if you interviewed 50 people at a bar, you 'd find a large majority feel they're safe to drive home, with minimal relation to a measured blood alcohol level. So you've got consistant grading, but it's also incorrect. That's why the law states not higher than 0.08 rather than "any driver must maintain a sobriety level acceptable to drive".