Lessons Questionable science/validity of the Dunning-Kruger Effect: Implications for competency self-assessment in diving.

[-JD-]

In my continual quest for cognitive stimulation, I ran across a blurb-link to an interesting article on the "science" underpinning the oft-here-referred-to Dunning Kruger Effect. There are multiple threads on D-K in relation to diving and references embedded in others. But, what if D-K is wrong:

Quick executive summary (but read the cites below for yourself):

• Essentially: D-K is probably wrong.
• The analytical technique that produced the model (underpinning the conclusion that lower-skill individuals over-estimate their level of competence compared to higher-skill individuals who tend to under-estimate self-competence) causes the same result for uniform random data - i.e. the process produces the result, not the data.
• Instead, less skilled (or at least less-experienced) individuals tend to over AND under-estimate their competence level by a wider range than more experienced individuals, but most folks generally have a "good" or "adequate"(their terminology - see #3) self estimation with outliers (basically a Gaussian distribution).

Cites:

1. The Dunning-Kruger Effect is Autocorrelation – Economics from the Top Down
2. The Dunning-Kruger Effect Is Probably Not Real
3. Host page: How Random Noise and a Graphical Convention Subverted Behavioral Scientists' Explanations of Self-Assessment Data: Numeracy Underlies Better Alternatives Direct .pdf: https://digitalcommons.usf.edu/cgi/viewcontent.cgi?article=1215&context=numeracy

Questions/takeaways:

• I think we need to remember that "science" is a process. not an absolute (and behavioral science probably more subject than most to "common sense" bias.) It is conducted by fallible humans, who are fundamentally explorers, not omniscient gods, and make mistakes. It is the best process we have for knowing about the world - but Uncle Ronnie's quote seems appropriate - "trust, but verify" [Aside: ironic in that it is evidently a Russian proverb that he was using to reference trusting/verifying Russian actions.]
• Dunning-Kruger's conclusion, irrespective of the validity of the numerical model used to justify it, seems like it should be correct (newbies over-estimating competency), but it appears to be biased and literally half of the full story if cite #3 above is correct. So is this (D-K) common sense an artifact of our observational tendencies? Do we just see the newbies that over-estimate competence and get into trouble and the humble old-hats that turn in consistent solid performance with occasional save-the-day moments, while ignoring the quiet, over-achieving novice and chalking up the occasional failures of the "experts" as just bad luck?
• What are the implications of this new take for SCUBA - specifically training and dive-operators?
  • Is it unfair to treat newbies with a lowest-common denominator mindset to be safe and "productive" or is a more discriminating analysis really in order - pushing the naturals/fast-learners and forcing the over-confident into remediation or at least better awareness?
  • How should we as divers be better assessing our own competency and those of our buddies/teammates? Should we be expecting better self assessment by newer divers and being more evaluative of the "masters"?

[Hoping this is a good place for a fundamental discussion tangential to, but impinging on SCUBA training and diving - Mods, please forgive/move if you feel improperly categorized]

 

[Alaskan Scuba Dude]

In my continual quest for cognitive stimulation, I ran across a blurb-link to an interesting article on the "science" underpinning the oft-here-referred-to Dunning Kruger Effect.

Interesting and unusual concept. Unfortunately, there is no rationale which translates to the Dunning Kruger Effect when it comes to scuba training. You either pass or fail when it comes to training.

[Hoping this is a good place for a fundamental discussion tangential to, but impinging on SCUBA training and diving

In the Library of Congress, in a dimension yet to come, academics from various fields will come together to thoroughly examine the mindset of the objectionable individual who posted nonsensical comments archived by the defunct ScubaBoard.com.

 

[KitHuff]

My first reaction to reading about Dunning and Kruger was to start laughing at this ridiculous theory. Then I started thinking about how and why it might be right or wrong. I decided my reaction is based mostly upon how I feel about my knowledge and competency with any new skill. I add in what I see in others around me, what they are doing, and my perception of what they are thinking.

I would like to see how age affects the graph. I would also like it to take into account people's financial status as well. I think we would see different graphs for each of these different demographics. Overall, I think this particular graph is more about people who have been "successful" without a lot of hard work and failure. I think it does reflect the demographic that Dunning and Kruger would have been most exposed to in a college atmosphere.

I am not trying to minimize anyone's life experiences. I think that the harder someone has to work to attain something and the more adversity they have to overcome in order to achieve something, the more realistic they will be about their abilities and knowledge.
-----------------------------------
With all that I disagree with the DK theory, DK has nothing to do with training. It is not my evaluation of my skills that decides if I pass or fail. If the dive operator tells me that I cannot dive, it is their decision and not mine.

The only thing DK is good for is to remind us to be better aware of our actual skill level. Otherwise it is irrelevant.

 

[Scraps]

I'm not ready to throw away D-K altogether.

Having read (ok, having skimmed and scanned) the references attached to the OP, I still think the D-K effect may be usefully applied to certain categories of skills or knowledge: specifically, those endeavors at which proficiency is not self evident to the people performing them, and especially those that provide enjoyment even when they are performed badly.

Consider the game of golf. I play less than one round a year. Despite my inexperience at the game, I have a very accurate understanding of my skill level. Most golfers at all skill levels generally understand where they stand and what they need to work on to improve. The D-K effect does not explain much about golfers because the game tells them how they are doing on every hole. They know if they can shoot par, and they know about how much they have to cheat to pretend they broke 90.

Sports that involve one-on-one competition and those that are timed or measured similarly promote accurate understanding, even among the unskilled. Technical tasks like fixing an engine or writing a computer program may also be immune to the D-K effect. We generally know when we should call a mechanic or the help desk. The DIY-ers who flood their basements trying to replace a washer on a faucet are probably outliers. Bullfighters and parkour enthusiasts suffer few D-K related illusions for very long.

Other activities do not give objective feedback or permit one to evaluate his or her skill level. And the D-K effect may explain some behavior and perceptions in these pursuits. Activities that one can enjoy while doing them badly seem most susceptible to the D-K effect.

Untrained and untalented musicians who sing in the shower may convince themselves that there are six or seven songs they can sing well. Unless they take voice lessons, they may never learn how little they know about music. The D-K effect (along with alcohol) may explain how they might think it's a good idea to enter a karaoke bar when it's obvious to everyone else that they cannot sing well.

I assert that scuba diving is more like singing in the shower than it is like passing a grammar test or playing golf or fixing a car. I have seen innumerable divers dive badly, unaware of the silt they are kicking up, their unnecessarily high rates of air consumption, their poor trim, their dangling gear, or the other manifestations of unskilled diving. All they know is that they kept up with the DM, saw some cool stuff, and got back to the boat unharmed. The D-K effect may explain why these divers over value the evidence they see tells them they're pretty good divers while remaining ignorant of the many indications that they are not skilled.

 

[100days-a-year]

While I agree there is tendency to overestimate abilities inherent to certain people I detest the use of the D-K effect in conversation as it's usage isgenerally associated with condescending and patronizing Richards.

 

[Akimbo]

Dunning-Kruger Effect is just a fancy name for young males with excessive testosterone and limited experience. Looking back, the fact that I am still here is one of those unexplained accidents of nature.

One can argue cognitive decline, but there is no doubt that I knew it all at 16 and am awed at my ignorance now.

 

[JohnN]

Dunning-Kruger Effect is just a fancy name for young males with excessive testosterone and limited experience. Looking back, the fact that I am still here is one of those unexplained accidents of nature.

One can argue cognitive decline, but there is no doubt that I knew it all at 16 and am awed at my ignorance now.

"When I was a boy of fourteen, my father was so ignorant I could hardly stand to have the old man around. But when I got to be twenty-one, I was astonished at how much he had learned in seven years."--Mark Twain

 

[Kensei]

Dunning-Kruger Effect is just a fancy name for young males with excessive testosterone and limited experience. Looking back, the fact that I am still here is one of those unexplained accidents of nature.

One can argue cognitive decline, but there is no doubt that I knew it all at 16 and am awed at my ignorance now.

It is not supposed to be age related. For example, you should be able to run a D-K type experiment with a group of 40 year olds and supposedly get the same effect. I believe some of the research has basically done that when they were looking for differences in D-K effect size by age.

 

[wetb4igetinthewater]

"When I was a boy of fourteen, my father was so ignorant I could hardly stand to have the old man around. But when I got to be twenty-one, I was astonished at how much he had learned in seven years."--Mark Twain

For me that was 25 when I finally realized my parents did know what they were talking about.

 

[tursiops]

One can argue cognitive decline, but there is no doubt that I knew it all at 16 and am awed at my ignorance now.

“When I was a boy of 14, my father was so ignorant I could hardly stand to have the old man around. But when I got to be 21, I was astonished at how much the old man had learned in seven years.”​

― Mark Twain