I believe....
I don't cave dive, but I did when I was younger and foolish and ill trained. Thank God, I was also very lucky and was able to come to my senses BEFORE I came to my demise.
I thought I was smart, but I was just a dumb teenager, in love the idea that it was a cool thing to do.
I know 3 people who died cave diving and none of them were trained or qualified. Unlike me, they died before they learned the error of their ways.
Oddly enough, I am now a geologist and have an interest in such things (caves, karst topography), but time, money and two small kids keep me from oursuing the training I would need.
But, I know others who do pursue this sport, including a good dive buddy, and I believe that with their training and their equipment, the risks they face are no greater than that of many other sports considered "safer" in the traditional sense.
Cave diving, flying ultralight aircraft and sky diving are all considered dangerous by the general public, but all of them are reasonably safe when practiced correctly.
What common "sport" would I consider more dangerous than these "extreme" pursuits?
Riding motorcycles on the street comes to mind first. If you are in a cave, or flying a Weedhopper or jumping from an aircraft, your safety depends on your equipment and your ability to make decisions, both before and during the activity. On a motorcycle, your safety depends on a critical factor well beyond your control: The other guys sharing the street with you. Let me tell you, the number of idiots on the street is something you cannot control or deal with effectively.