To much safety is staying out of the water. ...
I agree. I have worked with some safety "professionals" who expect everything to done such that there is no risk, but they fail to acknowledge that all activities involve some risk. For me, I'd love for the the chance of injury to be less in the water than in the car getting to the dive site.
You should carry a minimal amount of gear (and training ) that (1) is considered prudent by a normal, reasonable diver in that situation, (2) provides you confidence that you can deal with possible risks during that dive, and (3) does not burden you with unnecessary risk.
Naturally, all three can lead different equipment in different environments, but so long as you have enough gear to satisfy (1) and (2) without breaking (3), you should both be as prepared as a reasonable person would be and have your own mind at ease.
Since I'm not feeling like going to sleep yet, I'll give an example,
For an open water dive to 50 feet in some current,
(1) reasonable diver may bring standard scuba gear, a SMB, a knife, and a whistle.
(2) perhaps you are concerned about getting lost at sea and you choose to bring a small marine radio in a watertight case
(3) as long as these items are stored properly they shouldn't be a burden.