I think the decision to go to a buoyant ascent would usually involve the potential for not being able to get there through your own power because you might lack that power for some reason. I would go buoyant immediately, for example, if I felt I were about to pass out.
In all things we do in life, there is an element of risk. Staying in bed perpetually risks bedsores. In preparation for risks, we have to weigh the potential for a bad event, the seriousness of that bad event, and the quality of the options for dealing with it. For dealing with the potential for running out of air during a recreational dive with a buddy, I would say that my experience makes running out of air very unlikely, but it is pretty darned serious if I do, so I make sure my buddies are nearby with alternate air sources and, as a backup, I am confident I can do a CESA from any recreational depth.
When I am on a technical dive with my Worthington LP 108 doubles (think anvils with air pockets), the risk of a loss of bladder is remote (but reasonably possible), and the effects would be catastrophic (I cannot swim those puppies up on my own), so I will not dive without redundant buoyancy.
On a recreational dive with a Worthington LP 85 and a 5mm wetsuit with no weights, I can swim it up with relative ease on an empty bladder, so the loss of a bladder does not concern me. But what if I have a loss of a bladder while simultaneously having a heart attack and losing consciousness? Sorry, that combination is too far remote for me to worry about it.