I always wear gloves. I'm a shore diver, and dive from rocky (lava) coastlines that require a bit of climbing using your hands to grip sharp rock at the entry/exit.
Maybe I've "softened" a bit in middle age, but I've found that water-softened skin combined with hauling my big self (6'5" / 260 lbs) plus gear up sharp lava rock can easily shred my fingers and palms.... good gloves prevent this.
I also shore dive, sometimes walking (barefoot) and climbing (gloveless) over a'a, and even though I am technically middle aged, evidently my 6', 180 lb body does not shred my fingers and palms (and soles). Perhaps I maintain original human skin strength by using said skin regularly like an original human.
I'm of the opinion that gloves in warm water is one of the reasons to call someone a warm water wuss
I'd still dive with you, but only if the fact that I will make fun of you is not an issue