I wouldn’t want to. My wife is my best friend and favorite travel companion.
You touch on an interesting topic, multi-factorial with a lot of variability. I've seen a post on SB by someone who dives almost exclusively with a spouse and if one could no longer dive, the other would probably give it up, as it's a 'couples thing' for them.
Some divers are married to a diving spouse, but their interests differ (e.g.: avid diver doing more difficult dives wed to casual diver only interested in very benign conditions).
Some are wed to non-divers and do compromise vacations. There are a range of different arrangements for that.
Some of us leave the non-diver at home and go cram a bunch of diving into the sporadic trips we want to do. That opens up liveaboards, remote destinations, cheaper trips (e.g.: some divers have lower amenities needs than some spouses). Some tend to travel with a known buddy; some of us go alone (e.g.: it's not a question of whether the spouse is a best friend and favorite traveling companion, because we don't need one for the trip).
And some of us have done a mix. My wife calls it a 'dive trip disguised as a family vacation.' I've taken 2 different approaches to this (taking mother-in-law along on both, which helped).
1.) Cram diving into a portion of the trip, aiming for 4 dives/day, and leave some days afterward free for family time.
2.) Dive 2 tanks in the morning and spend afternoons with family.
Married couples are like individuals; every one is unique, with their own 'psychology.' Some spouses are practically joined at the hip, and some have a range of activities the other's not involved in.
It works both ways. My wife loves to do driving road trips to stay at a non-1st floor ocean view hotel with her old high school buddy, look at the ocean, yak, drive and saunter around some shops, not really planning anything. I'd go mad.
I think the 800 lbs. gorilla in the room, as they say, is the question of how high maintenance (or not) is the non-diving spouse? And how big of an issue is time together for the diver?