Honestly, that 'vote' would highly piss me off! I paid good $ to be on a DIVE trip, and I expect the boat to honor their commitment to me as a paying customer for a DIVE (not football watching) trip. As you can likely tell, I couldn't give a 'rat's azz' about football, but whether it was football or whatever, the other divers don't have a right to vote me out of diving, if it's billed/sold as a dive trip, that's what it better be! If the rest of the guests want to change THEIR plans, that's fine, but I'd better be allowed to do my own dives, and the boat better have a DM stay behind with me too, if I need a buddy.
I agree with you, but this was a GROUP (D2D from scubadiving.com) who had chartered the entire boat. It was their decision, and if I remember correctly (it's been eight or nine years so I can't be sure) it was a unanimous decision. As a captain, I would never change an itinerary for such a thing unless it was requested by all the guests.