I agree with everyone else. My wife and I (and our children when they still lived at home) have been there many times, and I am the only diver in the group. In fact, the reason I took up diving in the first place is because we decided we liked Cozumel so much we decided to vacation there regularly, and none of use were divers then. I figured that if I were going to be vacationing regularly in one of the world's greatest dive locations, I'd be damned if I would miss out on it. The diving turned out to be so spectacular that it turned into much more than than the annual excursion I had originally intended, and we still love everything else on the island.
In my opinion, you should avoid the all-inclusive places. They may be lots of fun, but they could be anywhere in the world--you would not be experiencing Cozumel. Explore the island. Eat at the many fine restaurants in San Miguel. Meet the people.
One thing no one mentioned is the safety factor. I really don't think there is anything like crime to speak of.
I once spoke with someone who knew someone who had been on the police force for years. He said that they had once arrested a belligerently drunk tourist at night and released him in the morning. That was the only arrest while he was on the force. That is a third hand story, but I don't doubt it. You never even see police except when they are stopping traffic in town to let pedestrians cross the road. The exception to that was the night my son decided to walk the couple miles back to our hotel from downtown late at night. He said four police cars stopped by him during the walk to ask if he was OK. He felt that he was being monitored for safety the entire time.
Compare that to Bonaire, where you are immediately warned upon reaching the island not to leave any valuables unattended in a car or truck.