Prerequisites: O2, first aid, and radio on board; enough room on the boat for you, your gear, and enough room to gear up without tripping over another diver; appropriate safety procedures (check-in, check-out procedures, exit and entry procedures, site selection based on current conditions); drinking water provded; good dive briefing.
Better dive ops have, in addition to drinking water, other drinks and snacks. I prefer a boat with a head and some shade. I want to set up my own gear, but some help schlepping things around and changing tanks is appreciated. Separate rinse tanks for masks and cameras, and better yet, a camera table, are a real plus.
All staff, from the office, to the boat captain, to the dive master, to the mate, should be friendly and helpful. Divemasters should be able to direct you to points of interest or help find interesting critters. Allow appropriate freedom for individual divers. I think this varies based on location and dive site. On a wall dive in Coz when the current is ripping, you need to stay together as a group. You should not have to surface when the first diver runs low on air - buddy teams can ascend separately. In very calm water, open the pool and let us go. I don't mind a reasonable time limit, say 50-60 minutes. It's a business and they need to get the next group out. Just don't schedule them so close together that I have to cut dives too short. Dive sites should be selected to meet the needs and desires of the divers.
I am not as concerned about grouping divers by experience/ability. I am often diving with a group trip from the dive shop and there are often divers of different levels of skill and experience. This means there are some sites we don't visit (like the southernmost sites in Coz), but that's the trade-off and I like the social aspects of group travel. When I want to do those sites, I will go without a group. I also think it is good for less experienced divers to dive with more experienced divers, so they have good examples to copy.
I don't much care where we spend the surface interval, but if the boat's too small to provide a reasonably stable platform, we probably ought to hit the beach.
Mostly, it's about attitude - are they taking care of the customers?
BTW, what do you consider a "cattle boat?" What is it that makes it a cattle boat? Size, procedure, the way they treat customers.