Ginnie is a pretty amazing place. They do allow OW divers to dive into the “ball room” cavern, although I would advise caution, and maybe even a guide on your first visit. It’s a pretty safe place, but it would be a serious issue if you has to make an emergency ascent from the back of the cavern.
Blue Grotto is another favorite, that is OW friendly. Which is good and bad. It silts out pretty fast when people are there teaching groups. If you can get there on a week day, or very early it is just. Spectacular dive.
Another of my favorites (for open water diving), is Orange grove sink, which is within Peacock Springs state park. Viz conditions vary, but when it’s clear, it’s a geogeous place.
There are a couple places that are good, if you just need a good place to splash. Troy Springs, Fanning Springs and Alexander sprigs are all good for a quick dive, but I would not go too far out of my way to get there.
Another site down near Ocala, is Paradise Spring. It’s privately owned, they seem to be pretty friendly to OW divers. I would certainly call this out as a “cavern” Dive where you need good lights and good buoyancy control, or you risk silting out the place. It has a max depth of 100’ in the Cavern and you can see lots of fossils and even cave crayfish if you are patient and have a good eye. If you really like this kind of diving, I suggest seeking out a cavern class.