Knotical summed it up very well on all points. My wife and I both used fleece suits which we obtained about 10 years ago for tropical diving. Mine was an Aeroskin and by wife had an Oneil fleece that was not as thick as the Aeroskin. The fleece suits are very comfortable. I originally used the fleece as a solution due to my height (67) since it was flexible. I have since found wetsuits I can use.
I have used the fleece under a wetsuit since I didnt have a 5 mil at the time. It worked ok. A thicker wetsuit would be better.
I found early on that I tolerate cold much better than my wife so I either dive in surf shorts, a thin skin or a wetsuit depending on temps. My wife gets cold in 85 F water so she now uses only wetsuits and now uses a 3mm Henderson Hyperstretch (plus core warmer if necessary). I am not sure why but both of us have stopped using the fleece.
The only downside to the fleece suits, as mentioned, is that it takes longer for them to dry out. If we are on the edge of airline baggage weight limits flying to the location then unless the fleece is totally dry (more time), it can put us over limits coming back. Hand wringing the fleece suits can be an exercise to speed up the drying process before hanging. As an aside, my wife did not like the fuzz from the fleece coming off while wearing it.