From quick Googling at
SeaTemperatureInfo.com on St. Croix:
"
Water temperature in Saint Croix today is 80.8°F. Based on our historical data over a period of ten years, the warmest water in this day in Saint Croix was recorded in 2010 and was 83.5°F, and the coldest was recorded in 2009 at 80.1°F. Sea water temperature in Saint Croix is expected to rise to 81.7°F in the next 10 days. May average water temperature in Saint Croix is 81.7°F, the minimum temperature is 79.9°F, and the maximum is 84.7°F.
The swimming season in Saint Croix lasts from January to December, twelve months per year! During those months, Saint Croix water temperature does not drop below 68°F and therefore suitable for comfortable swimming. The average water temperature in Saint Croix in winter reaches 80.4°F, in spring 80.6°F, in summer the average temperature rises to 83.3°F, and in autumn it is 84.2°F."
That page should give you some idea of temperature trends. Then, just avoid the windy late winter period, and perhaps the hurricane season if you don't feel lucky. Looks like June - Nov. are pretty warm.