All of the above recommendations are excellent for Humpback Whales. If Humpbacks check the box for you, then please read on. Just be forewarned that this reply is long.
Barring some unusual circumstance, Silver Banks, French Polynesia (includes Rurutu, Moorea and Tahiti) and Tonga are pretty close to a sure thing. I've done Silver Banks twice, Rurutu once, and Tahiti once. I swam with Humpbacks at all of them. I have never done Gorgona in Colombia,, Moorea in French Polynesia nor Tonga.
Of the ones that I have done, the one that I liked best was Rurutu.......by far. In most other places where you can swim with Humpbacks, you go out and motor in open water looking for blows. If it is rough out, it isn't the most pleasant way to spend 4-8 hours of your vacation time. Also, when it is rough out, the blows are much harder to find. And even after sighting a blow or blows, a water entry is not necessarily a sure thing: if the whale or whales are on the run, all you might get is a fleeting glimpse, at best.
So why Rurutu? When we did Rurutu in September 2022, we motored about 45 minutes by boat to a bay, and once at the bay, we always found a Mother and calf sleeping there. For five straights days. And the encounters typically lasted for hours.
BUT.....the cost: airfare alone will probably run you close to $2000 roundtrip per person from the US to Papeete on the island of Tahiti, unless you can find something cheaper. You may have to overnight in Papeete before flying to Rurutu the next morning. Add another $150 for hotel plus maybe $700 roundtrip per person for the dometic flight. In Rurutu, transfers, lodging, meals, and daily snorkeling was close to $2000 per person total for the five in-water days. We stayed at the Vaitumu Lodge. Perhaps staying at a pension might be less.
It doesn't get a lot better costwise if you do Silver Banks (liveaboard out of Puerto Plata in the Dominican Republic). A one-week trip with 4 1/2 days of snorkeling runs around $4500 pp and up before tips and airfare.
Costwise, Tahiti and Moorea are a possibility - no domestic flight required. But you may find that land expenses are more.
Bottom line, all of the above are right up there in the $5000 per person range, which is at the limit of what you wish to spend. I encourage you to price these options on your own and see if you can do better.
If you can't find anything less expensive, one iffy suggestion (of two) that I have for you is the Sea of Cortes in Mexico in the May-June timeframe. Here, I have been in the water with Humpbacks and Blue Whales and have even seen Fin Whales and Sperm Whales from the boat. Fin Whales are nearly impossible to get into the water with - they're too skittish. Sperm Whales, it depends on the operator. Some allow it and some don't, altho the official Park rules prohibit it. BUT....these whale encounters are a lot less of a sure thing as Silver Banks and French Polynesia.
One other consideration about the Sea of Cortes is that May-June is mobula season. That, in of itself is quite a spectacle that can quickly become an experience of a lifetime if orcas show up to feed on the mobulas (I've tried three times, saw the schooling mobulas but not the orcas).
The other iffy suggestion is the Sperm Whales of Dominica in the Caribbean. I did this this past March for 6 days and swam with the whales everyday. Hereagain, the issue is cost: most operators offer packages that run from $5500 and up per person plus airfare. However, you might look into the Ft. Young Hotel in Roseau, Dominica, which earlier this year was offering an all-inclusive one-week package with whale swimming for under $3000 plus airfare for the Summer just past. The only caveat is that most of the other operators offer whale swimming packages only during the Winter months so it makes you wonder if it's any good during the Summer. I can't answer that but I do know that the whales are resident to the area so it stands to reason that they might be around.