Agreed. Wind over tide can make a seemingly harmless 3-4ft swell pretty uncomfortable as those waves start stacking up. I'd also trust a skipper who is willing to call a trip rather than hope the divers bail just to keep the fee / deposit.s.
The period also matters.
Is that for the reef/inshore or is that for the gulf stream. Depending on the wind direction it can get a lot rougher out toward the gulf stream.
Also remember that those are not max waves 4 would have some 5s and perhaps few 6s