You would be called back, as soon as your divers got out of the water.Yes, it matters, and if the port had closed for the size boat you were on after you had left the dock, you would have been called back.
Last year we were out fishing across the channel when the word went out that the port was closing because of the approach of a large strong thunderstorm. We were disappointed at having to hook up and run home, but after we had been ashore for 15-20 minutes we were glad we did!