Well, you could take a piece of elastic a little less than the circumference of the port. And cut a circle of thin neoprene large enough to cover the port and however far you want it to go down the sides. The you would stretch the elastic to fit around the edge of the neoprene (overlapping the elastic a little) and sew it on. The edge of the neoprene will be longer, so what you do is mark both, say 8 marks evenly spaced around the circle and on the elastic. Pin it in place at each mark. Then when you sew, you stretch the elastic so it can line up with the edge of the neoprene and be more or less even This might work for a small port. You might try it with a piece of plain fabric first for practice and to see how it works out.
The other thing you could try is to make a tube to pull the elastic through. Like use a long enough piece of lycra and sew the edges to the neoprene, or better yet fold it in half over the edge and sew it around the edge, as that would finish the edge too. Then you need to snake the elastic through the tube (use a safety pin) and tie or sew the ends together. Does that make sense at all?
For something bigger, sewing elastic around a simple circle won't work very well as it will get very bulky and the elastic might not even stretch far enough to make it around the circle. The large commercially made ones will generally have shaped wedges of neoprene pieced together to eliminate the extra material. Or you could cut wedges out of what would be the sides and sew the new edges together. Not sure if I have described that well either?