I know this area well and have dove where you are talking about a bunch.
On the first map, the little cove to the right of the * is the one in between the 2 restaurants, with the sea wall, and the one with the '9' is just to the right of the second restaurant, toward halibut. I have never entered there, but have dove there by swimming from folly or halibut a bunch of times - you are right it is pretty cool there, a bunch of huge slabs of rock. It seems that is a collection spot during a storm as well - I've seen hundreds of mangled lobster traps there amoungst the rocks, I don't know if it is still full as I have not been there for a while. It does get quite deep quite fast there as you have noted.
Be cautious of the house there if you are trying to enter - I have heard stories about them not being happy with divers trying to enter near the houses. Where did you enter?