Thanks for the comments everyone.
When I first started after the divers, I didn't realize there were 3 of them. One outside, two inside. My back was to them when they swam up so I thought there was only one in the cave. I figured he wouldn't go too far since his buddy was outside and seemed a bit scared. I thought the other one might have just been brave enough to take a quick peek inside.
When I dropped down into the first cavern area and saw no sign of him, I swam in a bit more to see where he went. At this point I hadn't even bothered turning my light back on because I expected to find him right there. Instead, I saw TWO of them inside with some small handheld HID's and I watched as one of them swam OVER the top of the sign so close his knee almost hit the edge of it.
They took off swimming towards the back and I ended up near a reel that had been placed on the mainline to complete a circuit. They stopped and appeared to be debating following this new line to see where it went. That's when I caught up with them and as I closed the distance I turned my 24w HID on. (There was still enough ambient light from them for me to see my surroundings).
To say the least they were startled when the cave suddenly lit up. They had no clue I was there. I signaled to them to get their attention and pointed to the exit. Then I waited for them to pass me and I followed behind to make sure they both got out.
When we surfaced, I told them "Guys, I'm not trying to be a d**k but you have no business being back there. That's a good way to get yourselves killed. If you want to see whats in the cave, go get some training."
One of them responded with "Yeah, I was scared. That was creepy, don't worry, I'll never do that again. I'm not stupid." I replied with, "Well I won't call YOU stupid, but that sign you almost ran smack into pretty much said it was stupid to dive any further without appropriate gear or training."
While I was inside with the other two, Donna brought the third one to the surface and was busy scaring the beejesus out of him. The three of us stayed at the surface until the three of them started to swim back to the boat. Then we exited and watched until they motored off.