Not exactly an experienced diver, but the closest I came to real panic was when, during the descent on a night dive, I descended into a school of anchovy-like tiny fish that were shimmering in the light thinking it would be real pretty to watch them move away in a group.
Apparently, they didn't get the notice, and simply enveloped me, leaving me unable to see anything but mad, thrashing silver fish and flashing lights - most of the other divers were also left unable to see.
All that was still ok, until one fish found it's way into my ear and thrashed about madly trying to burrow through. I literally had to poke my fingers as far as I could into my ear, then pinch the end of the tail and yank it out. Couple of seconds later, one went into my other ear and I repeated the same thing. I basically sank to the (fortunately shallow) bottom with my hands tightly cupped around my ears and waited for the school to disperse.
Seems quite silly, but trust me, it is really quite terrifying to feel something writhing away inside your head, and not knowing if they have sharp teeth to bite the insides of your ears. Because of that, I wear a buff over my ears for all night dives now.