So The important Part is too realize how you are feeling, do not ignore it, but also know that what your doing is something really wonderful that you enjoy. If that doesn't work then by all means take the necessary steps and stops and slowly ascend. My Buddy rule is always that anyone can cancel a dive at anytime. No questions asked.
If you get a hard time from your buddy, choose a better one next time.
It is not natural to be breathing under the water and it is part of our primal instinct to be afraid of the darkness, the cold, and the water. These are our own self-defense mechanisms that have kept us alive for thousands of years.
As Mark Twain said... "Courage is not the absence of Fear, but the mastery over it...."
This is a great post! Thank you molksmith.