I had a similar thing happen to me regarding flying in an airplane. I've been flying regularly since I was 14 years old, and, over the last thirty years, have averaged about 10 to 15 flights a year for business and pleasure.
About fifteen years ago, for some unknown reason, I began to feel very panicky about being on an airplane. I still flew, but each flight was a minor terror for me, at least until after takeoff.
After about a year or two of this nusiance, the feeling gradually passed and now flying doesn't bother me at all. Why my flying phobia came, and why it went away, I don't know. I had no bad expereinces on a plane. The only thing I could thing of was that, at the time, both of my daughters were still very small, and I may have had a bigger concern about the impact of my death on them. The good news is, these problems can often go away with time as our situations change.
If you really WANT to dive, this fear can be handled like other phobias (a phobia is a severe fear of something, like fear of high places, or fear of snakes) --- by slow, gradual and supervised exposure to what you are afraid of. Start by swimming a lot, then swimming with a mask, then snorkeling, then supervised pool dives, etc. You might try simple things, like showering with your mask on. The key is going slow and having a lot of backup people around so you are reassured you are not in trouble. But don't force the issue...you are young and have plenty of time to dive.
I don't know your medical background, but certain medications or drugs can also induce panic attacks. You might consider a medical checkup. Also, if you find other activities begin to cause panic problems, not just diving, you should definitely seek medical advice.