Close your soft palate and that will solve your issue right quick.
A bit of an anatomy lesson. The Soft Palate is the valve up and behind your Uvula, the dangly thing at the back of your throat. The soft palate closes off the airway between our nose and your throat. Your Epiglottis is the valve that holds your breath, located between your adams apple and lungs.
Having independent control over both will solve your nose breathing issue, and can eventually become second nature. IE you'll just naturally close the soft palate up when you want to keep air from escaping your nose, or when you don't want to inhale through your nose.
To accomplish this, follow this 2 part exercise.
Cover your mouth with your hand and breath out your mouth alone. You should be preventing this with your hand; no air should be coming from your nose (this means your soft palate is closed).
Quickly release your hand and air should escape immediately from your lungs. If it doesn't it means you were holding your breath (epiglottis was closed). Repeat until you get it right.
Part 2 of the exercise. Repeat, hand over your mouth, breath out, but prevent air from escaping.
Now with your hand still covering, and you still trying to breath out your mouth alone, quickly change to allow air to escape from your nose. You should feel a jolt up and behind your uvula. This is the soft palate opening and fluttering as air rushes past it. Your soft palate opens towards into your throat, which is the reason for the jolting feeling.
If there's no jolt, it means you held your breath (closed your epiglottis) then opened you soft palate and epiglottis at the same time. Remember you're shooting for independent control, so try again.
If you can do part 1 & 2 right, then quickly alternate between the two. You now have independent control of both your soft palate and epiglottis.
Now you can focus on which valve you want closed when you're in the water.