I was diving with a group of former students, and one of my former students came to me laughing after having finished his dive. He said I reminded him of an old vaudeville joke. A man tells his doctor that it hurts when he does something, and the doctor says, "Don't do that."
What he was talking about was that earlier in the day he had described a situation similar to the one in this thread, and he asked what he could do about it. He said my response was along the lines of "Don't do that." He had just done a dive, and my advice worked perfectly.
What you are describing is very common with new divers. It is very much like a new bicycle rider tipping over. You'll eventually get over it without doing anything more than trying not to do it. You especially see it when people first learn to dive with steel double tanks. All that weight on top of them keeps tipping them over. When I first started diving doubles, I wondered if I was wearing the tanks or the tanks were wearing me. Then, suddenly, like a miracle, that all goes away.