He said "If you were my brother, I would tell you to never dive again" Now I am not looking for medical advice, no doctor-patient relationship here, all rights waived, etc etc...
OK, I think that I contribute enough here for people to realize that I am always happy to help as best as I can. Therefore, I hope that you aren't frustrated because I don't give you a simple answer.
First of all, I really can't tell you whether to dive or not - no one can, but certainly not over the Internet. For legal reasons, for ethical reasons and for common sense reasons. That doesn't mean that I won't try to help by giving you background and context...
Unfortunately, there is no calculation that you can make that says "If you dive within X limits, then your chance of permanent hearing loss is Y%, you make the choice". As you realized, you got your first - likely permanent - injury diving in a pool, not doing an uncontrolled ascent from a technical dive... You may have stressed your ears more than you would during easy diving, I don't know. But remember that even if you "equalize early, often, descend slowly, don't push yourself into the region of discomfort" there is always the risk of a reverse block. Descents are optional, ascents are mandatory. So there is no real way of telling you that the risk is minimal, even with CT scans, surgical exploration, etc..
Now let's look at this from your doctor's point of view. We are all so immersed (pun intended) in the diving world, that it is sometimes hard to take a step back. For most of the people in the world, SCUBA diving is an arcane activity that they know very little about. And if there is a non-zero chance of serious injury, most people would say "OK, so I won't do that, of course!".
Suppose you were told that you have an allergy to turtle brains, and that if you ever eat any you have a 1% chance of a fatal allergic reaction. I guess most of us wouldn't think of that as a big lifestyle imposition, right? We wouldn't think twice about it..
Your doctor realizes (correctly) that you almost certainly have some non-zero chance of further (possibly complete) hearing loss if you continue to dive. In your case, the risk is more than the normal background degree for someone with no history of ear disease. All of us have that minimal background risk, to be frank - it could happen to any of us, even after years of diving. Just like the "undeserved DCI hit" that lurks out there, it is part of the risk you take when you dive. You do what you can to minimize it, but the only way to make it go away completely is to not dive.
So it is not reasonable for you to expect your doctor to reassure you that it is OK to dive. He is giving you his best advice. A doctor who dives might well tell you the same thing. You are the one who has to put it all together... and as much as it would hurt me never to dive again, I would think long and hard if I had any sort of hearing loss after diving... I can't comment on your specific situation, but I hope that you get what I am saying...
Mike