My suspicion is your ENT may well be more interested in covering his or her own rear than in helping you make a genuinely informed decision. I have nerve deafness in one ear. I've been 100% deaf in that ear since the day I was born. When I was a kid, I had an opportunity to learn to dive, but the ENT my parents took me to said that was nothing I would ever be medically fit to do. The rascal put a number of other "thou shalt nots" on his list for me, although I really didn't much care about the others.
A couple of years ago, a neighbor who dives and knew it was the kind of thing I'd enjoy told me he suspected I'd gotten some really lousy advice those many years ago. I went to an ENT who dives and specializes in diver's ear problems. After a very thorough exam, he told me it was too bad I'd gotten screwed out of a quarter of a century of diving for no valid medical reason, go have fun, and be careful because, obviously, with only one ear that works, my margin for screwing up is diminished.
I'm no medical professional. I suspect that the temptation to tell a patient not to do something is a handy default for many. If I were in your shoes, I would do my research to find a highly credentialed ENT who dives, and get a second opinion from that doctor. You might start getting a recommendation from DAN, or your LDS may be able to give you the name of an ENT who dives.
Good luck.