I used to have the same problem, just feeling
a bit dizzy when ascending. Never really worried, since someone told me it was a matter of ears not equalizing at the same spped.
BUT on a dive (approx. 1 year ago) I had the worst diving experience ever. I had this dizziness getting stronger and stronger, so I stopped the ascent, signaled my buddy that something was wrong with aqualizing. I stopped and waited, but even though I stayed at exactly the same depth, the dizziness became pain.
I was loosing equilibrium, I couldn't go shallower nor deeper, everything was swirling around me; and I nearly passed out ! Took me (believe me) 30 minutes to be able to get back to the surface, trying to bear with the pain (thank God I'm not a big air-consumer

)... Worst dive in my whole life.
My diving doc told me it was a problem of ears not equalizing at the same speed, and he adviced that I should take great care of them (i.e. rinse them with bottled water straight after sufacing, using some kind of spray on a weekly basis, etc)
I think you should be careful about that, cause this "little dizziness problem" I underestimated looks a lot like yours, and it's not something fun.
