Good morning
@GaetanBidaud ,
There could be a number of factors at play here. The most worrisome would be contaminated air. If the breathing gas source was the same for both dive trips, I would be concerned for that.
Another explanation could be a condition called alternobaric vertigo. This is where one middle ear space equalizes before the other, typically on ascent, and you're left with two different pressures inside your middle ears. This can lead to transient vertigo, which usually disappears immediately when the offending middle ear space eventually clears. If you're diving with congestion, this would be higher on the list of possibilities.
Another potential cause of post-dive vertigo is inner ear barotrauma, but given your description of it clearing up between dives, this would be pretty low on the differential for me, as would inner ear decompression sickness.
Best regards,
DDM