There are many factors that may contribute to narcosis and many people experience it differently.
Narcosis affects divers like alcohol—you feel fine, but actually experience progressive impairment beginning with the first drink (or couple of atmospheres).
Drinkers function with narcosis better than non-drinkers because they are used to being impaired.
CO2 build up due to exertion will greatly affect narcosis. It’s easy to see why when looking at the degree of narcosis based on gas:
Note how CO2 is much higher than other gases. A slight increase in CO2 can significantly add to narcosis (and He significantly reducing it).
Narcosis adds to your baseline level of anxiety. If you’re nervous, a new diver or not fully comfortable in the underwater environment, narcosis only compounds one’s anxiety. Despite the tables and physiology being the same, the psychological burden is a contributing factor. For instance, drysuit diving with heavy weights in dark, cold water with low vis is quite different that diving in warm clear water wearing little more than a t-shirt and bathing suit. Experienced divers are more comfortable than newer ones, so the anxiety baseline is less.
Similarly, seasickness medicines, fatigue, drinking alcohol or not getting enough sleep the night before are contributing factors.