Oh wait I see the flaw in the logic. The example of opening a cylinder of air at pressure in a room at 1atm is a compelling anology. But no this is not a fair analogy, the PPN2 in the cylinder is way above 0.79ATM and that drives the diffusion. If you filled a cylinder to 200 bar absolute but PPN2=0.79 (with the rest say oxygen or anything else) nitrogen would not be driven out of the cylinder it would move back and forth freely and randomly.So I am camp it should make zero-negligible difference due to what @inquisit and others have said. Some of @Iowwall's thought experiments are interested though, and potentially relevant. What I beleive it boils down to is the half life of 2 gasses reaching absolute pressure equilibrium when there is a difference (this will be a function of how the 2 gasses are allowed to mix). I suspect this is much quicker than the inert gas half lifes - and hence not a driving or significant factor, especially as we ascend continously rather than instantaneously. Furhermore, our bodies are not like the example of opening a cylinder in a room, biology and perfusion will change this. I believe there are some models out there that model the interchange from perfusing to diffusion but I don't know much about them.
Ignoring biology off-gassing at 6m or 3m on O2 will result in the same off-gassing. I say ignoring biology since there could be effects like higher PPO2 causing more vasodilation, promoting more inert gas diffusion. But no studies have showed effects like this yet.