...my spare mask lives in my left pocket along with the other stuff that I don't expect to use... like my spare cutting device, spare double ender, spare p-bolt and dive alert. The stuff is always there but never needed if everything goes as planned.
Yes... I can do ascents without a mask. I have even done training ascents sans mask with a deco obligation making the necessary stops and switch to the deco bottle (buddy calling the 70' stop.)
But should my mask get kicked off I would prefer to get out and don my spare, locate my primary mask, make the switch, re-stow the spare and continue the dive.