For those who dive with the same buddy team or solo, then team protocol or your own decision determines how you deal with the pony bottle.
For OW students, I teach that a "pony bottle" is emergency equipment and should be left on. My reasoning is this: It is crucial that an OOG diver receive gas immediately to avoid a potential furball (e.g. panic or drowning). In a non-solo, unknown buddy (insta-buddy on the boat), out of gas situation, you don't know whether you or an OOG diver will be using the pony (e.g. he grabs your pony instead of your primary). Sucking on an "off" regulator will likely increase OOG diver anxiety, leading to a potential furball (e.g. OOG diver stabs you with his machete while sharks are around to distract you while he steals your primary). Even if the OOG grabs your primary, you may be dealing with a panicked diver--it isn't the time to fiddle with a tank valve or deal with additional tasks that could have been avoided. Therefore, to minimize problems, the backup gas source should be on at all times.
If you're using the bottle to extend range or dive time, it isn't a "pony bottle"--it's a stage bottle. I charge stage and deco bottles, check them for bubbles, shut down, and leave them unpurged.