Summary of posts so far for those of you just joining:
1. Leave the air on or off, but be consistent.
2. Leaving it on will not damage your reg (yeah!), but in really extreme conditions (Norwegians diving in Egypt!) you could blow an o-ring (bummer, but not generally a catastrophe).
3. By leaving it on you may detect a leak and you reduce the chances of diving with your valve off (not so good).
4. If you turn your air off, purge so you reduce the likelihood of sucking H2O (very bad day, indeed).
5. Do not omit pre-dive checks. Make sure your air is on before you plunge. Breathe your regs and watch your air pressure.
I think those are the high points.
So, what will I do in the future? Unless there is a compelling reason, going forward I will leave my air on until I dive. My reg doesn't leak air and it seems to me the most extreme inconvenience is that I will have to replace an o-ring, which I have to do about 25% of the time on a rental tank anyway! I always do predive checks and pre-breath my regs anyway and so this gives me the biggest safety margin balanced against the cost of a 5 cent o-ring and about 45 seconds of inconvenience changing it out.
In the relatively rare instance that I decide to turn my tank off, I will purge it, which is my normal routine anyway, along with a pre-dive check, and that way I achieve nearly the same safety margin.
Thanks for all the responses. This was very helpful for me and I hope it was for others as well.