You should head over to the regulator forum and you'll find lots of info about this. You can rinse and soak your regulator off the tank as long as the dust cap is securely in place. Leaving it on the tank is also fine for rinsing. However, most regulators really should be soaked, not just rinsed, in clean fresh water after use in salt water. If you have a pony tank it's easy to soak the reg while pressurized, but if not just use the dust cap. Don't press the purge while soaking, otherwise water could start to make it up the hose into the first stage from the 2nd. Some 2nd stages have a seat saver, which could theoretically do the same thing, but as long as the 2nd stage is a soaked a bit deeper than the 1st, there's usually no problem. When you're done soaking, it's not a bad idea to put the reg on a tank and blow a little air through it, just to make sure there's no water where it shouldn't be.
When you disconnect your reg from the tank, it's common to see people blowing out the water from the dust cap with the tank valve. Unfortunately, lots of times I'll see people actually blowing tank air right at the 1st stage inlet, which just drives any water drops present right into the filter. The best thing to do is to splash a little fresh water on the reg where it meets the valve while it's still connected (a friend carries a squeeze bottle on the boat just for this purpose) then wipe it dry, remove the reg, then you can use either your own breath or a little tank air to dry out the inside of the dust cap.
