Good point. I have been know to get distracted from ladder to seat and lose track of it. Depending on the circumstances I keep it in my mouth until ready to clip off. If needed you can clip and temporarily switch to the bungee backup. I've done that too.
I have been thinking about doing something like that. I.e. when I hit the surface, switch to backup and clip off primary. When I climb the ladder and get back on the boat, I can spit out the backup as I make my way to my bench.
Good idea!