Wait, do not fully inflate my BC at the surface? Why not? My figuring was more buoyancy=less of me in the water=less drag. Does inflating it all the way increase the drag?
I think you mentioned swimming on the back while on the surface. If you fully inflate your BC, especially a wing, it will likely push your torso up out of the water and rotate your legs downward. Additionally a jacket style BC may also squeeze the bejeebers out of you making breathing difficult. Do what you prefer, maybe something you can work on in the pool trying different methods, see what works best for you. As to drag, that is a hopeless pursuit. The best way to reduce drag is to reduce equipment encumbrances and trim as horizontal to your intended direction as possible/comfortable.
I do often surface swim on my back with a wing and I do not fully inflate. I get enough air in that I am buoyant but with an 18 or 22 pounds of lift wings, it is not much.
It is not recommended loitering on the surface to remove your regulator. And for a long surface swims, most use a snorkel and swim face down. That may or may not be advisable in some locations.