Crush
Contributor
If you have trouble swimming to 10 ft when 4-6 lbs buoyant you might want to look into some good quality free diving training.
In the bad old days (pre-BC) divers would weight themselves to be neutral at whatever their operating depth was. You would have to kick like heck to get down the first twenty to thirty feet. With a few, very rare exceptions, the end of the dive meant a buoyant ascent (at rates which would make your computer freak out) with no safety stop. I am not advocating that we return to those days, but swimming down while under-weighted is very doable.