The optimal choice of how you distribute your weights will depend very much on your other gear choices, as well as your specific physiology ... the idea is to use your weights to "balance" your body in the water when you are diving ... what you see referred to on the board as "trim". You want to distribute your weights in such a way that when you stop all motion, you will neither tilt head down nor feet down ... basically, you should be able to stop all motion and remain where you stopped without your weights wanting to pull you one way or the other.
For some people, integrated weight systems work out very well. For others, a weight belt is optimal. And for still others, distributing your weights between BCD and belt turns out to work best.
The best way to find out is to see how you trim out with the weights as you are using them, then move them around on subsequent dives (a little at a time) until you feel balanced. Generally speaking, integrated weight systems will move your weights higher up on your hips than a belt, so if you find yourself getting "feet heavy" you want to move more weight to the BCD ... and if you find yourself going head down you want to move more weight to the belt.
There are several weighting "solutions" available ... generally speaking, ankle weights are not an optimal solution for trim issues. They work well enough for people who tend to get air in their drysuit boots ... or for people who have fins that are positively buoyant. But I view them as more of a last resort than a first one.
(edited to add) ... Weighting considerations should be done while diving, not surface swimming. As someone has pointed out, the surface swimming issue is more one of technique than weight distribution.
... Bob (Grateful Diver)