This explains why it's important to be horizontally trimmed:
Horizontal diver trim is the foundation for good buoyancy control and efficient propulsion. It also helps conserve the marine environment.
scubatechphilippines.com
To achieve this you need your center of mass to be directly below your center of buoyancy when horizontal. People aren't naturally like this. Our legs are mostly muscle and bone which are negatively buoyant and our chest includes our lungs which is a big space with a bunch of air in it. That's why people's legs hang down when they relax in the water.
If all your weights are around your waist it will exacerbate this problem. Moving weight towards your head alleviates it. A steel backplate is an easy start to moving some weight forward. If it's not enough, trim weights on the upper band takes advantage of the additional lever arm from being a bit further forward.
A couple of years ago somebody came up with a good set of diagrams to illustrate all this. I tried to find them without success. Maybe someone can supply the link?