As others have stated, the primary reasons for this are ...
1. using drysuit for buoyancy control instead of BCD
2. being overweighted
3. wearing a drysuit that fits poorly
There are simple fixes for 1 and 2. But if the suit doesn't fit right, and you own it, the fix may end up costing you some $$.
For the first, as you've already noticed, air tends to shift around in your drysuit more than it does in your BCD. That can be remedied by using your BCD for buoyancy control, and only leaving enough air in your suit to prevent squeeze. Many agencies teach using the drysuit for buoyancy control because ... in theory ... it simplifies the number of air spaces you have to control as you change depth. In practice, using your BCD for bouyancy control isn't any more difficult than using your drysuit. The trick is to leave your dump valve (assuming it's in the standard left-shoulder configuration) almost completely open. Then, when you raise your BCD inflator hose to vent, the drysuit will dump excess air automatically.
For the second, perform a weight check. There are several ways to do this. My preferred method is to do it at the end of a dive, with about 500 psi in your cylinder. At a depth of about 10 feet, establish neutral bouyancy and hover without moving while breathing normally. If you are properly weighted, you will have no air in your BCD, and only enough in your suit to prevent squeeze. Now breathe out completely. You should sink. Breathe in completely, you should rise. If you let all the air out of your BCD and you sink while breathing normally, you are overweighted.
Finally, drysuit fit. A shell suit has no stretch, so by design there will be some extra material in the suit when you are standing ... to allow a full range of motion. Put on the undergarment and suit. Do a deep-knee bend. You should not be restricted to a full squatting position ... and you should have no excess material in the legs when you achieve this position. Excess material in the legs will trap air when you are diving, and this air will create buoyancy problems. Likewise in the torso, while standing see how far behind your head you can move your arms. You should be able to move them as far with the suit on as you can without. Look in a mirror at how the suit fits under your arms and in your upper torso. As you reach back, the material should come pretty close to tight. If at a full extension you have any bagging, the suit will trap air. If the suit restricts your range of motion, it's too small. Now check the boots. There should be adequate room for two pairs of socks (I prefer fleece, for warmth), but no more. Boots that are too big will cause floaty feet. Boots that are too small, or too tight, will cause cold feet.
If your suit doesn't fit properly, it can usually be modified. But it can be spendy ...
... Bob (Grateful Diver)