The answer to your question comes out of the name of the device: It is a buoyancy COMPENSATOR. So, if it is compensating, what is it compensating for?
The answer is two-fold. One, it is compensating for the weight of the gas you are carrying at the beginning of the dive, which you are going to exhaust into the water by the end of it. With relatively small tanks, this isn't much -- about five pounds with an Al80 (which is actually, by historical standards, a rather LARGE tank). Before buoyancy compensators were developed, divers compensated for this weight swing with their breathing. Nowadays, we use the air bladder in the BC to do it, but if you realize it can be done with the lungs, you'll recognize that the amount of air required is fairly small.
A bigger consideration is the compression of exposure protection. Neoprene floats, because it insulates by having tiny gas bubbles trapped in the rubber. As you descend, these compress, and the neoprene gets less "floaty". If you were perfectly neutral in five feet of water, you will be negative in ten, and more negative in 100. By how much is determined by how thick the neoprene is. So you can see that, in cold water, you will need much more air in the BC at depth than you do in warm water, where you may be using no exposure protection at all, or a suit which is very thin.
Most dives will have either a square or a check-marked profile -- you descend fairly steadily to your deepest point, and either remain there, or work your way slowly shallower. In either case, the greatest amount of air in the BC will be at that early, deep point, where you are compensating for both gas in the tank and exposure protection compression. As the dive proceeds and you exhaust gas, and move shallower, you will slowly but steadily vent air from the BC. Proper weighting means that at the very end of the dive, in very shallow water, you will no longer have ANY air in the BC at all.
Does that help?