Kat,
There are more than one way to skin a cat. For ascending, you can either inflate your BC and float up or swim up. Either way, as you're approaching the surface, you will have to start deflate/venting your BC because the air in your BC is now rapidly expanding.
When you ascend by inflating the BC, your ascension becomes harder to control as you rise and the air in your BC expands very rapidly and next thing you know, you'll rocket to the surface. Of course, when you rocket to the surface, you'd risk all kinds of issues with the bends or lung embolism. Speaking of embolism, that's the main reason why you shouldn't be holding your breath on the way up. The air in your lung will expand and if not exhaled then it will rupture your lung. There's nothing wrong with aiding your ascension by breathing in deeply and fill your lungs. Just make sure that you breath out. And during your dive, you shouldn't be holding air and using your lung like a BC because that will lead to CO2 buildup (blinding headache among other things). You don't have to breath in and out like a runaway train either. Just breath in and breath out normally in controlled rhythm.
A lot of divers unfortunately never get their dive weights done correctly either because they don't own their gears and have to rent (it's tough to figure out your dive weights when you use different gears all the time), or they don't care enough to bother with it. But once you've properly weighted yourself, diving becomes so much easier and effortless. This is very evident at ascension and doing safety stop.
I pretty much got my dive weight adjusted down to a gnat's ass (pardon the French) and when I need to ascend, I simply swim up slowly and I will rise slowly with each fin stroke. As I rise, I stick my corrugated hose out horizontally with the same plane as my shoulder and hit the vent button. In this manner, at whichever depth that I'm at, I automatically have enough air in my BC to maintain buoyancy. By the time I reach 20-ft or so, my BC is empty of air and I maintain neutral buoyancy by breathing in and out (not holding my breath and use my lungs as BC - VERY IMPORTANT!!!) and do my safety stop effortlessly.
If you're too lightly weighted, then you will pop like a cork to the surface at this point. If you're too heavily weighted then you will have to fin hard to stay in place or use your inflator button and risk rocketing to the surface. So if you see a person going up and down, up and down, up and down, struggling at the safety stop, that generally tells you that the person is too heavily weighted. Most divers at this point would just grab onto the anchor line or kelps in order to maintain their safety stops. The only time I would need to hold onto a kelp or an anchor line is if there's a current and I don't want to have drifted a couple of hundred yards away from the boat by the time I finish my safety stop.
So, my suggestion is during ascension, instead of holding the inflator above your head and vent, keep it outward at shoulder level and vent.
