Do yourself a favor... right now. Take a look at nearby clock with a second hand and watch 30 seconds go by. That's a "30ft/min" rate, and it's a relatively l-o-n-g time. Certainly longer than it would take to "swim up to the ladder" from 15ft.
I'm guessing you're going too fast, and don't even realize it.
(We can save the can-of-worms discussion about doing a 60ft "deep stop" on a 100' recreational dive within NDLs for another time.
Haha, yes, lets not rehash discussions on this thread that come around at least once a quarter. That said, relax. It takes 30secs to travel the 15' at a constant rate. At ~6 sec a breath (just using normal above water breathing times), that's 5 breaths. I'd guess that I take a whole lot more than that just trying to judge the ladder's up and down motion so I don't get bonked in the head when I surface. I'll bet it takes me 2 mins to move up, position and go for it.
Really, though, I don't share your extra attention with surfacing from 15' vs the rest of the dive.