Well, read the post, read the responses, saw some guy in what looked like cheetah panties in a "minimalist" rig. Wanted to barf a bit after that last part.
I don't think what you're looking to do is silly, crazy, or suicidal. I do find it laughable that you describe San Diego water as warm (listen up all you FL divers), but that aside, I spent plenty of hours as a kid in SoCal waters body surfing and what not without a wetsuit.
To your question, at that depth with a sandy bottom that you'll encounter, you could push off the bottom to get to the surface, no problem. While emotionally scarred for life by the guy in the cheetah panties, I would recommend what he had, a stainless steel backplate and maybe a stainless tank or weight belt to properly weight you. A thin me needs about 10lbs to sink properly, but I'm a tall person. A SS backplate will give you 5 - 6 lbs over your lungs. Just use continuous nylon webbing to strap it to you. If you dive an AL80 then maybe a weight belt with a few lbs that you can ditch at the surface if you get in trouble.
I'm not an old fart, never dived a horse collar, and I don't think what you're contemplating is particularly dangerous if you're a competent swimmer and diver.
Beware the SucbaBoard nanny state. It's a real thing.