Sigh...
I read all these posts, and frankly, I see a cowboy too. But, so was I when I first started out. I wouldn't let cost persuade or dissuade you at all. I know a guy who is single, no kids, no debts that makes $15.00/hour. He moved from recreational diving to cave diving earlier this year. After seeing me in a rebreather he decided he wanted one. So he worked a good deal on a used one, worked a bunch of overtime and this year has a rebreather. That's after buying cave equipment earlier this year. But it takes dedication and singular focus. It doesn't happen overnight. And if you kill yourself, it really isn't going to happen. So don't cut corners, but do look for good deals.
The other thing is, if you're not an a-hole, people will bend over backwards to help you out in this community. Lots of people donated time, energy, classes and gear to me when I was starting out. In turn, I've done so for others. Be a good guy and this hobby will come easy. Trust me, there's so few of the good guys, people are really willing to help the non-aholes in cave diving.
And I've said this 100 times. This is not a sprint. It's a marathon. Today I have 2 rebreathers, more tanks than I can count, and a reg to go on everyone of them, drysuits, wetsuits, half a dozen masks, etc. etc. etc.... it didn't come overnight. It came after almost 2 decades of diving. Start now, take your time, make good decisions and you'll be where you want to be before you know it.
I think the first step would be in taking a good cavern course. You can do it in the equipment you own right now. It will give you some insight into whether you will even like cave diving. After-all not everyone is cool with being underwater and underground hours away from fresh air. And you simply don't know it until you try it.