+1! on your decision to get neoprene that fits properly as your first purchase. Absolutely right.
You can rent virtually everything else, but the key is finding a shop that rents gear that is well maintained. Too many shops just rent out regulator sets that breathe very stiffly, because that requires less frequent service than regs that are tuned light.
I wouldn't make a computer a purchase before regs, if only because many shops won't let you use your computer instead of the one they want to rent you with the regulators. It's too much trouble for them to take two minutes to switch hoses. You could do it yourself, but you'd need to be taught. And as you gain experience, you may regret your computer choice. You probably won't be pushing the envelope just yet, so any computer will likely serve you well.
Once again, find a shop that treats you right, in exchange for maybe buying through them later, instead of online.
Renting bcd's is a great way to find one that truly fits you properly, before you plunk down $200-600. Backplate and wing is very popular on Scubaboard. I dive that way, too. But there are good back-inflate bcd's as well for the occasional diver. Setting up a wing is an exercise in frustration for a new diver. And if you like a jacket, that's fine! They make you feel very safe at the surface. It's not like it'll be the only bcd you ever buy. That's why we have eBay...to sell stuff!
My recommendation? Aqualung SolaFx 8mm/7mm soft neoprene chest zipper for NorCal waters, or a Henderson. Hard to get into, but very warm. Lots of sizes.
Then find out what brand of regs are serviced locally. Being able to get good local service is important. Once you have a locally serviced brand, see if you can rent the various models. In any case, get lots of advice before you buy.
Neoprene, regs, computer, bcd.
My 2¢.
Cheers! Safe diving!