SlugLife
Contributor
My advice is start with one thing or component at a time. Don't try to master the entire list of fins/regulators/tanks/bcds/etc. It's also okay to start with relatively inexpensive gear, with the idea that you'll replace it later.Right now it is drinking from the firehose trying to digest the amazing amount of accumulated knowledge on the board!
That said, if you have a hint of DIY in your blood, a good place to start is with a single-tank backplate+wing BCD setup (abbreviated "BPW"). The basic parts are (a) a backplate (b) the wing/BCD/bladder (c) the harness. You can get entire packages, dive gear express has a few good and more affordable ones. The harness itself you can even go for a "once piece" setup, which is basically some webbing and then add a buckle and d-rings as needed.
edit: There's usually a health used-fins market as well, and fins aren't that complex. Finding the "perfect" or "right" fins for your needs might be complex, but initially you'll mostly just need a pair of fins that aren't terrible. I'm always seeing lots of used fins on CraigsList. Ideally, "open heel" fins intended for SCUBA. Mares Avanti Quattro are a good "all around" fin that I often see used, but there are dozens of other good fins to choose from as well.