Things no to skimp on:
Regulator. Make sure you get something that breathes comfortably and has decent internals. There are obviously some less expensive ones out there and those are probably okay if you don't plan on diving a whole lot. But remember the old addage you get what you pay for. Do some research and ask around to see what's good. You don't necessarily need to spend excessive amounts of money to get good gear. I think we paid about $400-$450 for my wife's regulator (Atomic Z2) and it's nice and breathes easily. But in doing the rescue scuba class I'm working on, I and a couple of my classmates discovered that when trying to hold the 2nd stage in place for a diver that's being rescued, if you hold your hand over the front of the reg too tightly, the valve can't open all the way and it makes the reg hard to breathe. I personally dive a Hog D3 first stage that I think I paid about $250 for and so far I have no complaints at all. And where the hoses attach it swivels making it much more comfortable to dive and look around, etc.
Fins. Blades or splits, it's really what's most comfortable for you to kick. But be aware that when trying to kick against current, splits leave much to be desired. A good, stiff blade fin will be a little harder to kick but you'll get more out of them especially in current. Again ask around and see what people are using and why. Maybe your local dive shop will let you try a few pairs in the pool. My choice here are the Hollis F1 "Batwing" blades that I think run about $170. They're very heavy, but in current they're hard to beat.
Dive computer. And this is just my personal opinion, but get something that's air integrated. Whether it be a wrist-mount or console, get something that will tell you how much air you have left in your tank because it will also tell you about how much bottom time you have. And it eliminates the need for an SPG. I run one of the more expensive consoles (Atomic Cobalt II) but I previously had an Oceanic Pro Plus 3 that I really liked a lot, too. It now belongs to my wife because the Atomic was originally for her and she didn't like that the characters were so small. Plus she said it was too busy and there was too much information on one screen.
Obviously you don't have to buy everything at once. But don't buy something just to have it. Do some research, figure out what's good, and spend wisely. Get something you won't want to replace in 6 months/year. And some advice on a BCD...try a jacket style AND a back inflate and see which one you're more comfortable with. A jacket style will be a little more stable on the surface but can tend to roll side to side underwater; whereas, a back inflate will make you want to pitch forward on the surface, but keeps you nice and horizontal underwater. It's a personal preference thing.
Hope this helps a little.