Some interesting and thoughtful opinions here, good stuff.
I have to agree with the sentiments expressed that it will really depend on 1) what kind of diving will you be doing, where, and 2) what would make you more comfortable. Fie on the boilerplate recommendations of your LDS or buddy!
The first things I bought - after mask, snorkel, fins & boots & gloves, which we're required to own before certification - were a 7mm jumpsuit, 5mm jumpsuit and a wrist computer (Oceanic Veo 180, still use it, great & pretty cheap). I had access to rental gear otherwise; I wasn't terribly put-out by the mechanisms of different BCs, and as long as the reg provided air I didn't really mind that either. I'm not a claustrophic person, and I adapt pretty quickly to new environments and equipment.
For someone who is not naturally comfortable underwater, and needs constant reassurance of the workings of their air and buoyancy, I'd say buy a reg & BC first.
For someone like the above who lives in and plans to dive a colder climate, buy a very well-fitting exposure suit first - you should be reasonably comfy and relaxed before you enter the water. (Sometimes this means getting fitted for a drysuit right away, which does really help some people continue diving after certification.)
For someone planning on traveling, and doing a lot of repetitive diving (which is what I did straight out of the gate), I have to say a computer is indeed essential - diving by table on a liveaboard (or even consecutive daily boats) will guarantee you will be back on the boat way before anyone else, and you'll miss quite a lot; you need that constant credit for depth that computers provide.
Yes, you can likely rent them at your destination, and some places even require that you have one whether rented or owned - but do you know how to read it? Do you really want to spend the whole dive trying to decipher what the thing is telling you? Is it telling you to ascend or stop?? (It took quite a bit of getting used to my D9 after years on my Veo 180.)
The nice thing right now is that some manufacturers are doing package incentives, so you can get everything at once at a discount (- which is great for new divers since they may not know what features they may want down the road, offered by other mfrs, which may otherwise result in analysis-paralysis.)
If asked the question by a new student, I'd have to say "buy the thing(s) that affected you the most on your first dives; what would make you more comfortable, what would make you less preoccupied and more into the fun of it?"
