Fins and mask would be important, given that these are generally considered personal items, and also having the most to do with comfort in the water (thus reducing stress). Also, these items are generally cheaper than most other equipment, thus making it easier for divers start with them.
With buying fins early on, you also get the advantage of being able to start practising propulsion skills such as frog kick, helicopter turns, back kicks etc.
As for a snorkel, I haven't really found a big difference between different types of snorkels. However, seeing as how they are potentially the cheapest equipment that you can start with, might as well purchase one of them. I have one of those foldable snorkels that I tuck into a pocket till I need it.
If you are in an area that requires exposure protection when diving, a personal wetsuit or drysuit would also be recommended as one of the earlier purchases. This would be due to the fact that exposure protection are quite personal items, and also that a good fit would make a beginner more comfortable in the water.
Moving on, I would advise you to go for a BCD, as that would enable you to improve your buoyancy skills. Personally, I feel that in my period of renting BCs, I was never able to fine tune my buoyancy skills as the properties and characteristics of rental BCs were always slightly different. After purchasing my own BC (a BP/W), I observed that I could retain most of my improvements in buoyancy and trim after a training session.
Regulators I would recommend leaving till the last, as they are quite cost prohibitive, and can be a nuisance to take care of, especially if you dont dive too frequently. Although some opine that having the peace of mind that your life support equipment is guaranteed to not have been abused does help with increasing comfort, I have personally found that rental regs are almost always adequate for the mission at hand. Therefore, I would recommend leaving regs till the last.
Of course, there are a lot of accessories that you can purchase whenever you have some extra money. For example
Compasses - useful for navigation
Torches - Useful for night dives, low visibility dives, or for signalling
Gloves - Useful for protecting your hands and keeping warm, albeit sacrificing dexterity
Slates/Wetnotes - Useful for noting objectives, lessons to be learnt, coordinates etc
Lastly, the sequence above does not have to be strictly that way, just purchase whatever happens to have a good deal and is affordable at the time, and sooner than you know, you would already have a full collection of scuba gear.
