Unless you are learning at a resort, fins, mask and snorkel (and sometimes weight belt) are usually required for the course so these will be the first you purchase. Go cheep with the snorkel. The mask needs to fit properly. Before you buy it, look for reviews of your particular mask to see how well it doesn't fog or get the advice of other local divers If you can put off buying fins until you have tried some you are probably better off.
After you have completed the course, you will want to start looking at other equipment. I would put the BCD at the top of the list of things to buy. Knowing exactly where everything is and having a really good fitting BCD will increase your safety and enjoyment of the sport tremendously. You will get a feeling for what you are looking for in a BCD during your open water dives. I know that people say to buy your regulators early, but for the most part, a rental is as good as your own. If you trust a dive shop to service your reg, you can trust it to rent you one that works properly. After the BCD I would look into a wetsuit. The thicker the wetsuit you require, the harder it is to get an exact fit, and the harder it is to get a suitable rental. You can often get used ones at decent prices. Just wash it in cold with woolite and the previous owner smells will come right out.
A computer is also a very nice thing to have but evaluate this purchase based on the knowledge you gain from the OW course you take, and the dives you plan on diving.
The decisson to purchase a regulator will be based on two things: How much money you spend on rentals, and if you want to customize it to streamline your scuba unit. I don't have a full console, and with my airtrim BC, I tie my SPG to my low pressure inflator hose (using a rubber strip and a tie wrap) in order to keep it in place. I couldn't do this with a rental rig. Also, as you do more advanced diving, you will want to purchase a regulator that you know is suited to your dives.
A tank is a tank is a tank (unless you want to switch to steal or higher capacity), and the decission to purchase one will be primarily based on how much money you spend in rentals while diving locally. You can't take your tank on an airplane so I would rent for a while before making this decission.
I guess what I am saying is don't rush in to buying everything all at once. Figure out what suits you and buy it a piece at a time. Oh...and don't buy a wetsuit online. I learned that lesson the hard way.
And one last thing..when you get sick of not knowing where you are going, you will buy a compass.