Welcome to a site that can provide a wealth of information regarding mans greatest activity (man meaning human, not male, of course). Well, maybe it's a tie being the greatest.
Nevertheless, all good points above. I certainly would suggest the essentials first - mask, fins, boots, hood and gloves(some dive shops around here dont rent those) , etc.. But after that it's dealers choice.
I was faced with the same dilemma. I extensively researched every possible piece of equipment that I might want to buy. It was sensory overload picturing myself in all that wonderful gear. Unfortunately, a little thing called bank overdraft would have become my new worst friend if I had my way so I needed a system of purchasing.
More expensive before cheaper items? Life support before extras? What it all came down to was- what was most exciting to me at the time! One of my first purchases was of a Halcyon SS backplate and Pioneer wings as my BC. Many would argue that they would buy a reg first, or a wetsuit, or whatever, but for me I had read volumes of posts on this site as to the benefits of such a system. I became completely engrossed with this item and researched it and decided that this piece of equipment should fit in to my present and future style of diving. I say future because I have a long term plan set up on what type of diving I would like to progress to someday.
Think to the future, might someday you like to dive doubles? Deep wreck dive? Drysuit? If the answer is even a maybe, than try to start your purchases with equipment that you can use to get to those goals and use once you are there. As Rax stated so correctly, there will always be more to buy. My philosophy is to buy whatever you are most interested in right now, buy quality gear that will last, and buy gear that you will be able to use if your diving style progresses to that "next level" so many of us are trying to achieve!