Here's what I did and why; take it for what it's worth.
I bought personal gear (mask, fins, snorkel, booties) right before my OW class because it was required, and it wasn't a part of the gear rental package. I also bought a whistle and mirror on a lanyard, since we were supposed to have an audible and visual signaling device, and these were small and cheap. In retrospect, they didn't check to see if we had them, the whistle was kinda garbage, and the mirror is not a good alternative to a safety sausage. I now have a Storm Whistle and a DSMB and reel instead.
A couple weeks after completing OW, I decided to buy a computer. I wanted my own so I'd be familiar with it, so I could download my dive profiles, and so I could have a wrist instead of console mount. Also, I was hesitant to buy a lot of bulky, expensive dive gear when I wasn't so sure what kind of diver I was going to be; I started out thinking I'd only dive occasionally on vacation and would just rent everything when I got there. But even then, a basic computer was only $200 and very small, so it still made sense to own vs. rent. I bought the Cressi Leonardo, which is OK. I ordered it online; I think if I'd had the chance to try on a couple models in the store I would have picked something smaller. I cut the excess length off the strap, but I think a bungee mount would've been even better. The ascent alarm is also annoyingly sensitive, but other than that it's working out fine. I still think buying the computer next was the right order of operations for me. I did AOW and Nitrox shortly thereafter, and it was nice being familiar with the face so I could easily keep an eye on my depth and remaining NDL.
My next purchases were a wetsuit and a compass several months later, after lots of rentals. Some friends of mine bought their wetsuits right away because they didn't want to wear one someone else had peed in, but it didn't really bother me. I wanted to try lots of brands before committing. Ultimately the suit I bought was a make and model I'd never used, but I knew enough by then about how a properly-fitting wetsuit felt putting it on dry, and about how much stretch the average suit had, to know this was a good one. I also learned from my computer experience and bought a compass with a bungee wrist mount.
I bought a backplate and wing after trying it out with a couple rentals. Not many shops rent them, but my LDS does, and I really liked how it felt. I did some research but ended up just buying the Halcyon, the same model I'd been renting, because I liked it, and while many said it wasn't worth the extra money, nobody who had one seemed to complain about it.
I bought the reg next, when I took GUE Fundamentals Part I. Again, I did some research, but ended up kinda just going with what was handy. I bought it from my instructor, who offered to order my choice but had some Atomic Z2s on hand. I'd had that on my short list anyway; the ones he had were in colors that matched the rest of my gear, and he set it up with the long hose and bolt snap on the primary and bungeed necklace for the second. I'm not 100% decided whether I'm going to take Part II and continue with GUE, but I'm glad I had a chance to learn about my options because I really like this equipment setup.
I thought about just renting tanks and weights forever, but decided I wanted the flexibility of always having everything ready to go diving anytime. (Clearly my original idea of just being a casual vacation diver did not pan out; I've been diving locally at least a couple times a month since I got certified.) I also really like my 100cf tank; I've got my weight dialed in and I always have plenty of air.
Overall I'm really happy with my choices of what I bought and when. Feel free to ignore anything that doesn't resonate.