Where do you dive? If its cold water, after a mask (try them on and make sure they fit!) for me would be exposure protection, either a wetsuit or drysuit. Everything else can be rented until you see what you like, but being cold will make diving more difficult. The sooner you are warm and get used to the bouyancy of your suit, the easier your diving will be.
I would guess most LDS in temperate waters would also rent appropriate exposure protection?!
If the LDS had suitably fitting and effective wet/dryuits for rent, then it is a minimal priority for purchase.
For me, the mask is No.1 I hate the idea of using a mask that has been filled a 1000 times with the phlegm of previous divers. Also, a properly chosen mask will mold over time to fit your face excellently. A leaking/fogging mask will ruin dives and can lead to rises in stress...and then to accidents.
Fins are also a very personal choice...and there is nothing worse than having your diving ruined by blistered and chafed feet. Just like shoes fins and/or booties are best once they have been 'worn in'.
Computer...these are costly to rent, but cheap to buy for basic air/nitrox recreational diving. Once you start diving with a computer, you will feel naked without it.
Regulator....a neglected rental regulator can cause you serious dangers when diving. Sell you mother and buy a decent regulator. You will never have to worry that the little scuba hovel (I mean dive center) at your next holiday destination gets its regulators serviced by an illiterate man who earns 20cents a day....
BCD / Wing...take some care and do the right research when purchasing and you will own something that will last you years and become like a second skin. Familiarity with kit is essential...and you use your bcd a lot when diving.
Wet/drysuit... if purchased diligently, it will offer better fit than any rental suit you could get.
Other items...not really in the list, but cheap...not often available to rent....and every diver should have them....
a. small DSMB
b. small reel (finger reel is ok)
c. whistle
d. knife (small, sharp and titanium)
Don't waste your money on other useless gadgets...the diving industry is full of them...and you look foolish as a walking christmas tree. Beware that many 'top-of-the-line' items just contain a myriad of pointless gimmicks (who on this planet would ever need a bcd with 12 d-rings?!?!?).
The 'best' stuff is determined by simplicity, effectiveness of operation and robust design.