This debate goes on way too much. It's really simple to make the decision once you understand some of the basic math, but far too often I read "hp are lighter", or "lp hold more air", and while both of those statements are true, there's a little more to it than that.
If you're filling in cave country, LP tanks are the best value. Here's basically the the LP/HP equivalency if you're in an area that overfills, and then I'll try to explain it so you don't have to just take my word on it.
LP85's = 100's
LP95's = 120's
LP104/108/112's=130's
LP120's=HP149's
Once you take a basic cave/cavern class, you'll learn about "tank factors" which basically says for every 100psi, the tank has ___ cuft of air. Here's how that's calculated- (tank volume(in cuft)/rated pressure*100)
For example, a LP104 has a tank factor of 104/2640*100 = 3.9. This means for every 100psi, you get 3.9 cuft of gas.
A 133 has a tank factor of 133/3442*100=3.9. This means for every 100psi, you get 3.9 cuft of gas.....see a pattern?
So at 3400psi (34*3.9) you get basically 132 cu/ft in either tank.
Now, one argument for LP tanks is that on dive boats, often times you can't get a "full fill" on a HP tank. Let's say that you can only get a 2640psi fill. Well then, the 133's and 104's both hold 104 cuft at that pressure. Basically figure out the size tanks you want in regards to volume, and if you can't get overfills where you're diving, get HP tanks, if you can, you might can save money by getting LP tanks.
You'll hear HP tanks are lighter than LP tanks. While this is true, remember what we just learned. Saying that a LP120 is heavier than a HP120 is true, but the LP120 can hold 149cuft when filled to 3500ish, while the HP120 can only hold 120cuft. Compare the LP95's, and you'll see that the weight is nearly identical, because they're the same size tanks.
Many shops consider overfilling LP tanks dangerous despite the fact that it's been going on for nearly 2 decades without a single known incident. I've had my personal tanks filled to 3600 every single time without complaint.
At this point, if buying new, I'd buy HP tanks, the price difference just isn't enough to make it worth ALWAYS getting tanks filled in cave country....sometimes it would be nice if I could get my tanks filled in Orlando before heading to Ginnie for a midnight dive.
Also, if you know you want to cave dive, I'd start out in double tanks ASAP. No reason to make more of a "learning curve" there than there needs to be....."Keep the end in mind".
I hope I was clear, let me know if I wasn't.