The recreational limit for scuba diving is 130ft. But oxygen doesn't pose a substantive toxicity issue until 218ft. BUT the record for compressed-air scuba diving is 509ft! So WTF?!?!
The recommendations are based on perceived risk.
Take the 18m (60ft) recommendation for new divers that some agencies use. This makes perfect sense - the theoretical models that Lynne refer to start to change at around that depth, so that the NDLs become much shorter. At 18m, the typical breathing rates of new divers are such that they are unlikely to be able to spend sufficient time at those depths to put them close to the NDL... something that is true for deeper dives. Variation in gas consumption is linear, reduction in NDLs is exponential.
So it is just a tool for managing risk. It doesn't mean that it's a law. My first post-OW dive was to 27m... there's nothing stopping that, though in retrospect it was perhaps somewhat foolish.
Think of it as a speed limit - you can go faster, but if you do there are implications... in terms of potential for an accident, fuel consumption and so on.
Similarly the 30m (100ft) recommendation makes sense - this is the depth where nitrogen narcosis starts to become apparent for the majority of divers. So again, another guideline is a appropriate.
The 40m (130ft) recommendation is, to my mind, the deepest you can make a sensible dive on standard(ish) scuba gear - taking into account the effects of narcosis, short NDLs and gas consumption. It's the point where really, any miscalculation in how long you spend at that depth is going to have serious implication. As soon as you have a decompression obligation and you haven't calculated how much gas you need for the ascent - then you are playing a lottery, but one you will never win... just someday you will lose.
Yes, the record for breathing compressed air at depth is way beyond this limits. I suspect that it wasn't done on "standard" recreational scuba gear. Yes, it is way beyond the point at which air becomes toxic - but what was the bottom time? Hyperbaric chambers regularly run patient schedules with a ppO2 of 2 or even 3ATA. Those patients tox, but it's not straight away. It takes time - and that time will vary from individual to individual.