Take this with a grain of salt: I just recently nitrox-certified (never had reason to before) and done exactly one dive on nitrox. (Though have a tank full in my garage awaiting the next opportunity...)
Maybe BECAUSE I just went through the course (and was shocked at the 1.4 and 1.6 rule, which was far lower than I had learned in my 1985 OW class), I think there's a bit of confusion. There's CNS toxicity that leads to the 1.4/1.6 rules, and there's the "oxygen clock" type which is pulmonary toxicity. The former causes convulsions underwater and then you die of drowning unless you don't. The latter (if I understand it correctly) "burns" your lungs over time. (My interpretation of "burns" is based on a guess that it's the altered redox potential?) When I learned to dive way back when, I don't think CNS toxicity was fully appreciated, hence less conservative limits.
The drag with CNS toxicity is that response is likely to vary a lot person to person. Are you prone to it or not? Who knows! (Leaving aside the exercise/CO2/drug interaction issues that raise the probability.) If you're not prone to it, you might be fine. If you are, well, not so much.
Pulmonary toxicity strikes me as being related to laws of physics such that you just can't avoid by being genetically lucky. Maybe a bit of variation around the edges from person to person, but not the "all or none" of having a seizure or not at a particular partial pressure.