I think that if you look for the most consistent motif in this thread, it is uncertainty.
In the most well known cases of oxygen toxicity, there is no question about it, as in breathing 100% O2 at 100 feet. In other cases, though, if you read carefully, O2 toxicity is offered as a possible explanation for the fatality, with nothing else presenting as a more likely cause. In some cases, O2 toxicity is not even the most likely cause but is considered a possibility.
I remember reading the full DAN article that cautioned about taking drugs like Sudafed before diving on Nitrox. Although the article does finally warn against it, the majority of the article is devoted to explaining why there really isn't enough clear evidence to support that warning--they are just acting in an abundance of caution. The few cases upon which the warning was based were not clearly O2 toxicity, and it is impossible to tell that if O2 toxicity were involved, it had anything to do with the Sudafed. The warning is actually based on vague symptoms in mice given a drug similar to Sudafed.
So the bottom line is we really don't know what constitutes a safe level for any individual diver on any individual dive, so most of use choose to err on the side of caution. If you make a mistake in the other direction, you probably won't be able to apply that lesson to future diving.