Most seizures last two to three minutes and resolve, but there is a period of reduced awareness and confusion afterward which varies tremendously in time. If the stimulus causing the seizure is not removed, you can have repetitive or continuous seizures, known as status epilepticus. A person who is actively seizing probably has a closed glottis, and should not be moved upward in the water column until the active convulsion has ended. Subsequent to that, it would be desirable to move them upward as quickly as is safe, to reduce the ppO2.
Few people have survived oxygen toxicity seizures underwater. It requires the immediate presence of a buddy who recognizes the problem and takes precisely the right actions immediately.
That said, oxygen toxicity seizures are extremely unlikely if O2 concentrations are kept below 1.4 ATA. It's easy to do the vast majority of dives that recreational divers will do on 32% and respect that limit. One might lose out on a few, deeper dives. Use of a standard mix in this way also permits keeping one's computer (or one's mental computations) set the same way all the time. Cumulative OTUs are not much of a concern unless one is doing multiple deep or long dives on multiple days.
I do all my diving on 32% if I can (which I can at home). But I pay a blanket fee for a year's NITROX which is an extraordinary deal, so it makes it very reasonable financially for me. I do dive 32% on trips if I can, although I have shaken my head at myself for paying the extra for NITROX to do 15 foot deep cave dives in Mexico
So, to answer the OP, there are disadvantages to diving NITROX all the time if a) you want to do particularly deep dives; b)
you are diving multiple times over multiple days, or c) if it substantially increases your cost for shallow dives where the increased bottom time is irrelevant.