My concern would be oil flashing over inside the compressor if it is not specifically designed to make breathing air. I'd worry that the filtration stacks might not eliminate the poisonous byproducts that can be created that way, or could saturate and allow them through. Like everything else you breathe, their effect increases with their partial pressures. If the thing runs cool enough, inside the compressor itself where you cannot measure it directly, and you are using a compressor oil made for breathing air compressors, *maybe* it's fine. I have no clue how to turn "maybe it's OK" into "it's all good."
Have you ever had the air tested for that compressor? The results would be academically interesting. However, even if it were performed under worst-case conditions (hottest ambient temperature, highest pressure, longest continuous run time, oldest filters) I'm not sure I'd declare victory if the results were good. "Just because you're paranoid doesn't mean your compressor isn't out to kill you."
That all said, life is risk (shrug). Pick the ones you want to take.