I am a little confused about the swimming issue. I was never a good swimmer, could not float, and was terrible at holding my breath and swimming underwater. I always loved being on or around the water though. The first time I was introduced to scuba I fell in love with it. I did pass the swim test but did not like it. In my limited diving experience I have yet to see where swimming skills both above and below the surface contribute to scuba. Scuba is an equipment dependant sport(there would be no scuba without it) and the skills required to be efficient are exactly the opposite of most swimming skills IE no use of hands, no holding breath, no fast pace etc..
For my experience level I consider myself to be a fairly good diver but a poor swimmer. I would consider a good working knowledge of math with a mechanical apptitude to understand your equipment more useful tools for diving than being a good swimmer.
Maybe someone can explain the errors of my way but until then that is my observation.
I certainly can't show you the error of your ways, but can explain why I feel that a high level of swimming and in-water skill is important. As has been pointed out before, people can't breathe water and as such are fearful of being placed in a situation where drowning is a possibility. The more comfortable you are in the water and the higher your degree of confidence, the less likely it is that you will panic when faced with an emergency.
You are correct when you say that scuba is an equipment dependent sport. We do not need to be mechanical engineers to appreciate that mechanical things fail. It's not only that they may fail, they WILL fail; it's only a matter of time. This is why computer hard drives specify a MTBF number (mean time between failure), as an example. I can certainly elaborate on the several several equipment failures that I have experienced underwater (should you be interested), but the point is when a failure occurs, how will you handle the situation?
I would suggest (although there is no guarantee) that you likely will respond to the problem in openwater more calmly if you are a good swimmer, than if you can't swim at all and your mechanical supplements fail. Panic is one of the largest enemies of any diver.