This site interests me as well, looks intriguing, close to home (~200km), very nice diving in area - I'll be checking it out within the next few years, just haven't had the opportunity as of yet.
In my opinion (which in all honesty isn't worth much

) I say
naturally occuring. The tectonic activity in the area is mind-blowing, and who knows exactly what circumstances are necessary in order to create the magnificent structures we see at the location in question.
Along the west coast of Taiwan there are islands known now as Penghu (previosuly, Pescadores) near which there is a "wall" that upon initial inspection looks man-made. Little vigorous debate has ever occured regarding this site ... few have dived it, and seemingly less even care ... but some early expert opinions claim volcanism/tectonic/seabed upheaval/etc. Again, another site I'd like to see for myself, for no other reason than that it is underwater
Just along our NE shore (again, about ~200km away from the Yonaguni site), we see some remarkable rock structures - flat surfaces that may appear to have been chiselled by master masons, 90 degree angles, step-like formations, wonderful stuff, really - but nobody argues these to be anything near spectacular (or signs of visitors from another planet? oooooo ...) since they are located curbside, not 30-40m below the surface.