I have seen that happen exactly there.You just described my experience at the Blue Hole in Santa Rosa, NM. I was one of the first divers in that morning, hanging out and enjoying the crystalline water, when a n00b came hurtling past me, bottom-bound, and smacked the dirt. Aaaand there went the viz for the entire day...