One point hinted is how absolutely disorienting it is to not see the wall/reef/bottom. This is a good point about swimming out from the wall - you could end up in the blue with no point of reference.
If you have never experienced this, try. Swim away from a way from a wall until its all blue. And bring a compass
Actually, there's another "disorientation" that has been mentioned in passing a couple of times:
bubbles.
When I got (presumably) sucked down by a downcurrent, one of the first clues I noticed was that I was surrounded by dense, tiny bubbles, which did not seem to be behaving like all the other bubbles I had ever experienced underwater. These were not the big, wide, flat, slow-moving bubbles I'm used to seeing by divers exhaling which slowly rise straight up towards the surface. These bubbles were round, fully spherical, and tiny - like maybe the size of a BB or smaller. It just seemed strange at the time, and they were so dense around me, it was momentarily difficult to see through/past them. I was just a few feet off the wall, but the reef and the deeper blue water were no longer in view, only bubbles.
I have not discussed this experience with my spouse for a long while (probably a few years); she went through it, too. Seeing the interest and recent updates here, I brought it up with her yesterday, asking if she remembered the incident. She said oh, yes, she remembered it clearly, and the first thing she commented on were the tiny bubbles:
"It was like being in a giant champagne glass, the bubbles were very thick but so tiny, and they weren't just going straight up like other bubbles do."
In one of the linked articles they mention that in a downcurrent you may see bubbles doing unusual things, and actually sinking.
I can tell you, if you're underwater and watching bubbles, and those bubbles are not going up towards the surface, something very unusual is going on, and suddenly the world seems a bit crazy. When
"follow the bubbles, they always go up to the surface" is no longer a valid assumption, it may lead you to question lots of other things (like your sanity), if only briefly.
I was only disoriented for a couple seconds, and it was more of a general confusion/shock than feeling "lost." I knew the wall was only a few feet away, though I lost sight of it.