I prefer horizontal mostly because most of my diving involves looking at things on the bottom, and being horizontal gets my eyes about 5' closer to what I am looking at. I also dive in a lot of silty environments, so being horizontal keeps my kicks away from the silt. It also puts me in a better position to move quickly to help out a buddy, or to move quickly to a buddy to get help.
On a drift dive along a wall, it doesn't matter as much since you can get your eyes close to the wall equally as well in a horizontal or vertical position, however, if you get too close, it is much easier to back up in a horizontal position than in a vertical position (providing you can do a back kick).
On an ascent, I think the difference matters even less, but I use this time to practice my horizontal trim to make the rest of my diving more enjoyable and of course the buddy aid thing still applies. If doing a safety stop in a current, I think a horizontal trim would make you more streamlined and better able to maintain position.
Basically, I can think of a lot of good reasons to be in a horizontal position, but the only one I can think of for vertical is that it tends to be a more natural and comfortable position. So, I am working on trying to get a horizontal position to be more natural and comfortable for me. Currently, I have to focus on staying horizontal and in trim, and consciously correct myself if my head or feet start to drop, but hoping to get to the point where it is all automatic.
Even though I prefer the horizontal position, I would never call someone names for diving vertical. I may point out a silt cloud caused by a stray kick, but I would do the same to someone in a horizontal position. If you dive vertical, don't stir up the bottom, and don't kick the reefs, then I would be happy to dive with you.