Specifically hostile behavior towards maritime life (annoying fish, riding turtles) or damaging coral reefs ( by bad diving skills or actively breaking/poaching).
The type and severity of behavior would be a determining factor for me, along with who is performing that behavior. Not everyone will agree what is or isn't bad behavior as it relates to marine life.
Within my first 13 dives over 3 years - all in St. Croix with Scuba Shack/N2theBlue - the DM's I had were helpful and stressed good etiquette. I remember being told that the black coral on the Rosa Marie wreck had great value and that you could be in big trouble if you tried to take some. I received a great start to my diving.
I was first confronted with what I thought was bad behavior on dives in Aruba (still didn't have 20 dives to my name) by a diver who was an expert in anything we talked about during surface intervals. For 2 of my 4 days of diving, he took his camera with the long extendable arms and crowded in as close as he could get to his subject with no regard as to the coral he was hitting with the arms. Although I was concerned about the marine life, I was more irritated that he was getting in the way. Because of the dive plan used by the dive op (Dive Aruba), the DM wasn't in the water for the last ~25 minutes of our dives, so he didn't see a lot of this. No-one said anything to the diver, but my buddy and I were glad to have him off the boat.
Have to say that I have seen divers and DM's alike pick things up off the sea floor and put them back, but not purposely break anything. Have seen plenty of divers accidentally kicking coral or letting gear dangle and hit it. My thought is a good DM or dive buddy will catch this and give you a signal to ascend and discuss it topside.
Handling sea life (not of the coral variety) to me isn't as big of a deal. Harassing it would be. To me handling it is not the same as harassing it. I look at it as an educational experience. I don't go out of my way for the experience, but if it presents itself, then great. The first sea life I saw handled was by a DM from Ocean Frontiers in Grand Cayman - a sea cucumber that he allowed my wife and me to feel. Since that time I have either had a guide, DM, or allowed myself to feel some sea critters - dolphin, stingray, balloon fish, starfish, sea cucumber, and reef shark and a great white - just kidding if you're still reading. Have watched DM dig octopi out and put on my dive buddy's arm and watched huge grouper allow divers to touch it. Will admit that the octopi were in the harassment category as they want to stay hidden, but it was fascinating to watch....once.
As to confrontation, when I was younger, I wouldn't have said anything.
Getting older, if I happen to see it or if a diver aware of the problem asked, I might make a suggestion or bring up the situation in a discussion on the boat. I find most people to be receptive to learning. The few know-it-alls are going to do what they want to do. But my main goals on a dive are to enjoy myself and keep an eye on my buddy - not watch everyone else and worry about how I want to confront someone. I'm 6'4" and, unfortunately, 15 lbs more than my normal 225, but if I see a guy in Walmart bigger than me, who took off his mask after getting into the store, I'm not confronting him about why he took his mask off and I'm not worrying about it! You used the term "hostile behavior" in your post - I don't need to see if a person has hostile behavior topside.