It's a somewhat controversial topic, but since you seem to be implying malice in a behavior that is extremely common in my area (NYC wreck diving), let me politely present a counter argument. Our wrecks are rapidly dissolving into piles of rust on the ocean floor. So some people feel that removing and conserving artifacts allows these historic wrecks to speak to many more people than if they were to just remain buried in the sand or hidden under tons of collapsing steel.
I personally usually just take pictures, but the few pieces that I have mean a lot to me, and are displayed where non-divers can hear their story.
Casualties of warfare have different considerations, as do wrecks that are owned by someone. I'm not an expert on that type of law.