I think the issue requires a large degree of balance as there is no clear right or wrong.
As noted above, we tend to play fast and loose with the war grave argument when applied to battlefields in general, as they are not immune to being turned into money in the form of strip malls and housing developments for up scale types who want a great view.
The point noted above that shipwrecks and sunken warships in shipping channels have trraditionally been dynamited or broken up and drug essentially flat with wire cables (by the government no less, normally with no public hearing) that certainly do not respect the remains exposed and then scattered in the wreckage.
In contrast, a diver visiting a wreck with the approprioate degree of respect is no worse than a Sunday afternoon in a cemetary or on a battlefield.
Where it gets a little greyer is where you have divers going down with tools for the express purpose of looting the wreck for artifacts that are then displayed on private mantels. To some extent you can argue that with salt water wrecks the artifacts are being preserved for posterity, but when they are tucked away in private collections, stuffed in boxes in garages, or even worse, sold on e-bay, that argument gets a little thin. Donating or even loaning artifacts to museums for temporary or permanent display display would go along way toward supporting that argument, but I have not met many divers who actually do that.
In deep, cold, fresh water, environments where wrecks can last several hundred years in a very well preserved condition, the argument that the artifacts are being "preserved" by being recovered has virtually no merit at all unless they are being recovered for the express purpose of being displayed in a museum or for officially sanctioned archeological purposes.
