And beyond the environmental issues, to me not taking specimens is simple courtesy to the guy on the next dive. If you think that shell is really cool looking, don't you suspect the next person to see it will also?
This thought is what has prevented me from picking up some really cool shells. Other than to look at them and then put them back down. But as mentioned by another poster, I remain uncertain that taking a shell has anywhere near the impact that taking fish does. Especially since I am talking about an individual taking a couple shells for their own collection, not the hundreds or thousands that are scooped up by the commercial shops. Same difference between a guy surf casting his reel and the commercial fishing boats.
Let me clarify, too. I pick up garbage, unless something has moved in, or unless it has coral growing on it. Yes, I would pick up a gold bar if I found it (not that it's going to happen) and I suspect virtually all would. Hell, one bar would pay for the next few dive trips... No, I am not going to take anything from a wreck. A shell could be considered a renewable resource. A piece of a wreck is not.