I did a blind taste test using a 2 pounder and a 16 pounder once. I gave a piece of each to my friend’s wife, who loves lobster, and she couldn't tell the difference. If you really sat and thought about it while you were eating it you could tell a slight difference but it wasn't like night and day, it was a very subtle difference.
Like anything you eat, a lot of what determines the flavor, texture, and over all quality of the food depends on how it's taken care of before you prepare it, how you prepare it, and of course how you cook it. Not to mention the environment it's been raised in and the food it's eaten. Also different parts/cuts of the animal have different flavors and textures.
Anyway, I've had some pretty bad pound and a quarters .....