That was a Spotfin Butterflyfisn, Chaetodon ocellatus. They are by far the most numerous Butterflyfish swept up into the northeastern US by the Gulf Stream. Some days, in some places like the NJ inlets, you can see dozens in just a few minutes of diving along inlet rockpiles. The fish usually called a Reef Butterfly, is C. sedentarius, very similar to a Spotfin, but with an outline that is more squarish than round, less brilliantly white, and retaining a black bar at the lower part of its caudal peduncle which Spotfins lose as they grow to maturity. Reef BFs are exceedingly rare north of the Carolinas. Spotfins are the most numerous here in NJ/NY/ Conn/ RI.
Four eyed BF (C. capistratus) juveniles are seen with regularity, Banded BFs (C. striatus) rarely, only occasionally, Reef BFs very seldom. I've caught all of them (they are all juveniles up here) and raised every species to maturity. Spotfins and Reef BFs are easy, Four eyed and Banded much more difficult. Live blackworms are essential in their early maintenance. I've been fooling around with this sort of thing since God was a boy, volunteering as a collector with various university and aquarium projects. Sometimes we've found amazing things. I once caught a Caribbean Blue Tang, about halfway through its transition from yellow to blue in Menemsha Inlet on Martha's Vineyard. I think this collecting was the most fun of any diving activity I've ever done.