This is a thought-provoking question to answer. I have stayed at and dove with the Village and would recommend both if you go to Pohnpei. When I was last there it was a basic two dive a day type operation. Pohnpei diving is decent (reef sharks, mantas), but due to winds, I was never able to dive Ant Atoll. Friends who have dove Ant Atoll liked it. On one dive I had 5-6 mantas and on another dive I saw at least 25 grey reef sharks in a pass swimming into the current (quit counting at 25). I met an Australian family who combined some diving with surfing there and they had a good time.
The Nan Madol ruins are definitely worth seeing and the waterfall at the end of the Nan Madol tour was a great way to cool off. If you are looking for big beautiful island beaches, Pohnpei is not the place to go. I am glad that I dove Pohnpei but I was living in the Marshall Islands at the time, so for me travel to Pohnpei was easy. If you are island hopping on Continental Micronesia between Guam and Hawaii, definitely stop and do some dives. You could then also dive Chuuk and Kosrae. If you are going just to dive Pohnpei, it will take some time to get there and you will have to pay the airfare that the one airline that serves Pohnpei charges.
If you are looking to check an exotic dive spot off your list and/or you like diving where you may be the only diver, or with only 1-2 other people, Pohnpei may definitely appeal to you. If someone were to offer me an all expense paid dive trip to either Pohnpei, Palau, North Sulawesi or Sipadan (dove all of these), I would probably debate between North Sulawesi (with Bali included) and Palau. Although, I would not mind seeing what Ant Atoll has to offer.