I was in Yap this past January and there wasn't much to do besides dive. If you want to see the natives, you will see a lot because Manta Ray is in the middle town.. if you want to take a cultural tour, Manta Ray offers that too for about $50 I think. Kayaking, cultural tour, and other land activities can be done after a dive day, and they seem to be scheduled like that too so people could dive then do something else after with more than enough time.
We were usually back by 1-3pm and still had lots of time to kill. Check out "Beyond the Reef" too, a smaller operation, less crowded(there was only me and 2 others), VERY personble service. I.E. the dive master will ask what we wanted to see that day, like a good macro site, the mantas, or turtles, etc. something specific, and he'd do everything he could to get us to see what we were looking for. Manta Ray was good too though, I dove with them on one of the 3 days so I could get some video footage for an extra $50, but the boat had about 15 divers. If I went to Yap again, I would dive with Beyond the Reef again though, diving with Manta Ray was like taking any other day trip dive tour... diving with Beyond the Reef felt more like just going out with a few friends to go diving. They both go to the same spots too.
Most of the divers I met there were doing 3-4 days in Yap and then staying a week or more in Palau. I personally woudn't mind staying in Yap longer because of the stockpile of Mantas they have there, but most people seem to spend more time in Palau. Have fun in betel-nut country!