Ke'ei and the Place of Refuge are better that Kealakekua Bay is for the most part. There's some diving out front of the Cook Monument, but it's frankly a better snorkel than a dive... lots of plate corals, but most of the variety is above 15-20 feet or so. Manini beach on the south side of the bay offers some shallow lobe coral reef, but nothing spectacular, the wharf area is really not particularly nice, you're better off heading to places you've already been.
Puako has lots of dives, if you didn't spend a couple days of diving up there it might be worthwhile spending some more time up there and checking out different public accesses. Mahukona is pretty nice up north. There's a couple of guys here who know the northern shoredives better than I do, hopefully they'll chime in later.
Down in town the old airport park offers a couple of nice shore dives. The dive off the King Kam beach out to the right beyond the pier is surprisingly OK for a change of pace. Outside of Honokohau Harbor to the left off the beach is a very nice dive site. I've only done Pahoehoe park (about the 3.5 mile marker on Alii drive) once, once you get out to about 35 feet it's nothing but finger coral reef stretching out seemingly forever. I did a shore dive out to the north of the mouth of Keauhou Bay once, pretty good. There's lots of other spots I've heard of people dropping in and finding good diving. The only "random" choice dive I've ever done here that sucked was when a friend wanted to try diving off Kukio beach by tge 4 seasons.... 35 minute dive in 9 feet of water, saw one helmet shell and nothing else, it was the worst dive I've ever done in Hawaii. A year later another friend suggested we dive the little beach just a couple hundred yards south of Kukio, it's sort of a man made beach with imported sand, it was a very nice dive.
I've dove the south end of Milolii down by the park, it was quite nice. I only would go down there with a group, with someone to watch the vehicles, due to it's reputation for not being over visitor friendly. If you want to spend a day on taking a chance, I did a couple of very interesting dives down at Manuka one time. I wouldn't make that drive without 4WD and pretty much a guarantee that the water's flat. Lots of lava flow that's probably only 40-50 or so years old, little coral, but a buich of swim through and crawl through structure that hasn't caved in yet. Kona Paradise is another interesting dive down south.
Anyways, there's plenty of spots for the adventurous sorts.