It's not about right or wrong; there are some objective facts about a location ("it's known for currents" or "the viz here is usually amazing" or "turtles abound"), but diving is very much a subjective experience.
Of course it is totally a subjective experience. But some "objective facts" (I'm not sure I'd use this term, maybe reliable observations?) apply to Raja Ampat as well. The coral is off the charts! Especially in the Misool region. These are some of, if not THE best coral gardens in the world. This is what you should expect when diving here. Secondly some of the sites such as Cape Kri, have so many fish and so many kinds, it is easy to lose sight of your dive group. If you did not experience this then I think you had some bad luck. I lost sight of my buddy in the fish soup at Cape Kri and it was slack tide! I would hope this is what most people would experience from a dive trip at Raja. 50 feet of vis is certainly enough to witness this.
This is an excerpt from a trip report I wrote a couple years ago and basically describes my entire trip, if yours was not like this, I am sorry:
"Despite the claims of no sharks in RA I was surprised to see six black and white tip reef sharks almost immediately. Other highlights of our first dive were two mantas, (12ft and 6ft) a Wobbegong shark, four blue spotted stingrays, Big Napolean Wrasse, Giant trevally, huge batfish in small schools and several Barramundi Cod that would dive under bommies when a diver approached. The barramundi sighting would be a regular occurrence on almost every dive and I remember seeing a very large one the size of a small dinner table that would not come out of its hiding place.
On the second dive a huge marble ray the size of a large dinner table swam right through our dive group. It was soon followed by an eagle ray that swooped by as we perused an amazing sloping wall of soft corals and giant sea fans. A group of Juvenille Batfish was found among the millions of glass fish hanging around the lettuce corals. Another Wobbegong shark, a large school of yellow line snapper and a larger school of juvenile barracuda ended our dive after an hour.
I will refrain from boring the reader with descriptions from my dive logs but I wanted to make clear that this was the typical underwater experience we had on our trip; something rare and amazing was found about every five minutes of the dive. "
---------- Post added January 30th, 2013 at 04:48 PM ----------
So this is an interesting question. What do people consider the best dives or signature dive sites in Raja? Besides the oft-photographed Boo Windows (if that's even on the list) and the manta sites (not unique to Raja)?
On my earlier trip the excellent Indonesian DMs said they felt Misool was the best diving. We never made it there so I don't know.
For me it was Sardine reef, Cape Kri, Pearl Farm Pier (macro + HUGE school of resident scads), The passage, Manta Sandy
In the south it was, Nudi Rock, Boo Windows, Kaleidescope,... and I know I'm forgetting one... or two... or four

---------- Post added January 30th, 2013 at 05:11 PM ----------
Raja Ampat is very remote and there are still many other places in the world to dive, so that was probably it for us. We're still interested in giving the other half of the island (PNG) a try, though.
Oh and by the way, PNG is very much like Raja Ampat. In my opinion both are amazing, but what you get in certain parts of PNG, like the outer islands of Kimbe Bay and Kavieng, are SHARKS. There are some sights with huge schools of Barracuda+ Jacks and the coral is, like Raja, incredible. I can definitely recommend the MV FEBRINA's signature Itinerary. If you are not happy with this trip, I really don't know what tot tell you.