A lot depends on the mix of the guests at the resort. I was recently with SWV for four days and went to Sipadan two of those days. The resort was full, but it was full of Malaysian, Singaporean, and Hong Kong Chinese; about 90% of Chinese people can't swim, so they snorkel, and they do use some of the passes for snorkeling at Sipadan, but mostly they snorkel someplace else. This may also explain why at CNY ocdiver was able to get to Sipadan so many times.
Two years ago my wife and I had the same sort of experience at SWV as Quero. We were there for three days of diving (i.e., a "5-day package") and went to Sipadan one day, which is what we expected though we of course hoped to luck out and get two out of three days. The resort had a large group of Asians on a trip hosted by their employer, and many were non-divers. The resort was otherwise lovely. My wife and I were lured by the bungalows on pilings over the water--something we had never done before--and our bungalow was indeed wonderful. I can highly recommend SWV as a "couples thing." That said, some friends of ours had been at Borneo Divers a week earlier and dived Sipadan twice in four days of diving. We were envious, but it was our own fault. In hindsight, three dive days is just not enough at any of the resorts to have any reasonable likelihood of visiting Sipadan twice.