I don't know anyone who's done it as a boat dive, but there is a little info here, which might help you out:
http://www.seaotter.com/marine/html/kvi.html. Every time I've done the site, I've done it by boat, and it looks like quite a long surface swim and hike in with gear.
I don' t know what shore access is like to the site; you could take the main Vashon highway from the Fauntleroy dock down to SW 204th, turn left, head to the water on Ellsport Rd, and then left again on Chautauqua Beach Road along the coast. That should get you close to the site. I'm not sure what the private property situation and parking is like down there. There's a really good aerial photo of the site here:
http://www.harle.cc/sites/kvitower.asp which clearly shows a road entrance on the west side of the KVI property.
It's a good site; currents have always been minimal and it makes for a relaxing dive. I personally find the best things at this site, in particular, are in the 85 - 100 foot range. I usually can find four or five octos there, hiding out in the boulder field, and there's also almost always a few groups of ratfish down there as well. My very first Puget Sound boat dive was on this site, at around 5 PM one December, and I managed to spot a huge GPO out and free-swimming, searching for supper.
The last time I was there, I jumped in and almost landed on a big dogfish sitting in around 40 feet of water. I've also found a few varieties of sea squirts there that I've never seen anywhere else in the sound; these, again, were down around 90 feet or so. So, even though I don't consider this one of the best Puget Sound sites, mostly due to the boring terrain, I always seem to find something interesting down there in the way of fauna.
The shallows at KVI tower, contrary to the review on the site I pointed to, seem to be less interesting to me, even though I enjoy small critters and photography. I think there are better shallow dive sites to be found with lots more life and more interesting natural terrain instead of the artificial reef.
Vashon / Maury has a bunch of great sites all around it that you could try out if this one is too far to swim.
