Just got back from Tobermory a week ago. It was AWESOME! They have some great wrecks. I did some shore diving (including a night dive) down near the tugs, that was pretty cool. Also did 4 charter boat dives. The water was really calm and the weather couldn't have been more perfect, but that can change from one minute to the next on the Great Lakes. We dove the San Juncinto wreck, City of Cleveland Wreck, Niagara II wreck, and Whetmore wreck. All really cool dives. It took me 6 hours to drive there, from Ann Arbor, MI. On the way back it took 7 hours and 45 minutes. The extra 1 hour and 45 minutes was caused by the delay at the border. Tobermory is definitely worth the drive. Will be going to Alpena, MI in September, but not sure how much shore diving there is there. 3.5 hour drive from your house though.
Hope this helps.