I would try Cairns to start off with.
As someone has mentioned there are hostels on the Esplanade of Cairns which is the main street along the water and Trinity inlet.
Myself I would try Hides hotel.
It is the next street back off the Esplande and there are two parts, one back packers and one hotel.
I think we paid $65 a night for a double but has probably gone up.
It is much more private then a hostel and ill be honest im girly and like my own bathroom espeically if i have dive gear needing washing.
As someone else has pointed out diving in Cairns can be expensive expecially going to the outer reef.
Honestly it depends on what your budget it. You have said you want inexpensive but how inexpensive are we talking here?
If you want the best bang for your buck I would go a liveaboard which when broken down has diving 4 dives a day, somewhere to sleep that night and food for the day.
You have the Pro dive out of Cairns, Cairns dive center, Deep sea divers den and for a day trip only Tusa dive out of Cairns.
If you head to Port Douglas you have Quick Silver but you are getting more expensive there as well as with the spirit of freedom and taka dive from Cairns.
I would do some googling and look up these places and their web sites will show you their dive sites and prices and go from there.
If you want 1 day diving fine. If you do more it is probably cheaper to do a liveaboard.
If you are staying at Hostels I think the YHA hostel people may still have a members card you can buy and it gives you discounted nightly rates at their hostels and there hostels are world wide.
Sydney id try the harbour and over towards manly.
Melbourne is apparently getting the HMAS Canberra but it wont be sunk for a while.
Brisbane has the HMAS Brisbane and Western Australia has the Swan however that is a bit south of Perth.
Of course if you want white sharks South australia and i always liked Rodney Fox.
Try searching the forum as well as im sure there is some previous threads with helpful information and web links in them.