Suddha,
Generally I see it takes about 50 dives before people really relax enough to get their effciency up to near peak on air use. If you are a good athlete and are in excellent shape, then your O2 consumption rate just will not allow you to have a low SAC rate without raising your CO2 levels, and that is not good. This is one area being a bit out of shape is a bit of an advantage. But with proper diving skills you can still be very good on air.
As was said before long slow deep breaths. I too am a big guy, with very large lungs, and am on the very low side of air usage now. Plenty of good advise has been said already, but to help you visualize it here's another take.
Those who are really good on air get into a 'zen' while diving. I'll not go into the technical view but you will find soon enough, that you'll get to interacting with something lots of time will go by and you'll find you hardly used any gas. It has to do with your not 'thinking' about breathing just like you do when you're not diving. However to get to that point you need to develop good dive habits, like long slow deep breaths, excellent bouyancy skills and such.
There are a few tricks to help yourself, for instance remember when in early grade school and you were told to carry a musical note, by going aaaaaaaaaaaaa try that and you'll see you can carry that on for a very long time. Try doing that while diving. Another trick is to use your snorkle on the surface for a couple minutes 'preparing' for the dive. Someone could write a book on this but for now just get out and dive, stay relaxed, take your time, and really enjoy what you're doing.
As far as why most women use less air, there are a few reasons, one, generally their muscle mass and resulting need for O2 and two their lack of ego. Many guys I've dove with that are relatively new, still swim a bit fast, a bit of gotta get there attitude. By taking your time and really looking at and for small animal will help develop a slow pace, resulting a lower consumption need for O2. Combined with good breathing patterns will result in your air supply lasting longer. Try letting your wife lead the dive and see how that changes your pace.
So go dive, enjoy and don't worry about it, with experience and dive skills it will come.