Generally, women don't get bigger from lifting weights, unless they are taking something anabolic. Weight lifting will help greatly with the upper body strength, something that most women will suffer from, compared to men.
Ive always been stronger in the upper body than most women and found the Air Force OTS PT test was quite easy for me to max, as far as the women's version. It was weighted heavily on upper body strength and I found the standards were not difficult. However, I came from a lifetime of riding horses, grooming, stall cleaning, feeding and lifting hay bales/feed bags. Basically, I grew up lifting weights. This helped me greatly because it was an area of training that I didn't have to focus on.
If the OP is coming from a non-military background, I would recommend reading as much training material as possible. Try to get ahold of the academic materials for the dive school and any other material that you possibly can. I found that my biggest stress was simply learning all the militarisms since I came from a non-military background. It gave me a huge disadvantage,since most of the other students in my OTS class were prior enlisted.