I think it all depends on your comfort level in the water, and the conditions in which you'll be diving. I did one dive before moving on to AOW, but with that being said, I grew up free diving and am very comfortable in the water. I already knew how to clear my mask underwater, and I was even a lifeguard in college. I was ready to take my AOW, but for some of the people in my OW class, I would have highly recommended they wait and get some more dives logged.
If you have to ask if you think you need more dives, you probably do. Get more comfortable in the water. Plus, for AOW, usually gear isn't provided, so you either end up renting, or buy from the LDS. This could be a push for you to dump money into their shop. Ask the instructor as to WHY he/she thinks you should take the AOW class. If they give you a reason like, "It will make you a better diver," go and get some more logged dives. Get your buoyancy skills perfected, and be able to clear your mask with one hand. The more challenging a dive is, the more difficult it will be to overcome small obstacles. Things like mask clearing and buoyancy control should be second nature before you take AOW.