A few recommendations - in increasing order of quality, flexibility, and cost!
1) Olympus TG4 (without housing) currently about $349-379 on sales. The TG4 is a fantastic camera for beginning underwater photography. It has a fast F2.0 lens, zoom, built in flash that works well for shallower dives and close up shots, does not require a housing (up to 50 foot dives), and is capable of expansion with a housing, tray, arms, and strobes. Amazingly for its price, it also shoots in RAW. Best choice for beginnings, shallow dives, snorkeling, and also great as a backup system or a grab and go underwater system.
2) Sealife DC1400 Pro (includes LED strobe, arm and tray) currently on sale on Amazon for $499. This is a bargain. My very first UW setup was a Sealife DC500 Pro. The Sealife DC series are simple to use, easy to hold (very small), pre-setup for UW photos with dive mode built in, and really as close to point and shoot as you can get underwater .. nothing to do but charge, dive and shoot. On the downside, you cannot shoot in RAW, cannot change lenses, and are pretty much stuck with Sealife accessories (not many). The lack of RAW mode is a real deal-killer on this camera for me though. If you are planning to post-process, you will probably tire of this camera quickly. I recently shot some pics with the DC1400 I will send you.. you can see that the lack of RAW mode really kills post-processing.
3a) Olympus TG-4 with PT-056 underwater housing and UFL-3 strobe - expect to pay a total of about $1,100. This is my travel system (I have the UFL-1 which is a bit larger). All of the features of the TG-4, plus a great small housing and tiny strobe that is easy to travel with or stick in a carry on. With an adapter ring, the Olympus wet mount fisheye and macro lenses can be screwed on. The UFL-3 strobe is powerful (guide # 22), has good wide coverage, and is digitally controlled by the TG-4 using imperceptible pre-flash signals, with the result that your flash power is usually perfectly adjusted. A point and shoot with some serious semi-pro features. I just bought this system myself to replace my E-PL3 system below. It is not any better in image quality, but it is about 1/3 the size versus the E-PL3 and housing.
3b) Buy my Olympus E-PL3 with PT-EP05L housing with compact tray system, 2 handles and long arm to mount flash (you would have to buy a flash - works great with Olympus UFL series flashes and Sea & Sea flashes). Has RAW mode, mounting options for wet macro and fisheye lenses, etc. and has an excellent 14-42mm lens that is perfect for underwater .. Not as compact as the TG-4. The housing is currently sold for $799, and the camera for $229 (on sale currently)... I paid well over $1,200 for mine, not including the tray, handles and arm. The E-PL3 was backup to my Nikon system for several years.. it was only used on 4 of my dive trips. When the TG-4 came out I decided to use that as my backup system due to its smaller size so I don't need the E-PL3 and housing anymore. I would sell the system (no flash) for $750 including the tray, handles, strobe arm, flash cable, and a couple of extra batteries. Add flash and you are ready to go. I will send you some pics I recently took with this system too.