Hmmm... As you were told, the sun is perhaps the most dangerous thing over here.. If you don't drink very very often and don't put a hat on your head you may easily get a sunstroke, plus, dehidration is considered as a factor that may increase DCS (in addition, you are also breathing dry air both from the tanks and from the desert
)...
The second other risk factor are the taxi drivers in Sinai, especially at night.
Then there is diving itself, which some people (mostly non-divers) and most insurance companies tend to consider it as risky. I never figured out why, but that's the way it is.
I don't think you should be afraid to visit Eilat, but if you really are, you have to consider two things: Most of the interesting sites (and clubs) are out of town. Actually they are closer to Taba than to Eilat...
If you want to do anything else rather than staying all the time at Taba's hotel lobby (shopping, restaurants, cinema, sea activities, and so on) then Eilat is probably the only available place to go...
Enjoy your stay and your dives!