What I am getting at is your situation OP, would be an extremely scary situation for me.
But it need not be. So let me walk you through some of the mental processes. (And FTR, I'm a longtime NAUI instructor with over 6000 dives and have taught literally thousands of students over the years.) What you're describing, and this is an observation not a criticism, is the path to why people panic in situations like this. So let's review step-by-step what's happening and going through our brains.
1. Everything is hunky-dory on the dive. I've got plenty of air, it's a pleasant dive, and I'm going to duck under the bow to see if there's anything interesting there.
2. I come out three minutes later and I don't see anyone in the group.
3. Nothing has changed from three minutes earlier. I still have plenty of air, it's still a pleasant dive, and I simply don't see other people.
4. I decide to abort the dive.
5. I'm now going to do a safety stop. This mean I've got five minutes (like to to do extended safety stops) top assess things. No need to make rash decisions, no need to do anything other than hang.
6. I look around to see what my choices are. The reef in front of me tops out at maybe 15-20 feet. I could surface there and there probably wouldn't be any boat traffic because it's so shallow.
7. I also carry two SMBs (Surface Marker Buoys) with me so would also be able to deploy them.
8. I also know I could swim back to the stern mooring line - 300+ feet away, length of a football field - and maybe our boat's there. But if it's not, I may have created a different problem, because on the surface that's further away from any reef and more prone to boat traffic.
9. I also note that the mooring line to the bow is taut and when I look up, I see there's a boat tied up to the bow mooring.
10. I know it's not our boat - this one has two engines, we have one - but I also know that surfacing on a line next to a boat is safer that surfacing on top of a reef. So I decide that that's what I'll do.
11. Still have plenty of air - over 1,000psi - still have plenty of no-deco time left, and am simply a little embarrassed at having lost everyone.
12. As I'm finishing up my safety stop, I look in the direction of the stern of the boat and see the dive guide coming towards the bow, obviously looking for me.
13. I signal him that I'm OK, he signals back and motions towards the stern, I head that way to find the rest of the group (three other divers) on the stern mooring line, we all ascend, no harm no foul.
And the point in all of this, and not to diminish the fear that you said you would feel had this happened to you, is that that type of fear is something we make up in our minds. It's not based on anything that's actually happening in reality. And that's what leads to panic. But if you can simply rationally think things through, there are numerous options that produce a successful outcome.
I think one problem and the root of the fear you said you might feel - and this is totally my own opinion/observation - is that when we train you, we needlessly put the fear of being alone in the ocean in the back of your mind. I also do fatality investigations and am always amazed by the number of times we see a fatality where there has been an inadvertent buddy separation, there's absolutely nothing wrong (plenty of gas, plenty of deco time, no entanglement, etc.) yet one diver ends up dead because they end up panicking and create a bad situation for themselves.
When we train you initially, we emphasize the buddy system and if we don't say it directly, we at least imply that if you're by yourself, you're in mortal danger. No you're not. Even if I had had a buddy with me in this situation, we would have had the same choices as to where to surface.
I would like to see us as an industry, in a basic class, add a skill that teaches you to be self-reliant if you lose a buddy. What I've done in the past with my students, in the pool, is have everyone (including me) get out. Then I have one person go under and swim one or two leisurely laps around the pool while I observe them from the surface. Repeat for each student. This isn't meant to encourage them to dive solo but recognizes that, especially with newer divers, sometimes things go wrong. And this hopefully gives them another skill and the feeling that if I'm all alone in the water, it's not the end of the world and they can safely decide to abort the dive, do a safety stop, and surface.
Just some things to think about. End of lecture.