I'm a research psychologist, with a focus on emotion and cognition. Panic, like all emotions, results from what we call bottom-up "core affect" and top-down automatic construals.
Core affect itself consists of valence (i.e., positive vs negative affect - general feelings of goodness vs badness) and arousal. When you wake up on the wrong side of the bed feeling groggy and in a bad mood for no reason - that's core affect. Core affect is largely physiological. For instance, drugs like caffeine or epinephrine induce arousal, and many recreational drugs (including alcohol) induce positive affect. Panic is built around negative affect + high arousal. Which, of course, can be physiologically induced by hypercapnia, and that process is involuntary. (Although, as folks have pointed out, experienced divers may have the skillset to prevent those physiological responses, or in much more mild cases of panic use techniques like square breathing to manage/reduce their intensity).
How we experience that core affect as an emotion depends on our automatic construals - essentially how our brain interprets that affect, and what we think it "means." For instance, you're at the top of a roller coaster: your heart is racing, you can feel your pulse, your fingers are trembling. Are you excited or terrified? Depends on how you feel about heights and rollercoasters.
These aren't just semantics; feeling excited versus terrified is real to the person experiencing it - they really do feel different to us. Most construals happen fast and automatically - when we say that someone has "learned" a fear or panic response, this is often what we mean. For instance, coming to associate loud noises as a sign of danger, means that subsequent elevated arousal in response to a sudden loud noise will be felt as fear (not excitement), because our brains are putting together a recipe that high arousal + loud noise -> danger!
Those construals are automatic, and habitual; they happen largely outside of conscious awareness or attention. But we can also override them, either through slow conscious thought ("it's just fireworks") or by repeated exposure to the "threat" in safe environments that rewires our initial associations to change the automatic construals itself - this is the basis of exposure therapy, which is the gold standard for treating anxiety. In fact, in clinical psychology, one approach to treating panic and anxiety disorders is teaching people NOT to interpret their physiological indicators (fast heartrate, sweating, etc) as signs of anxiety, but to consider other possible benign interpretations ("maybe I just ran up the stairs too quickly"). And doing this reduces panic attacks.
All of this suggests a lot of reasons for some people are going to be more prone to panic than others, and why experience and training matter (but may not prevent panic entirely in extreme situations). In psychology, we have something called the "strong situation" hypothesis, which is that in sufficiently extreme situations, individual differences cease to matter. How much you as an individual love or detest running isn't going to matter if the building you're in is on fire - almost everyone is going to run out of a burning room, given the opportunity.
But, to bring it back to topic: some divers are going to be more prone to hypercapnia and the changes in core affect it produces, either by virtue of their physical fitness, their equipment, their dive plan, conservatism, etc. The more likely you are to experience the core affect needed to produce panic....the more likely you are to panic.
At the same time, what the brain "does" with that core affect matters. Research by Jim Blascovich for instance shows that novices experience heightened physiological responses as a "threat" (and tend to do poorly on tasks under high arousal), whereas more advanced folks experience those same response as a "challenge" (and tend to do better on tasks under high arousal). That effect replicates widely across contexts - for instance, being watched hurts performance in beginners, and helps performance in experts, because being watched by others induces arousal. Whether that's good or bad depends on whether it interferes with your skills - which for beginners who are using a lot of bandwidth (as someone mentioned above), it does! Same physiological response, different results.
More experienced divers are not only likely to know how to prevent hypercapnia, if they do experience it (at low levels) they are more likely to recognize it and correctly attribute those feelings to external sources ("I'm taking a CO2 hit", not "I'm out of my depth and about to die"). They are more likely to experience difficulties as challenges, rather than threats. They are more likely to have skills down pat, so that at low levels increased arousal is actually potentially beneficial (or at least not harmful) to their ability to get the job done. In other words, there are many reasons why experienced divers may be slower to "interpret" the physiological changes in core affect brought on by CO2 as "panic" per se - and thus, less likely to panic. And those factors may change not only from diver to diver, but from day to day, or from dive to dive.