It's the bands, and perhaps the technique you are using to load it.
Your best choice may be a Riffe midhandle, midsize gun, with a rest fin. Use a light shaft, and make your own bands from the heavy rubber, just lengthen them to the point you can just cock it to the rest fin, then shift the gun but to your chest and bring it from the rest fin to the back fin. repeat for the other bands. By suing a longer gun you'll be getting more "stretch length" in the bands so you can decrease the overall pull of each band and still be able to hit fish. The midhandle makes for a ore compact package that handles about like the "regular" guns do with the trigger under the shaft retainer, but two sizes smaller. You may also want to think of getting a "custom" gun that will take 4 bands.
Riffe and Ridell are the only two guns I know of that are both fairly readily available toploaders that will take a "fin" attachment without a slider, enableing use of the rest fin about 2/3 the way down the shaft.
Work on the loading technique first though. Load should be a single movement to rest or first notch, then shift to center of chest and finish the load.
FT