A few minutes? I remember doing a dive as part of my full cave course. Average depth like 60 feet, but it was 2 hours long. The instructor not thinking that I would be able to do a 2 hour dive (as all our prior dives were in high flow caves like Ginnie) didn't bring his deco bottle. He had 20 minutes of back gas deco, I had 5 minutes on Oxygen. Getting out of the water 15 minutes sooner is worth the $5 in oxygen that it cost (though obviously we got out of the water at the same time after he finished his deco).
Now that being said at the cavern level, you won't need it. Deco is prohibited on all cavern certs, and most first level cave certs. I started bringing my O2 bottle when I was in the intermediate level between the first level and full cave (called Apprentice Plus under the NSS-CDS program). As I was diving to thirds and deco wasn't unusual.
But if you are planning to do AN/DP level dives, having it will often save you money in the long run unless your shop follows typical cave country pricing (ie bottle is $X per day it comes full, and you pay to fill it back at the end of the rental at $Y per cubic foot along with filling it up as needed throughout the rental period).