You are a self described recreational diver with no interest in technical diving, and you want to be as safe as possible diving recreationally in cold water. The clear answer IMO is single tank with a good sealed regulator, good cold water technique, and a reliable buddy. You might consider a pony bottle, but it should not be necessary. It might make you feel safer, but it's unlikely it actually is safer, at least statistically. I'm not saying its a bad idea, but among the potential dangers in recreational diving, running out of available air should never be one of them. This is what gas management, buddies, and immediate access to the surface are for. Both are basic fundamental tenants of recreational diving.
Safety is based on diver judgement and behavior, and especially partnering with other safety-minded divers. Equipment is really not the issue here, other than making sure your gear is in good condition and reliable. Simplicity in set up and your familiarity and ability to manage the gear are probably the most important factors. One thing I would suggest is that you have some redundancy or contingency in buoyancy; likely a drysuit, which you may have mentioned and it flew by me, because cold water divers will eventually be in drysuits or they will quit diving in cold water. Nobody likes freezing. But if you're diving wet and you are using a lot of weight, make sure it's possible to ditch enough to swim to the surface and maintain control on the surface in the event your BC fails.
There is no reason to go for double tanks, either backmount or sidemount, unless you are interested in technical diving, and that comes along with technical training, part of which will be managing the new, more complicated gear arrangement.
I admit I haven't read this whole thread, but I see that some people are just encouraging you to dive like they do. Not always the best advice.