Yep, I carry all of that. I may get too old to carry all of my gear to the boat at once, but not yet. The pony & reg adds a lot, but it's the best buddy.
I do carry one of those in my dive bag to use on a boat if needed, as well as in my hiking pack and a couple in my car. I know that some have suggested one in a BC pocket, but I really don't think it would help in the water.
Some have said they carry a quart bottle of water hung from their BC. I guess it would be damned nice to have if floating long.
No, you need a dive canister. I got the smaller of these:
CANISTER for PLB: SCUBA Diver Emergency Locator Beacon/VHF Marine Radio. | eBay
Use silicone grease lightly on the o-ring.
There are a number for sale on ebay, but you'd need to know the inside dimensions of what you buy. You might want to replace the o-ring now and then as well as apply silicone grease, but even if it leaks a little - the PLB should be water resistant.
Get one that floats, and be sure the lanyard fits inside too.
And in my car as well.
So what part do you not think is a good idea. I hope they find your body whenever you go drifting.