There are actually some subtle pros and cons to both cameras, size notwithstanding. A6xxx series is limited to 1/160s flash sync speed, and while it's possible to overcome that by using HSS (high-speed sync), it's expensive - 2x€1,065 for a pair of Retra Pros, and another €259 for an s-TURTLE trigger to drive them - and you still lose considerable flash power when operating in this mode. This makes shooting into the sun difficult. Conversely, the RX100 series come with leaf shutters that sync with strobes all the way to 1/2000s.
On the other hand, RX100 cameras starting with M3 lack a hot shoe, which leaves the pop-up flash as the only method of triggering strobes. If you're shooting fast action, you may want to use bursts, and the pop-up flash prevents that. The only modern exception is the ZV-1, which replaces the EVF and flash with a hot shoe and microphone, but the only housing for that camera is from Ikelite.
When it comes to macro,
all of these cameras come with their own limitations and trade-offs. The A6xxx series has access to four macro lenses - Sony 30mm, Sony-Zeiss 50mm, Sony FE 50mm and Sony FE 90mm. The 30mm is too short to function as a proper macro lens, the Sony-Zeiss 50mm only supports CDAF which makes its focus slow and unreliable, Sony FE 50mm has its own issues with speed and image quality, and while the Sony FE 90mm gives great IQ, it is somewhat tight on APS-C crop (135mm equivalent) and isn't a speed demon either. I use a Sony A6300 and my primary macro lens is actually a Canon EF-S 60mm f/2.8 Macro USM on a Metabones IV adapter, which works reasonably well, but it requires good light to lock focus - with a focus light, it is generally fast and reliable, but without a focus light, it tends to just hunt back and forth forever.
RX100 macro pros and cons depend on the model. Models III through VA (VA - five-A - is the guts of model VI in a model V body and lens) and ZV-1 come with a 24-70mm equivalent lens, which can be used for macro with a diopter, but is a bit short for the really tiny stuff - you won't be filling the frame with a squat shrimp, for an example. Models I and II are a bit better in this regard, with a 100mm equivalent lens on the long end (and, incidentally, 28mm on the wide end, which is the target FL of most wet lenses - the 24mm wide end that you get on newer models requires a touch of zoom on these to remove vignetting) but they're old and slow, with only contrast detection for focusing. Models VI and VII come with a 24-200mm equivalent lens, which is potentially capable of some
really great macro - you can fill the frame with
incredibly tiny stuff - but exploiting this capability is challenging. Not only is finding, focusing and framing these tiny critters difficult on the photographer, but you need a number of different strength diopters - a +15 lens will let you shoot a tiny subject, but not a slightly larger one for which you may need a +10 or even a +5, so you need to get good at managing those underwater.
Then there is the issue of viewfinders. Many underwater photographers absolutely swear by their 45-degree magnifying viewfinders. They make it possible to get the camera really low to the bottom for macro shots while hovering above it and still having unobstructed view through the lens, and they also make it easier to shoot at an upward angle in wide scenes without going through crazy contortions. For mirrorless users, as opposed to DSLR, they also make it possible to review shots right there in the viewfinder, without switching between viewfinder and the LCD screen. However, while RX100 models III through VII feature a pop-up EVF, no housing that I know of has a viewfinder mount that will access it. SeaFrogs Salted Line RX1xx has a built-in viewfinder, but I tried using the one on my A6xxx Salted Line housing, and it's complete garbage. If you're shooting an RX100 series camera underwater, you're doing it through the LCD screen. Best you can do to mimic an SLR/MILC viewfinder is a magnifying hood that attaches over the screen, something like
this.
When it comes to video, A6xxx cameras do full-width sensor recording at 4K24P, but add a 1.2x crop at 4K30P. As far as I know, RX100V and upward shoot 4K without cropping, and they can do a short burst of slow-motion video up to 960fps
Both A6xxx and RX100 series can be shot as either flexible or dedicated wide/macro rigs. The Alpha cameras can use the 16-50mm f/3.5-5.6 PZ lens, which works well with both wet wide lenses (WWL-1, WWL-C, KRL-1, UWL-09) and diopters, although the 75mm-equivalent long end won't get you into supermacro territory. The long lens of RX100 VI and VII presents some challenges with housings, but it boils down to choosing between a long port and a short one. In a long port, you can shoot the full range of zoom, but because the lens retracts deep inside the port on the wide end, you cannot utilize wet wide lenses, and 24mm behind flat port (which reduces AoV) is not particularly wide, so you're restricted to fish portraits and macro. In a short port, your zoom ability is restricted, compromising macro (but you can still shoot it about as well as an RX100V or an A6xxx with 16-50mm), but you can mount wet wide lenses for about 130 degrees of maximum AoV.