Trying to get some data for underwater white balance

[grassybreakfast]

Hello!

I am currently writing some free software to do automated video processing for underwater videos (colour correction, denoising, LUT application, transforming from log encoding, etc), because I'm sick of green videos, and while I've always post-processed my own video manually... that takes a lot of time, and mainstream free tools don't really do it well (eg. historgram stretching is not a very good solution to boost red, and can result in weird artefacts).

I would like to calculate a white balance curve for underwater footages (indexed by depth instead of lighting colour temperature), but don't currently have the data. Would appreciate if anyone with a suitable camera can help collect some data!

I need to be able to calculate linear sRGB from the colours, so practically speaking, it needs to be 10-bit and in log encoding. That's GoPro 11/12/13 or DJI Action 4/5 (ideally both!).

What I need you to do:
* Set the camera to Log encoding, 10-bit, fixed 6500K white balance, and shutter time/gain on auto.
* Shoot a photo or a short clip of white slate (or something else even and white) at different depths! If you can record the depth (write on the slate?) that could be interesting, but that information is not critical, so don't bother if it's too much hassle.

Thanks!

PS. If you are a programmer, feel free to take a peek at the in-progress code here: GitHub - matthewlai/JaxVidFlow: Experimental video processing pipeline using Jax

 

[sea_ledford]

While I think this would be cool, I'm not sure it'll solve the problem. Two variables you aren't considering is the color of the water to begin with, and the distance from camera to subject, which would need to be fixed.

Then it would only work for that color water at that distance. Anything else and you'll be back to tweaking every clip. It might be fewer tweaks, but you'll still have to adjust.

I ended up making a few specific mesophotic LUTs for some ongoing work in 160-200' of water off Florida. Moving from one site to the next, diving at a different time of day or shooting closer or further away from subjects all requires enough adjustments I only use the LUT for quick previews so I don't have to stare at the washed out log footage.

Apparently that is causing an overall trend in desaturated colors in tv shows and movies because the editors are getting used to seeing straight log footage, so anything with normal color saturation starts to look garish.

 

[Pearlman]

Another variable would be I think amount of sunlight and cloudy conditions… I am not quite sure that depth can be the singular parameter to calculate a WB curve whereas I see good results with UCC using color temperature which is a realistic empirical measurement on the spot by the camera sensor.

 

[tursiops]

I agree with the previous responses. Depth is only one of several important variables, and perhaps not even the most important one. Also important are water type, sun angle, distance to subject (the light from the sun has to get to your subject, and then the reflection off of the subject has to travel through water to get to the camera), and the water type at depth may not even be the same type of water that the shallower water is.

In addition to commercial apps like Apple's AquaFix and Diver+, a lot of this has been covered in the work of Akkaynak et al, https://openaccess.thecvf.com/conte...er_From_Underwater_Images_CVPR_2019_paper.pdf.

Added: You might want to talk to an optical oceanographer....

 

[grassybreakfast]

Indeed, depth is just one parameter and there are others. But depth is probably the most important one in the context of underwater videos. With enough data points I can do something like principal component analysis to parameterize it in 2D or 3D, or even just keep the data points as is, and do some kind of k-means. What I am looking for here is essentially just building a plausible set of RG gains that we can search within.

The goal is definitely not to get to the standard of professional colour graded videos. It's about getting 80% there with 20% the work. When I shoot something quick to post on social media, I would like to have something half decent without having to do almost any work. That's really all this is for, and also as a starting point for colour grading if you want to actually make a good video.

 

[grassybreakfast]

And yeah obviously the best if we use a white/grey card for white balance at every shot, but most of us don't want to spend that much effort filming while diving.

 

[grassybreakfast]

Another variable would be I think amount of sunlight and cloudy conditions… I am not quite sure that depth can be the singular parameter to calculate a WB curve whereas I see good results with UCC using color temperature which is a realistic empirical measurement on the spot by the camera sensor.

By colour temperature do you mean colour gains? The problem with the standard colour temperature white balance is that colour temperature is completely the wrong curve to be on when under water. Even at 10000K (which would have the highest red gain), the red gain is still not nearly high enough.

But yes the surface lighting conditions would definitely need to be taken into account.

 

[grassybreakfast]

I agree with the previous responses. Depth is only one of several important variables, and perhaps not even the most important one. Also important are water type, sun angle, distance to subject (the light from the sun has to get to your subject, and then the reflection off of the subject has to travel through water to get to the camera), and the water type at depth may not even be the same type of water that the shallower water is.

In addition to commercial apps like Apple's AquaFix and Diver+, a lot of this has been covered in the work of Akkaynak et al, https://openaccess.thecvf.com/conte...er_From_Underwater_Images_CVPR_2019_paper.pdf.

Added: You might want to talk to an optical oceanographer....

Thanks. I have read that paper actually. It's very interesting but a bit out of reach for the scope of this project (eg I'm not going to be able to get the whole spectrum response of the sensor).

 

[Pearlman]

By colour temperature do you mean colour gains? The problem with the standard colour temperature white balance is that colour temperature is completely the wrong curve to be on when under water. Even at 10000K (which would have the highest red gain), the red gain is still not nearly high enough.

But yes the surface lighting conditions would definitely need to be taken into account.

Right - A camera sensor with UCC implemented records the separate RGB values and applies corrections algorithmically to the recorded images. You are right that the red gain is nearly never enough even with UCC and which is why we all do additional color correction in software to achieve the results we want.

A quick google threw up this document and study that summarises the different variables that impact color balance underwater Color Correction for Underwater Photography … and it details why relying solely on depth as a numerical value will not provide better results.

 

[grassybreakfast]

Right - A camera sensor with UCC implemented records the separate RGB values and applies corrections algorithmically to the recorded images. You are right that the red gain is nearly never enough even with UCC and which is why we all do additional color correction in software to achieve the results we want.

A quick google threw up this document and study that summarises the different variables that impact color balance underwater Color Correction for Underwater Photography … and it details why relying solely on depth as a numerical value will not provide better results.

Ah yeah that makes sense. Most action cameras don't have that (DJI Action cameras have some kind of white balance sensor, but no one really knows how it works). Yeah I think for now I'll just do histogram stretching (functionally the same as using a white balance sensor, but post-encoding rather than pre-encoding, so not as effective use of bits).