Jump to content




Photo

How does shooting Log grant more dynamic range? Re: noise floor, highlights, gamma, bits


  • Please log in to reply
13 replies to this topic

#1 Andy Zou

Andy Zou

    New

  • Basic Members
  • Pip
  • 7 posts
  • Director
  • New York

Posted 01 September 2015 - 02:54 PM

I understand that shooting Log on the gh4's new V Log profile is supposed to afford me two more stops by the way it packs the information in the allotted bits but...

Where are those stops coming from? Where were they before? If the sensor can acquire more stops than it can cram into rec709 h.264, how does it discard that information? How does this relate to the noise floor or clipping in the highlights?
  • 0




#2 David Mullen ASC

David Mullen ASC
  • Sustaining Members
  • 18789 posts
  • Cinematographer
  • Los Angeles

Posted 01 September 2015 - 04:01 PM

Those extra stops are what the sensor is able to capture.

 

Think of it this way, if the sensor, the recording, and the display device are all linear, no gamma curves applied, and the sensor can capture 14 stops and the display can show 11 stops, then is it better if the recording has 14 stops in it or 11 stops in it?  What's the point of three more stops of information if you can't display it?

 

Well, one advantage is that you can apply gamma curves, power windows, luminance keys, and knee and shadow compression to allow bright highlight detail and dark shadow detail outside of the display device's range to burn off to white (clip) or fall off into black more gradually while leaving the midtones with a more linear response to light.


  • 0

#3 Andy Zou

Andy Zou

    New

  • Basic Members
  • Pip
  • 7 posts
  • Director
  • New York

Posted 01 September 2015 - 04:58 PM

I understand that much, the advantages of log as an acquisition format. But why don't more cameras have it? Why did it take so long to develop for the GH4? Obviously I'm not a camera technician, but since they know what they can do with the RAW files from the gh4 for stills, how come it is so difficult or development heavy to simply apply what amounts to a logarithmic curve?
  • 0

#4 Tyler Purcell

Tyler Purcell
  • Sustaining Members
  • 2372 posts
  • Other
  • Los Angeles

Posted 01 September 2015 - 05:00 PM

The problem lies with recording formats, not the imager itself.

When the current digital technology was developed over a decade ago, there were standards set so everything would be compatible with broadcast television. Since broadcast already used REC709 color space, it was a no-brainer to make digital formats follow the same protocol. All be it, with more dynamic range then the standard definition versions that came previous. They also needed to come up with a way to record high quality images using slow processors and not very much memory. So they came up with an 8 bit 4:2:0 interlaced MPEG format, which only samples blue information on half the lines and red information every fourth line. This format has been carried along by Panasonic, Sony and Canon for years, mostly because of cost. This "problem" is all about money and when you buy a still camera that shoots video, they have to compromise somewhere.

The problem with 8 bit 4:2:0 is that it doesn't have very good dynamic range. The solution these companies came up with to make the format work is to compress the image's dynamic range. Like noise reduction on analog audio tapes, this encode/decode system will increase your perceivable dynamic range to allow more adjustment in post production. It really only works well with 12 bit YUV/RGB software like DaVinci Resolve.

Higher end cameras have 12 bit, 4:4:4 color space, but even they are restricted in color space when they hit a standard recording format like MPEG, XAVC, Pro Res, etc. So this encode/decode system is very popular in the digital age and it allows standard REC709 formats, to accept higher dynamic range images. Those cameras also allow RAW capture which gives you full dynamic range without being stuck in a Rec709 codec. However, they too are an encode/decode system and in the case of .r3d (RED CODE), Cinema DNG and the like, they need to be decoded before editing, which can be very time consuming. When you use standard REC709 formats, they generally work OK in most modern software without the need for pre-editing transcoding.

Why more cameras don't have S-Log is simply laziness.

Why anyone still makes anything that records at best with 8 bit 4:2:0 color space, is beyond me. It's the companies reluctance to license XAVC or Pro Res, which would raise the cost of their products. They're more interested in meeting a price point, then making a camera with the best quality image possible.
  • 0

#5 David Mullen ASC

David Mullen ASC
  • Sustaining Members
  • 18789 posts
  • Cinematographer
  • Los Angeles

Posted 01 September 2015 - 06:14 PM

There has always been some concern about recording log gamma in 8-bit codecs, if you try and cram too many stops of luminance into 8-bit, you may get banding artifacts on gradients.  Now in reality it hasn't been as bad as some people thought, plus the advantages of having more stops of information seem to outweigh the banding issues, but it is still an issue until you move up to 10-bit.

 

As Tyler mentioned, it's a no-brainer that they offer 8-bit 4:2:0 Rec.709 recording since that's more or less what gets broadcast; it's just limiting in terms of color-correction and vfx work.  To add log gammas or even worse, raw recording, is probably a strain on the still camera's processor and recorder.


  • 0

#6 Phil Rhodes

Phil Rhodes
  • Sustaining Members
  • 11234 posts
  • Other

Posted 01 September 2015 - 07:12 PM

I'm sure I read a particularly brilliant article on this at one point. Ahem.

 

http://www.redsharkn...tyle-recordings


  • 0

#7 Tyler Purcell

Tyler Purcell
  • Sustaining Members
  • 2372 posts
  • Other
  • Los Angeles

Posted 01 September 2015 - 07:31 PM

It's still poorly organized Phil. ;)

Nice article BTW. :D
  • 0

#8 Carl Looper

Carl Looper
  • Basic Members
  • PipPipPipPip
  • 1367 posts
  • Digital Image Technician
  • Melbourne, Australia

Posted 11 September 2015 - 05:49 PM

I'm sure I read a particularly brilliant article on this at one point. Ahem.

 

http://www.redsharkn...tyle-recordings

 

There's an error in the article. It's mostly correct, but the original log curve approximation of how our eyes see (doubling/halving of light = approx same change in perceived brightness) is not nearly as inefficiently stored as the way the article suggests, because the way in which numbers are stored in a digital system (as digits) has the same non-linear relation to the value being represented, as perceived brightness has to the otherwise linear value of the light (such as lux or footcandle scale).

 

In simple terms it doesn't require storing 255 digits to represent a value of 255. In a base 2 system it requires only 8 digits, and in a base 10 system it requires only 3 digits.

 

Another example, using base 10:

 

Between the numbers 0 through 9, and the numbers 10 through 99, is a difference of only one digit for a difference in value of ten times.

Between the numbers 0 through 9, and the numbers 100 through 999, is a difference of only two digits, but for a difference in value of hundred times.

 

In other words the relationship between the number of digits and the value being represented is logarithmic.

 

The symmetry between a base 2 number system (binary) and a base 2 log scale representation of light such as camera stops (1 stop = doubling/halving of light) means they have the same efficiency.

 

So the increase in efficiency by applying a second log function on top of the first (to better approximate how we see) is not anywhere near as huge as suggested in the article.

 

Carl


Edited by Carl Looper, 11 September 2015 - 06:03 PM.

  • 0

#9 Carl Looper

Carl Looper
  • Basic Members
  • PipPipPipPip
  • 1367 posts
  • Digital Image Technician
  • Melbourne, Australia

Posted 12 September 2015 - 04:14 AM

I suspect I'm confusing the situation a little or indeed a lot. I've been misreading Phil's article

 

While the number of digits in modern number systems are logarithmic with respect to the value being represented, this is really quite irrelevant. What Phil is saying, is that the value a sensor stores is logarithmic with respect to photon count, but by way of explanation he will propose a sensor that would do it differently: that would not be logarithmic with respect to photon count.

 

He will call such a sensor "linear" in the sense that it is linear with respect to photon count. The stuff about doubling threw me. Whether one doubles both or multiplies both by a million it would be the same relationship. A linear one. Purely proportional. And where he otherwise has the term "exposure" along the x-axis this is not exposure in the traditional sense, ie. as already log scale values, be they camera stops (as Kodak has started to use more recently) or base 10 log scale values (as Kodak previously used). The values there would be photon count values, or footcandles, or lux.

 

And so the straight line marked "linear" in the graph is not a linear map between a log scale value for exposure and pixel value, but between photon count (etc), and pixel value. And the displayed log curve is just the traditional log mapping between photon count (lux, footcandle, etc) and exposure value (come pixel value). Not the newer log scale curves.

 

C


Edited by Carl Looper, 12 September 2015 - 04:19 AM.

  • 0

#10 Andy Zou

Andy Zou

    New

  • Basic Members
  • Pip
  • 7 posts
  • Director
  • New York

Posted 12 September 2015 - 02:35 PM

So I just read this article:

http://www.provideoc...-l-for-your-gh4

 

They discuss V-Log being significantly noisier than a WYSIWYG profile.

 

Why would that happen?  I assumed that noise was something that "happened" at the raw sensor level, before the information is put into a container.  


  • 0

#11 David Mullen ASC

David Mullen ASC
  • Sustaining Members
  • 18789 posts
  • Cinematographer
  • Los Angeles

Posted 12 September 2015 - 03:55 PM

Possibly the display gamma recordings have some noise reduction applied but what is more likely that in viewing a log image with its raised blacks they are just seeing noise that will be harder to see once the footage is shown with a proper display gamma.
  • 0

#12 Carl Looper

Carl Looper
  • Basic Members
  • PipPipPipPip
  • 1367 posts
  • Digital Image Technician
  • Melbourne, Australia

Posted 12 September 2015 - 04:12 PM

So I just read this article:

http://www.provideoc...-l-for-your-gh4

 

They discuss V-Log being significantly noisier than a WYSIWYG profile.

 

Why would that happen?  I assumed that noise was something that "happened" at the raw sensor level, before the information is put into a container.  

 

Your assumption is correct: noise happens at the raw sensor level, before the signal is digitised. 

 

But there is another form of noise (or pseudo-noise) that occurs during digitisation, called dithering, which is done intentionally - making it possible to squeeze more information into the same bit width. So it could be this.

 

It wouldn't be in the "V-Log" mapping as that would be a strictly mathematical function where there is no such thing as noise. Statistics becomes the appropriate framework in relation to noise.

 

C


Edited by Carl Looper, 12 September 2015 - 04:26 PM.

  • 0

#13 David Mullen ASC

David Mullen ASC
  • Sustaining Members
  • 18789 posts
  • Cinematographer
  • Los Angeles

Posted 12 September 2015 - 07:48 PM

Adam says in the article it was still noisy after color-correction with a display gamma so perhaps his explanation is correct, that with V-Log, the limits of an 8-bit 4:2:0 compressed codec are too much for the heavier grade needed to get from log to Rec.709.  Or it could be that V-Log wants you to expose for more highlight information but this ends up increasing shadow noise.


  • 0

#14 Carl Looper

Carl Looper
  • Basic Members
  • PipPipPipPip
  • 1367 posts
  • Digital Image Technician
  • Melbourne, Australia

Posted 13 September 2015 - 05:44 PM

The additional noise is probably just the result of seeing more of the original sensor signal (seeing extra stops) - where previously it had been suppressed or unused - ie. because such additional information was noisy.

 

So why include it? Well it depends on how you use it. With an inefficient log/gamma encoding the extra stops captured could have looked overt and ugly - so better to just suppress such altogether (average out the extra stops) in camera. With a better log/gamma encoding it becomes possible to defer such massaging to later, where one can either do what the camera was originally doing anyway, as you might want to do if delivering the image in the original log/gamma encoding format, or one can custom massage those extra stops by whatever other noise reduction system is appropriate for some other encoding method one is otherwise going to use

 

The effectiveness of noise reduction depends on what the output format is going to be. Until you know what that is going to be you ideally want all the information you can get, including noise, because the relation between signal and noise is graduated - it's not something that occurs at some clear threshold. It takes knowledge of both the input signal and the output format to fine tune the transform between them - and if targeting multiple output formats (as the digital age provides us) this knowledge is only available in post (at the point of completion).

 

C


Edited by Carl Looper, 13 September 2015 - 05:59 PM.

  • -1


Broadcast Solutions Inc

CineLab

Paralinx LLC

CineTape

Abel Cine

Rig Wheels Passport

Zylight

rebotnix Technologies

Glidecam

Visual Products

Willys Widgets

Tai Audio

Ritter Battery

The Slider

Metropolis Post

Aerial Filmworks

Pro 8mm

Technodolly

Paralinx LLC

Aerial Filmworks

Rig Wheels Passport

Zylight

Metropolis Post

CineTape

CineLab

Glidecam

Broadcast Solutions Inc

rebotnix Technologies

Abel Cine

The Slider

Willys Widgets

Visual Products

Ritter Battery

Technodolly

Tai Audio

Pro 8mm