# Learning to Read and Use Histograms



## smcf (Dec 24, 2014)

Hi Everyone!

First post!!!! Yay!!!!

I come here with no formal education in photography. What I've learned, I learned through reading camera manuals, occasional magazine article, blog post, youtube video, etc. I've been using LR for approx a year or so and until now have pretty much ignored the histogram. I'm not really sure what it is. I'm not even sure I could really explain to someone what it represents other than in the most vaguest sense. 
I'm wondering if someone can recommend a primer to get started understanding and interpreting effectively the histogram? I might even need to take a step back and learn some more fundamental ideas. For example, I understand that my camera captures 12 megapixel images but I can't say I fully comprehend what is stored in each pixel. Broadly I understand the camera's sensor is capturing reflectance values and storing them at some bit depth. Any online sources, books, etc that could be recommended would be greatly appreciated. Cheers, and Happy Holidays!!!!

S.


----------



## Tony Jay (Dec 24, 2014)

Welcome!

The histogram is actually a very interesting and useful tool in digital photography - both for what it can tell you and for what it cannot tell you.
The histogram is a graphical representation of the distribution of tonal values in an image.
Variations in how a histogram is represented can allow one to see the tonal distribution in terms of the primary colours involved in capture - these are red, green, and blue. Each colour is referred to as a channel.

What information can one get from the histogram?
First of all the histogram, per se, is not a good indicator of whether an image is appropriately exposed or not.
However, when one takes into consideration the scene being captured and the goals one has in mind for the image shot then it is possible to use the histogram to considerable constructive effect.
Secondly, the in-camera histogram of an image that has just been captured has certain limitations. It is a representation of a JPEG derivative and not the original raw image. Because the JPEG image is, by definition, an 8-bit image and the original raw image a 12-, 14-, or 16-bit image depending on the camera being used the tonal levels that can be represented by the the JPEG histogram are limited to 256 while a 16-bit raw file actually contains up to 65 000 (and change) tonal levels.
One of the matters arising out of this is that an in-camera histogram may erroneously represent an image with the highlights blown, when, in fact, the original raw image has no blown highlights at all. 
Pleas to manufacturers to at least allow a raw-based histogram in-camera as an option have so far fallen on deaf ears.
Nonetheless, it is possible with practice, to learn exactly how much more highlight headroom one has with a raw image despite the in-camera histogram showing blown highlights. With my Canon 5D mark III this is about 1 1/2 stops at least.

I will continue this post later today - for the moment at least I need to go to work!

Tony Jay


----------



## smcf (Dec 24, 2014)

Tony many thanks. Sorry to hear you had to go to work, today of all days!


----------



## Tony Jay (Dec 25, 2014)

Continuing where we left off:

Whether one shoots JPEG's or raw one, as a general rule, does not want to blow the highlights. Sometimes though things like specular highlights can be left to blow. Sometimes, when shooting high dynamic range scenes it can be difficult to get the exposure correct when shooting JPEG's since placing the mid-zones in the middle of the histogram might well and truly blow the highlights. However, shooting the same scene using a raw format all one does is just make sure that the highlights are not blown (see above) and then use one's favourite raw converter to manipulate the tones to restore the image to roughly what it looked like from memory. Because there is much more tonal and colour information in the raw file even very extensive editing tends not to break the image. JPEG files tend not to tolerate much in the way of image editing even using a parametric image editor such as Lightroom.

The technique of shooting raw captures and pushing the exposure so the subsequent histogram tends to the right but does not blow goes by the mnemonic ETTR - *E*xpose-*T*o-*T*he-*R*ight. The rationale for this approach is all in what is called the signal-to-noise ratio. One shoots at base ISO.
Simply put every sensor in every digital camera has a noise signal with a specific threshold. It varies from sensor to sensor. If the light signal received by the photosensitive sites on the sensor, also called a sensel, which stands for *sens*or *el*ement, is close to the noise threshold for that sensor then that image will be perceived as noisy. If the exposure is increased (by altering either shutter speed or aperture - not ISO) then the signal-to-noise ratio is increased and the image becomes less noisy.
In practical terms it means placing the brightest parts of the image across to the right on the histogram, but without blowing those highlights.
In a high dynamic range scene the end result might place the midzones almost exactly where they might sit on the histogram if one chose not to specifically shoot with an ETTR bias. However in a scene with a more limited dynamic range if shot with an ETTR bias the midzones will end up considerably to the right. One then proceeds to fix that issue in Lightroom (or other raw converter) however those tones that end up becoming the shadow tones will have little to no noise.

It is true that some late model cameras sporting, in particular, sensors from Sony are challenging the necessity for ETTR however I can say, from personal experience shooting with a Sony A7r that is still useful for shooting very high dynamic range scenes where even the prodigious dynamic range of that sensor is challenged.

In low light conditions shooting ETTR-type images is only possible using a tripod or some other camera support since one is shooting at base ISO and long(er) shutter speeds are necessary that will exclude hand-holding.

Just to stress - ETTR techniques are only for shooting raw images. If one tries this shooting JPEG's then only an overexposed mess will result that is difficult to impossible to correct even with a parametric image editor.

Since you appear to be shooting raw lets loop around and cover some ground again in more detail.
I mentioned the difference between the 'real' histogram of a raw image versus the in-camera JPEG derived histogram that may be incapable of representing all the tonal information that actually resides in the raw image and so that histogram may subsequently erroneously represent the real situation.
However, when one imports that same image into Lightroom (or opens it into some other raw converter) the histogram that one views in that application will be a much more accurate representation.
In Lightroom when one applies tonal edits to the image the histogram will update in real time showing the effects of the edits.
As an example, if one increases exposure then the histogram will move to the right.

If you want a good primer covering a lot of background information on sensors and how they capture data and what that data is then try "The Digital Negative" by Jeff Schewe.

Tony Jay


----------



## smcf (Dec 25, 2014)

Tony, Many thanks again.
Perhaps taking a bit of a step back, to start I've been trying to visualize for myself what is actually stored in the RAW file (beyond header and meta data information). I get the idea of a grid made up of pixels, each pixel storing reflectance values picked up by the sensor. In my case the RAW files are actually 4256x2832. In the pic below I've tried to visualize in a simple way what that file might look like. In this case a simple grid of pixels with each pixel storing RGB and black and white reflective values (0-255 for each colour channel). From here I'd assume a histogram would then for each channel count the # of pixels with a channel value between certain predefined ranges (e.g. Blacks, Shadows, Exposure, Highlights, Whites). Am I on the right track? Thanks again. I'll definitely read your post carefully and search out the title you've listed. Cheers,
S.


----------



## DaveS (Dec 25, 2014)

Generally a raw file doesn't contain locations with each of red, green and blue from each sensor element.  Rather each sensor location stores one colour, in a bayer pattern.  (See: http://en.wikipedia.org/wiki/Bayer_filter for more information.)    Raw processors such as lightroom "de-mosaic" the raw data into rgb pixels.


----------



## smcf (Dec 25, 2014)

Appreciate the link. I had read earlier today but found it confusing. I'm looking to start with some broad concepts to build on. My loose understanding of raw files is that they are essentially derivative TIFF files. With this in mind I was thinking that each pixel in the raw file had multiple reflective values stored with it and the colour model configured via the camera settings defined how the data are stored in the output file. I suspect many may be rolling their eyes at my line of questioning. Apologies in advance if this has been covered before or if people feel this is not particularly relevant to LR. I'm asking because when I manipulate the controls in LR I want to have a better appreciation of what's actually going on. I've been tweaking photos for 1+ years now (with some good results) but I'd like to get a deeper appreciation of what I'm doing. It's sort of like driving a car. I know the brake and accelerator pedals but don't have a full understanding of the ICE! Cheers,
S.


----------



## DaveS (Dec 25, 2014)

It's sort of the other way around, tiffs would be the demosaiced derivative of a raw file.    If you break the raw array into a bunch of 2x2 sensor elements, there are 2 green ones, 1 red one and 1 blue one.      (Actually, they have no colour at all, but are just a brightness, but each element is covered by either a red, green, or blue filter to filter out all but that colour of light). Each element, consists of exactly one colour intensity value.   The raw file has no colour space at all defined.  it's not until it's converted to an rgb image (which the rgb in camera image is one) that it acquires a colour model or space.

When LR reads this blob of sensors, it then interpolates  to create pixels that are combinations of the various 2x2 bayer elements (the green ones (two as people are more sensitive to green) and a red and blue one).  The amount of each rgb value dependant on the elements that are around the "pixel" that is created.   It's not entirely accurate, but imagine an rgb pixel at the center of the 2x2 bayer sensor element array.  

The picture you have above, would be a good example of what the tiff file would look like once produced by lightroom from a raw image.


----------



## smcf (Dec 25, 2014)

Cheers, just for reference, here's the passage from Wikipedia that I'd read that suggests raw files are effectively TIFF files:

Many raw file formats, including IIQ (Phase One), 3FR (Hasselblad), DCR, K25, KDC (Kodak), CR2 (Canon), ERF (Epson), MEF (Mamiya), MOS (Leaf), NEF (Nikon), ORF (Olympus), PEF (Pentax), RW2 (Panasonic) and ARW, SRF, SR2 (Sony), are based on the TIFF file format.[4] These files may deviate from the TIFF standard in a number of ways, including the use of a non-standard file header, the inclusion of additional image tags and the encryption of some of the tagged data. ( http://en.wikipedia.org/wiki/Raw_image_format )​
Looking forward to continued dialogue and learning. Cheers,
S.


----------



## Tony Jay (Dec 25, 2014)

In fact that is not so.
A raw image only stores information as shades of grey ie tonal information only.
Why this occurs is relatively simple.
In front of the sensor is a Bayer array which is a filter.
This filter consists of repeating two-by-two patterns like so:







And the result is that individual sensels receive light filtered through either red, blue, or green filters like this:






When a raw file is converted a key process is what is called demosaicing.
This process takes all those grey tones and converts them to colours.
It is a complex algorithm however basically each pixel takes information from surrounding pixels to produce an RGB colour value since it is known, for each and every pixel, what colour light was transmitted by the filter..
Demosaicing is only part of the conversion process however, but it does explain how a grey-tone image can be converted to a colour image.
The contrast of these demosaiced images is very low however a gamma curve is applied that changes the contrast to something that results in an image that is a little more believable and much more workable.

Luminance information is intrinsic to the RGB data.
Three values are recorded - one for each colour channel.
so 255,0,0 is a very bright red.
0,255,0 is a very bright green.
0,0,0 is black and 255,255,255 is white.
125,125,125 is grey, slightly darker than middle grey.

More information:
A raw file has a colour gamut but it does not have a colourspace until one is assigned to it.
This does not happen in the camera but only in the raw conversion.
Many people erroneously assume that a raw file is assigned sRGB or AdobeRGb in-camera because the camera wants you to choose a colourspace, however colourspaces are only assigned to JPEG files in camera.

If you do your research you may come across new sensors that do not have Bayer filters and actually record all the RGB information per pixel de novo at the sensor level without the need for demosaicing. However cameras using this technology are relatively rare at this time.

Tony Jay


----------



## DaveS (Dec 25, 2014)

A very nice technical explanation of what I was attempting to portray in simpler terms.   :nod:


----------



## smcf (Jan 11, 2015)

Thanks Guys, I've found these posts quite useful as I keep coming back to them as I continue to evolve my understanding of the topic. Cheers.


----------



## Rob_Cullen (Jan 12, 2015)

This is where I started- simple was easy for me.

1. Imagine your image (start with monochrome) is on the wall and is made from many small tiles.
2. Now Imagine the 'tiles' fall off the wall into neat little piles sorted by their tone.
3. The 'Black tiles'  will be in a pile on the left of the histogram, 'Grey tiles' in the centre, and 'White tiles' on the right.
4. When there are enough 'tones' black to white, the histogram takes on the continuous line graph appearance.
5. There will be a separate graph for each colour- Red, Green, Blue. In the colour histogram the graphs overlap.






A "Mono" histogram


----------



## Victoria Bampton (Jan 12, 2015)

Great description I-See-Light, love it.


----------



## Conrad Chavez (Jan 12, 2015)

smcf said:


> Cheers, just for reference, here's the passage from Wikipedia that I'd read that suggests raw files are effectively TIFF files:Many raw file formats, including IIQ (Phase One), 3FR (Hasselblad), DCR, K25, KDC (Kodak), CR2 (Canon), ERF (Epson), MEF (Mamiya), MOS (Leaf), NEF (Nikon), ORF (Olympus), PEF (Pentax), RW2 (Panasonic) and ARW, SRF, SR2 (Sony), are based on the TIFF file format.[4] These files may deviate from the TIFF standard in a number of ways, including the use of a non-standard file header, the inclusion of additional image tags and the encryption of some of the tagged data. ( http://en.wikipedia.org/wiki/Raw_image_format )​


To maybe clear this up a little, saying "raw files are derived from TIFF files" or "raw files are TIFF files" is a little imprecise. A more precise statement would be "Raw sensor data is stored in a format derived from TIFF, and during raw editing the image is demosaiced into an RGB or CMYK image derived from the raw sensor data." 

TIFF is a container format, which is why it's used for so many types of images like all of the digital camera raw files above, plus grayscale, RGB, CMYK, LAB, flat, layered, etc. That is the only reason you can say that "raw files are TIFF files." It's just a technicality and doesn't tell you that raw data is fundamentally different than an RGB image, both of which can be stored as TIFF. While raw files may use a form of TIFF, they're not going to be recognized by most applications that claim to import TIFF because most of those apps only recognize images demosaiced into something like RGB.

Here's an analogy: You open your front door and a delivery man gives you two shipping boxes that look about the same. One box contains baking ingredients for making cookies, and the other box is full of baked cookies. They both came in boxes, but no one would say that they are the same thing. Also, no one would say that the ingredients are derived from the finished cookies even though the ingredients came in a more recent version of the box.

That raw files are stored in a variation of TIFF is just a bit of trivia that isn't very useful. What's important is knowing that you can't do very much with a raw file until it's translated into a derivative image (RGB/CMYK TIFF, JPEG, etc) that is usable by your web site, your printer driver, or your downstream software (page layout, video editing, image editing).


----------



## Tony Jay (Jan 13, 2015)

Conrad Chavez said:


> To maybe clear this up a little, saying "raw files are derived from TIFF files" or "raw files are TIFF files" is a little imprecise. A more precise statement would be "Raw sensor data is stored in a format derived from TIFF, and during raw editing the image is demosaiced into an RGB or CMYK image derived from the raw sensor data."
> 
> TIFF is a container format, which is why it's used for so many types of images like all of the digital camera raw files above, plus grayscale, RGB, CMYK, LAB, flat, layered, etc. That is the only reason you can say that "raw files are TIFF files." It's just a technicality and doesn't tell you that raw data is fundamentally different than an RGB image, both of which can be stored as TIFF. While raw files may use a form of TIFF, they're not going to be recognized by most applications that claim to import TIFF because most of those apps only recognize images demosaiced into something like RGB.
> 
> ...


Nice summary yet very precisely put!

Tony Jay


----------



## smcf (Jan 13, 2015)

So here's a question  
My camera outputs NEF raw files to compact flash. If I copy those files to my computer (in my case OS X). The finder window is able to render a thumbnail representation of the NEF raw file and the preview app is able to "view" the NEF image. What's going on there? Are the Finder/Preview apps demosaicing the NEF files (via digital camera RAW support) or is there an embedded image/thumbnail (perhaps jpg) demosaiced already in the file? Just curious.


----------



## Jim Wilde (Jan 13, 2015)

It's the latter. The camera does its own raw conversion to create a demosaiced RGB file with all in-camera settings applied. That file, in jpeg format, is embedded in the raw file (and is also what you see on the camera LCD after you've taken the shot), and it is that embedded file which is extracted for you to see in Finder, Preview, etc.


----------



## smcf (Jan 13, 2015)

I tripped across this web page which includes a description of raw files this morning and found it useful so sharing here. I also found at least one or two bits of info in it surprising (e.g. "_Some companies, Nikon and Kodak specifically, use a slightly lossy compression algorithm when saving raw files"_). 

http://www.luminous-landscape.com/tutorials/understanding-series/u-raw-files.shtml


----------



## Denis de Gannes (Jan 13, 2015)

The article is pretty dated but is still relevant.


----------



## smcf (Feb 4, 2015)

Just adding something to the discussion. I was watching a Lynda.com tutorial this morning on scanning called Scanning Techniques for Photography, Art & Design. It's a very good overview of scanning. It covers a lot of ground and while I'm an old hat at scanning I still found it interesting. But of interest here is the instructor's discussion of histograms (section called Evaluating and correcting images with histograms). Very simple and succinct treatment of the subject. If you're a Lynda subscriber like I am (I'm testing its value for a year) and want a simple explanation of histograms this training video is really really good. Here's a quick screen cap from the video where the instructor walks through each of these images and discusses why the histogram looks the way it does.


----------

