The Wayback Machine - https://web.archive.org/web/20080228155824/http://photo.net:80/learn/raw/

Home > Learn > RAW, JPEG and TIFF

RAW, JPEG and TIFF

by Bob Atkins; created 2004

There seems to be a lot of confusion among some new digital camera owners about exactly what the difference is between RAW, JPEG and TIFF files. This article is intended to be a very basic guide to these file types and how they are related in a typical digital camera.

First some basics

The digital sensor in the majority of digital cameras is what is known as a BAYER PATTERN sensor. This relates to the arrangement of red, green and blue sensitive areas. A typical sensor looks like this:

bayersensor2.jpg (27066 bytes)

Each pixel in the sensor responds to either red, green or blue light and there are 2 green sensitive pixels for each red and blue pixel. There are more green pixels because the eye is more sensitive to green, so the green channel is the most important. The sensor measures the intensity of light falling on it. The green pixels measure the green light, the red the red and the blue the blue. The readout form the sensor is of the form color:intensity for each individual pixel, where color can be red, green or blue and intensity runs from 0 to 4095 (for a 12-bit sensor)

A conventional digital image has pixels which can be red, green, blue of any one of millions of other colors, so to generate such an image from the data output by the sensor, a significant amount of signal processing is required. This processing is called Bayer interpolation because it must interpolate (i.e. calculate) what the color of each pixel should be. The color and intensity of each pixel is calculated based on the relative strengths of the red, green and blue channel data from all the neighboring pixels. Each pixel in the converted image now has three parameters: red:intensity, blue:intensity and green:intensity. In the end the calculated image looks something like this:

image.gif (11881 bytes)

RAW data

RAW data (which Nikon call NEF data) is the output from each of the original red, green and blue sensitive pixels of the image sensor, after being read out of the array by the array electronics and passing through an analog to digital converter. The readout electronics collect and amplify the sensor data and it's at this point that "ISO" (relative sensor speed) is set. If readout is done with little amplification, that corresponds to a low ISO (say ISO 100), while if the data is read out with a lot of amplification, that corresponds to a high ISO setting (say ISO 3200). As far as I know, RAW isn't an acronym, it doesn't stand for anything, it just means raw, unprocessed, data.

Now one of two things can be done with the RAW data. It can be stored on the memory card, or it can be further processed to yield a JPEG image. The diagram below shows the processes involved:

Flowchart1.gif (8759 bytes)

If the data is stored as a JPEG file, it goes through the Bayer interpolation, is modified by in camera set parameters such as white balance, saturation, sharpness, contrast etc, is subject to JPEG compression and then stored. The advantage of saving JPEG data is that the file size is smaller and the file can be directly read by many programs or even sent directly to a printer. The disadvantage is that there is a quality loss, the amount of loss depending on how much compression is used. The more compression, the smaller the file but the lower the image quality. Lightly compressed JPEG files can save a significant amount of space and lose very little quality. For more on JPEG compression see http://www.photo.net/learn/jpeg/index.html

RAW to JPEG or TIFF conversion

If you save the RAW data, you can then convert it to a viewable JPEG or TIFF file at a later time on a PC. The process is shown in the diagram below:

Flowchart2.gif (7658 bytes)

You'll see this is pretty similar to the first diagram, except now you're doing all the processing on a PC rather than in the camera. Since it's on a PC you can now pick whatever white balance, contrast, saturation, sharpness etc. you want. So here's the first advantage of saving RAW data. You can change many of the shooting parameters AFTER exposure. You can't change the exposure (obviously) and you can't change the ISO, but you can change many other parameters.

A second advantage of shooting a RAW file is that you can also perform the conversion to an 8-bit or 16-bit TIFF file. TIFF files are larger than JPEG files, but they retain the full quality of the image. They can be compressed or uncompressed, but the compression scheme is lossless, meaning that although the file gets a little smaller, no information is lost. This is a tricky concept for some people, but here's a simple example of lossless compression. Take this string of digits:

14745296533333659762888888356789

Is there a way to store this that doesn't lose any digits, but takes less space? The answer is yes. One way would be as follows

1474529653[5]6597628[6]356789

Here the string 33333 has been replaced by 3[5] - meaning a string of 5 3s, and the string 888888 has been replaced by 8[6] - meaning a string of 6 8s. You've stored the same exact data, but the "compressed" version takes up less space. This is similar (but not identical) to the way lossless TIFF compression is done.

I said above that the data could be stored as an 8 or 16-bit TIFF file. RAW data from most high end digital camera contains 12 bit data, which means that there can be 4096 different intensity levels for each pixel. In an 8-bit file (such as a JPEG), each pixel can have one of 256 different intensity levels. Actually 256 levels is enough, and all printing is done at the 8 bit level, so you might ask what the point is of having 12 bit data. The answer is that it allows you to perform a greater range of manipulation to the image without degrading the quality. You can adjust curves and levels to a greater extent, then convert back to 8-bit data for printing. If you want to access all 12 bits of the original RAW file, you can convert to a 16-bit TIFF file. Why not a 12-bit TIFF file? Because there's no such thing! Actually what you do is put the 12 bit data in a 16 bit container. It's a bit like putting a quart of liquid in a gallon jug, you get to keep all the liquid but you have some free space. Putting the 12 bit data in a 8 bit file is like pouring that quart of liquid into a pint container. It won't all fit so you have to throw some away.

When to shoot RAW, when to shoot JPEG?

The main reason to shoot JPEG is that you get more shots on a memory card and it's faster, both in camera and afterwards. If you shoot RAW files you have to then convert them to TIFF or JPEG on a PC before you can view or print them. If you have hundreds of images, this can take some time. If you know you have the correct exposure and white balance as well as the optimum camera set parameters, then a high quality JPEG will give you a print just as good as one from a converted RAW file, so you may as well shoot JPEG.

You shoot RAW when you expect to have to do some post exposure processing. If you're not sure about exposure or white balance, or if you want to maintain the maximum possible allowable post exposure processing, then you'll want to shoot RAW files, convert to 16-bit TIFF, do all your processing, then convert to 8-bit files for printing. You lose nothing by shooting RAW except for time and the number of images you can fit on a memory card.

Note that some cameras can store a JPEG image along with the RAW file. This is the best of both worlds, you have a JPEG image which you can quickly extract from the file, but you also have the RAW data which you can later convert and process if theres a problem with the JPEG. The disadvantage is, of course, that this takes up even more storage space. Many cameras also store a small "thumbnail" along with the RAW file which can be read and displayed quickly without having to do a full RAW conversion just to see what's in the file.


All material © Copyright 2004, Robert M. Atkins. Visit Bob Atkins' website at www.bobatkins.com

 

 

Readers' Comments


Add a comment



Bas Scheffersphoto.net patron prolific poster, June 09, 2004; 08:39 A.M.

Nice write up, Bob. I would just like to add that in my experience, unless you do no post-processing at all, RAW can be a lot faster. Every image, however well exposed with the correct white balance, needs some processing, if only setting the black and white points.

The speed difference is trivial if you shoot loads of images and select 2 or 3 for printing. But if you want to put about a hundred out of 600 images shot on a holiday on your website, selecting and opening them, navigating Photoshop's endless menus and saving again is a real time waster.

One option is a macro that does "auto color" on a whole directory of images, but I prefer a more manual aproach. My software of choice for this is C1 DSLR. The workflow is fantastic and the LE version is well worth the $99.

Using C1, the process described above takes me a couple of hours on a sunday afternoon and as a bonus it lets me be more "sloppy" when shooting. White balance settings is something I never worry about in the field and a stop under exposure is no real loss thanks to the 12 bits of data.

Serge Boucher, June 09, 2004; 11:11 A.M.

As an aside, some cameras have a TIFF mode. (Well, the Sony F-707 does. I'm assuming it's not the only one.) It works exactly like JPEG mode but theoretically gives better quality, since tiff is lossless.

(In the real world I've found it pretty useless since the improvement in quality from a jpeg is virtually invisible.)

Very comprehensive article, as always !

Serge Boucher

Antonio Giacomophoto.net patron, June 09, 2004; 11:45 A.M.

A very good introduction Bob, especially for non image procesing specialists. If you want to go into it deeper, try this link: http://www-ise.stanford.edu/~tingchen/

Antonio

Bob Atkinsphoto.net admin photo.net hero photo.net patron, June 09, 2004; 12:44 P.M.

Though a few cameras do have a TIFF mode, it's generally not very useful as has been pointed out. The problem is the size of a TIFF file. For a 6MP camera, an uncompressed TIFF file is 18MB in the 8-bit mode and 32MB in the 16-bit mode. This is huge compared to the 2-3MB of a very lightly compressed JPEG or the ~6MB of a RAW file. BTW RAW files can (and usually do) use lossless compression techniques.

TIFF does eliminate JPEG artifacts of course, but for light compression JPEG artifacts are rarely, if ever, a problem.

Jon Austinprolific poster, June 09, 2004; 03:11 P.M.

Bob: What's your opinion about the relative quality of doing auto or manual color balance* and manually setting the white point in a JPEG in Photoshop (et al), as compared to manipulating a RAW image to the same end in a RAW processor? (*I apologize; I'm not at my Photoshop machine, and don't recall the exact terminology it uses for these functions.) I've been scanning and digitizing a lot of decades-old prints lately, and my first image adjustment step is using this automatic color balance step, to (for example) recast the skies blue that have turned green in the aged prints.

Re: In camera TIFFs, my last digicam had this capability. It was a 3MP model, and the TIFF files were ~ 10x larger than the best-quality JPEGs of comparable resolution. Not only was there no discernable difference between a TIFF and a highest-quality JPEG image of the same subject, the time it took the camera to write the TIFF file to the CF card seemed like an eternity!

One comment on the article: "Actually what you do is put the 12 bit data in a 16 bit container. It's a bit like putting a quart of liquid in a gallon jug, you get to keep all the liquid but you have some free space."

In fact, it's more like putting 3 quarts of liquid in a gallon jug. (One gallon = four quarts, therefore, 12 bits in a 16-bit container <==> 3 quarts in a gallon jug.) I only mention this for those attempting to understand both the quantitative as well as the qualitative aspects of this analogy. Likewise, attempting to put a 12-bit file into an 8-bit container would be like attempting to put 3 quarts into a half-gallon jug, just for consistency of volumes.

(Man, the metric system is so much simpler...)

Bob Atkinsphoto.net admin photo.net hero photo.net patron, June 09, 2004; 03:51 P.M.

You should get significantly better results by color balancing a RAW file than by color balancing a JPEG. The reason is that the RAW file has all of the original color information, while the JPEG file has thrown away some of it.

To take an extreme example. Suppose you shot indoors under tungsten light but you had the camera set to daylight white balance. If you write a JPEG you'll be throwing away a lot of the color information information and you'll end up with a red/yellow looking image. You can't get that lost info back. However in the RAW file, all the color data for all the channels is there and you can recover the image.

William Nicholls, June 09, 2004; 08:12 P.M.

"If you know you have the correct exposure and white balance..."

Big "if". I think it's fair to say that most jpeg shooters are (rightly) looking for convenience, therefore few could be confident that they got everything dialed in at the time of shooting. Jpeg may not cost you much quality on a given image, but overall you'll produce better files using a raw workflow. A raw conversion application running on a powerful PC has big advantages and a lot more flexibility over the dedicated jpeg processor and limited settings in a digital camera.

And raw converters continually improve and deliver better results. I'm able to extract better images and hold more highlight detail from my Fuji .raf files today than I could a few months ago. The camera's jpeg processor can't even match the old raw conversions and it's a fixture of the camera. Shooting raw files is a hedge on the future, just like shooting negs would let you use better papers and chemistry as they came out (not to mention a hedge against your own evolving sense of good color and image characteristics). It's certainly a hedge against a poor white balance setting, too much sharpening, or oversaturation that you may just *think* you have dialed-in.

Shoot jpeg for convenience and speed and don't delude yourself about the compromises jpeg entails.

Jon Austinprolific poster, June 09, 2004; 08:59 P.M.

"Suppose you shot indoors under tungsten light but you had the camera set to daylight white balance. If you write a JPEG you'll be throwing away a lot of the color information information and you'll end up with a red/yellow looking image."

Makes sense. I shoot indoors in low tungsten light, and even with AWB or Tungsten WB, my shots still look pretty "orangey."

R. Falise, June 10, 2004; 07:43 A.M.

Great article - explains the process in simple terms. Just a couple of comments.

If you are shooting Nikon gear and use Nikon Capture Editor you can in fact change exposure settings . Capture has an exposure comp capability, so if you underexpose a bit you can bump the exposure in Capture. I believe Phase One DSLR can also do this, but I haven't tested the Mac version yet and I recently switched to Mac platforms for post processing.

I think there is too much made about the whole JPEG v RAW topic. On a recent shoot I accidently forgot to change my D100 settings and shot several landscapes in JPEG. Sure enough a client wanted a 16 x 20 of one of them. Starting with a JPEG file and using proper processing methods in Photoshop (nothing extreme is the key) the print came back tack sharp with beautiful color. It seems that shooting JPEG in daylight conditions yields beautiful, large prints from a D100. Indoor or difficult lighting is an entirely different matter.

Bottom line, I use both. If I am shooting landscapes or indoor stuff I generally shoot RAW (NEF) and use a combination of Capture and PS for post processing. If I am doing a client shoot (outdoor, natural light, candids) I shoot JPEG and hammer away.

No right or wrong here - just understand your gear and be adaptive. the most important goal is to get the shot. Worry about RAW v JPEG later.

Bob

Yaron Kidronprolific poster, June 10, 2004; 12:07 P.M.

Just a small addition on my side: TIFF as you know it, is not a well-defined format; it's a container, meta-format, very much a like AVI for video. A proper TIFF file MAY have a Jpeg compressor attached to it, and will behave (compression-like) very much like a stand-alone Jpeg file. Most TIFF implementations use LZH or other lossless algorithms, but there is no guarantee for that.

While not yet embraced, we may be soon see cameras supporting the Jpeg2000 format (which is long overdue, this standard has been around for a long time now), that allows 16bit per channel (compared to 8bit per channel for normal Jpeg).

Jon Austinprolific poster, June 10, 2004; 03:15 P.M.

Response to Yaron Kidron: I, too, am still waiting for camera manufacturers to incorporate JPEG2000 as one of the available image formats, since it offers lossless compression. I would also hope that JPEG2000 capability could be added to existing bodies via a firmware upgrade.

In the meantime, I'm seriously considering converting all my JPEG images to a different, lossless file format. PNG, anyone?

Bob Atkinsphoto.net admin photo.net hero photo.net patron, June 10, 2004; 04:22 P.M.

There's no point in converting JPEGs to a lossless format. The data is already gone and the compression artifacts are part of the image. You don't get data back or eliminate artifacts by converting from JPEG to TIFF, PNG or any other format using lossless compression.

Keith Mukai, June 10, 2004; 05:58 P.M.

"Actually what you do is put the 12 bit data in a 16 bit container. It's a bit like putting a quart of liquid in a gallon jug, you get to keep all the liquid but you have some free space."

This is inaccurate. In fact nearly the opposite. This is going to be long, but I think worthwhile:

The 12-bit Bayer sensor data is transformed and mapped into a 16-bit RGB space when the gamma curve is applied. This mapping inherently defines what is considered "max" and "min" intensity for the given transformation - and by definition (and if appropriate highlight/shadow clipping points are specified) values will indeed populate the range from 0 to 65535 in each of the three R,G,B channels.

This article implies that 12-bit Bayer matrix data is equivalent to 12-bit RGB data, meaning that (4095,4095,4095) is the max value attainable in a 16-bit RGB. If that were true, full 16-bit white (65535,65535,65535) would never exist in our converted 16-bit RGB file.

It's also tempting to think that 12-bit values just get multiplied by 2^4 to fit into a 16-bit space (e.g. 0*16 == 0, 1*16 == 16, 4095*16 == 65520).

But the real missing piece to the puzzle is that up to nine pixels from the sensor are used to calculate the value of each individual pixel in the final RGB image.

Combining nine 1-channel 12-bit Bayer pixels into a single 3-channel 16-bit RGB pixel results in *many more* possible values than can be expressed in 16-bit RGB.

This is where the extended dynamic range or "exposure latitude" of RAW comes from. It's not that you're given a quart to put into a gallon container, rather you're given *many gallons* - more than you can actually carry. Unfortunately the analogy breaks down when viewed this way.

A better analogy might be a rubber band. Your rubber band is, say, four inches long. An 8-bit image forces you to trim the rubber band to half an inch. A 16-bit image trims the rubber band to one inch. But *where* you make your trim matters (take from the left side, the right side, the middle). Also you can choose to stretch your rubber band before you trim it. Take a section that was only 1/4 of an inch and stretch it to fill the 1 inch that comprises your 16-bit image. And notice that even when stretched there are no gaps in the rubber band.

Kind of a sloppy, kludgy analogy, but I think you get the idea. Except for this one detail, this is a great article and I appreciate you taking the time to write it.

Jon Austinprolific poster, June 10, 2004; 06:35 P.M.

Bob: Everything you've stated about not getting back lost data and not eliminating the compression artifacts, when converting a JPEG to a lossless format, is true.

However, there is value (to me, at least) in converting JPEGs to a lossless format: once I've captured my large/fine JPEGs, I usually end up editing them in stages -- sometimes weeks (or months!) between sessions. A lossless format will enable me to tweak at will, without having to go back to the original file and start over each time, in order to avoid further loss.

I'm leaning towards JPEG2000 (JP2) as a lossless, but more space-efficent format than TIF or PSD.

Walter Schroeder, June 10, 2004; 06:53 P.M.

"If you are shooting Nikon gear and use Nikon Capture Editor you can in fact change exposure settings . Capture has an exposure comp capability, so if you underexpose a bit you can bump the exposure in Capture."

Bob (Falise) if you think that you can change false exposure after the image has been taken u missed the point - you can N O T . Bob (Atkins) tried to explain just that above. You are not alone being mislead by the "convinient slider" in the RAW converter user interface where you can "compensate" exposure, i see it in many comments.

All you can do there is just to shift the histogram in a convinient way. its just another way and place to do the same thing that you could do just as well after importing the image into PS (or other program). The only difference to do it here or after the import is in the case when you import the image in 8-bit mode (which is only called for in case of poor computing power). Except for convinience I recommend to leave this slider alone and import in 16bit mode. Do all adjustments there and finally convert to 8-bit mode to save disk space. (I keep a copy of the original for all "valuable" images). Sorry Bob - it just seemed such a nice feature of the optional software :-) but if the image is poorly exposed it can only be optimized (and this is better done with the full recorded information) but the wrong exposure can not be corrected - the information is just not recorded - no matter what the file format is.

Cheers Walter

Bob Atkinsphoto.net admin photo.net hero photo.net patron, June 11, 2004; 01:35 P.M.

"Combining nine 1-channel 12-bit Bayer pixels into a single 3-channel 16-bit RGB pixel results in *many more* possible values than can be expressed in 16-bit RGB. ".

I don't believe this is true. I agree that multiple Bayer cells (exactly how many on how they are used depends on the conversion algorithm chosen) make contributions to each pixel in the final image, but this is synthesised data and cannot contain more information than was originally present. You can certainly sythesise 16, or even 32 or 48 bit data from the Bayer pattern, and you could not only create red, green and blue channels, but as many color channels as you wanted, but they would still contain no more information than was originally present.

The Bayer data contains intensity levels from 0 to 4095 for each channnel. Take the case of a pure grey midtone, where each channel in the sensor has value of 2047 out of 4095. When you do the Bayer interpolation you get R, G and B channels each with a value of 2047 out of 4095. 12 bit data. You can't synthesise true 16 bit data out of 12 bit data. The Bayer interpolation interpolates color, it doesn't add any intensity information and it can't create true 16-bit data out of 12-bit information.

Anders Widman, June 11, 2004; 03:41 P.M.

Good article. Many people will find this information useful. However, it contains a few missed.

Firstly, you assume that a user would convert their RAW files to 8bit per channel images before printing - and therefore they could just as well shoot as JPEG.

There are a number of reasons for using 16bit per channel mode. The biggest is that Photoshops internal precision is the same as the bit depth. This means that there will be huge round-off errors when converting between colour profiles, LAB and also to/from CMYK. The maximum amount of colours can retain when converting colour space is around 2.5 milion in 8bit/channel mode. 16bit/channel gives better precision - even if the output is only 8bit/channel!

The other reason to still shoot RAW is that it gives the user more control to post-adjust the image to his liking without loosing to much visible quality. For every adjustment that is done on an image there is some loss of quality. If you start out with the 16bit/channel image you may only have 10bits left after doing some levels adjustments and other things. This is still more than the 8bit/channel that JPEG gives. If you start out with a 8bit/channel image you may only have 6-7bits/channel left when you have finished editing.

Another point is that users can still preview their RAW files. Photoshops internal file browser can preview and make thumbnails of RAW files. The user should at least have a image viewer from the camera vendor to view the RAW files without first converting them.

Steve Binghamphoto.net hero, June 11, 2004; 03:45 P.M.

A really GREAT explanation Bob. However I do take exception to "If you know you have the correct exposure and white balance as well as the optimum camera set parameters, then a high quality JPEG will give you a print just as good as one from a converted RAW file, so you may as well shoot JPEG."

I use the Fuji S2 and have done exhaustive testing with both the Fuji EX raw convertor (version 2.0) and Adobe Camera Raw convertor (version 2.2x25) and can say there is a very distinct quality difference between a jpg and raw conversion - especially with EX. There is more detail, less fringing (chromatic aberrations), no jpg artifacts, and less halo effect - especially if sharpened before printing. Lastly, you will end up with a fuller color gamut from which to work with. Granted that if all you want is a 4"x6" or 6"x9" print you will never SEE the difference. However, if you should later decide to make a large print, say a 16x24, you will be very disappointed! There is a considerable difference in quality. That's why I shoot 100% raw. Besides that, in spite of 50 years experience, I still blow exposure from time to time. Raw keeps me from having to bracket. Sometimes that 1/3 stop one way or the other can make a considerable difference. Lastly, using raw enables me to expand my dynamic range. If I discover later that the range of the scene is excessive, I simply make two raw conversions, one to save the highlights and a second to preserve shadow detail. (You will need a working knowledge of Photoshop layers)

For those who wish a simple "how to" on expanding dynamic range go here: http://dustylens.com/extended_range.htm This works best with cameras with very low noise floors, such as the Fuji S2.

Oh, and for Walter. This statement is not true: "All you can do there is just to shift the histogram in a convinient way. its just another way and place to do the same thing that you could do just as well after importing the image into PS (or other program)." I have proven this to be false many times. Try this simple comparison and see for yourself. Shoot a raw image with a small exposure problem. Now adjust the raw exposure before converting to a tiff. Now make a second 16 bit conversion without exposure compensation. In PS CS adjust the exposure of the second conversion. You will see lost information in the file where exposure was corrected in PS. Also compare the historgrams. There IS a difference. If you take the time to play around with this you will see that the raw file that incorporated exposure correction has more information! You would think the 16 bit tiff would contain all the information in the 12 or even 14 bit raw file. It does NOT. Emperical evidence rules over theory. Try it.

Anders Widman, June 11, 2004; 03:56 P.M.

There is still confusion about the "exposure compensation" option in RAW converters.

It is true. You can't get out more detail from the RAW file by changing this setting. What you do it to shift the dynamic range of the image data in the RAW file so it fits in the (much smaller) range of the monitor, making it visible on the monitor. The data, however was always there, but it was clipped out by the monitor or the display card! This does not have to do with the amount of intensity levels, but the intensity range (the maximum intencity difference between the darkest and brightest data that can be recorded/displayed).

Bob Atkinsphoto.net admin photo.net hero photo.net patron, June 11, 2004; 06:16 P.M.

" there is a very distinct quality difference between a jpg and raw conversion "

Well, that depends on the software used.

The JPEG file is, of course a RAW conversion. All images start out RAW and get converted to something you can display. With JPEGs, the RAW conversion is usually done by a dedicated firmware chip in the camera. When you do RAW conversions on a PC you use some other program to do it.

RAW conversion is not a defined process using a single algorithm. It's a black box. RAW goes in one end, JPEG (or TIFF) comes out of the other. What happens in the box depends on the programmer. There are fast algorithms of low quality and there are slow algorithms of high qualty. It's certainly possible for a PC converted RAW file to be different from one that's been converted in camera. In fact it would be very surprising indeed if they were the same!

For the typical photographer however, I'd still say that a high quality in camera JPEG conversion (such as that done by the EOS 10D) will be very comparable (if not pixel for pixel identical) to a RAW conversion to JPEG done in external software. I don't deny that there will be small differences visible if you look very closely, but they're unlikely to be significant to the typical photographer.

What I'm saying is that you shouldn't be afraid to shoot JPEGs. It's a lot easier, faster and I'd bet that for over 90% of all digital SLR shooters, the quality would be just fine.

Eric Willner, June 12, 2004; 06:52 A.M.

The ability to recover highlight and shadow detail from an image is a key reason for using 16-bit RAW files. It is the exact equivalent of lattitude in traditional film processing. The more bits you start with, the more lattitude you have to adjust brightness and contrast, or dodge and burn parts of the image afterwards without losing tonal detail. Noise is the other limiting factor, so to get maximum lattitude you need lots of bits and a low noise sensor. In fact, noise (or more specifically signal/noise ratio) becomes a problem long before the number of bits run out.

Gordon Richardson, June 12, 2004; 07:42 A.M.

The gamma encoding step should be made more explicit in the flow-chart (it takes place between Bayer Interpolation and Jpeg compression - the exact sequence is not clear to me). It applies in the same way as in film or print scanning (which is an analagous process).

Gordon Richardson, June 12, 2004; 12:05 P.M.

It may be difficult to comprehend a non-linear (gamma-encoded) space without some actual numbers: In a 12-bit linear space (RAW) the midtone of 2.5 stops below maximum (1/5.6) would be 730/4095 (not 2047). In an gamma-encoded space (Jpeg and 8-bit Tiff) the midtone would be half the maximum (or 127/255). A gamma value of 2.2 is normally assumed for PC's (or 1.8 for Mac).

At the lowest brightness levels (shadow area) a 12-bit linear value of 1/4095 gives a gamma-encoded value of 6/255, which (just) discernably different from black. A linear value of 2/4095 maps onto 8/255 (a barely detectable increment). Exposure adjustment can map this input to lower output results, although the lowest 12-bit linear values are sensitive to noise.

During gamma conversion several bits are discarded, particularly at high brightness levels, where many 12-bit linear values map onto the same 8-bit result (all values above 4080/4095 would map onto 255/255). This is acceptable since the human eye does not distinguish brightness differences of less than 1%. For example 4079/4095 would not be distinguishable from 4095/4095 (0.4% darker).

A 12-bit depth is certainly useful when making large scale adjustments in brightness and contrast. A full brightness range of 0 to 4095 is unlikely to be used, and an exposure adjustment would truncate this range. Major adjustments done at an early stage will almost certainly give better results than those done after gamma-encoding, where an 8-bit image would suffer posterisation (RAW images have more "headroom").

Since Tiff values are almost always gamma-encoded, the use of 16-bit depth has limited advantages compared to 8-bit. However for RAW values an 8-bit linear space would not nearly be enough (giving only a lower value of 21/255 when gamma-encoded). This is also mentioned in the previous article about Dynamic Range (scanning).

Walter Schroeder, June 12, 2004; 03:10 P.M.

Steve - you may well be right in adjusting false exposures better with the RAW converter and then the fine tuning in PS CS - rather than to do it all in PS CS without prior use of the exposure compensation in the RAW plugin. If so - it was good to point this out and I will give it a try and convert my workflow. So far I tried to avoid any changes in the RAW converter and do all manipulations in the PS CS since I have more visual control over what happens the way I have PS set up with 2 monitors and the channel histograms (and the rest) on the second monitor but the current image on the first monitor(21" CRT). But as Bob Atkins also pointed out, this is a "black box" and we do not know therefore what works best. But this actually was not my point. My point was that you can not bring back information that was not recorded in the camera due to wrong exposure by "compensating" in the RAW plugin. You know that , but this false view often appears in posts here in PN and other places. So I just want to clarify this again so it does not get lost in the workflow discussion.

Bob Atkinsphoto.net admin photo.net hero photo.net patron, June 12, 2004; 10:39 P.M.

Following up on Gordon's comments, it's woth noting that some (many?) Raw converion utilities allow the output of 16-bit TIFF files with linear encoding as well as gamma encoding. I think this is a subject too complex to get into in the comments section of this article and probably merits a seperate and more advanced write up. Combining linear and gamma converions can sometimes salvage images with slighly blown out highlights. Note I said "sometimes" and "slightly", if you really blow the highlights, they're gone forever. If you have them crammed up against the top of the histogram, sometimes you can improve the image and get back some highlight detail.

Gordon Richardson, June 13, 2004; 02:19 A.M.

Bob, thanks for the clarification (I don't have access to any of those tools). A 16-bit linear space would certainly allow scope for some significant adjustment, though standard editing tools might not work well (a complex subject).

A linear space is equivalent to using a gamma of 1.0 (this would look very dark on a normally calibrated monitor). Jpeg's are always gamma-encoded since they are 8-bit, and are assumed to be used directly for output.

Gamma is a subset of a broader group of curves and levels, though often overlooked (and fundamental to the display process). A full discussion would be beyond the scope of an introductory article, though similar issues in your Dynamic Range (scanning) article are relevant (I tried to add a link).

Steve Binghamphoto.net hero, June 13, 2004; 04:25 P.M.

A lot of very useful knowledge by all. It is these sort of discussions that make photo.net outstanding!

Juun Ea, June 14, 2004; 12:04 A.M.

Thank you again Bob, for a clear, simple, and enlightening article. I just bought and used my first DSLR (10D) this weekend and had questions specifically related to RAW to TIFF conversion. I came online and there was your article. How timely and helpful.

It's still not so clear to me, however, all the advantages there are to converting images to a 16 bit versus 8 bit TIFF file. I called up Canon and they seem to recommend 8 bit for convenience, mainly with respect to the accessibility of 8 bit files by the more programs.

Thanks.

Steve Binghamphoto.net hero, June 14, 2004; 09:51 P.M.

For a more detailed and somewhat scientific explanation of how expanded range works, go here and scroll to the bottom of the page. http://dustylens.com/extended_range.htm

Rodger Donaldson, June 16, 2004; 01:53 A.M.

Although I generally shoot RAW myself (figuring that it's better to have more data than I needed, rather than less), one potential fishhook people need to think about when archiving: RAW is generally dependent on the manufacturer's proprietary software. JPEG and TIFF are not. There is no guarantee that you will be able to find a computer that can read a Canon RAW format in, say, 10 years time.

It is very unlikely that computers in 10 years time will nothave some tools for working with JPEG and TIFF images.

Jussi Vakkala, June 17, 2004; 02:55 A.M.

"There is no guarantee that you will be able to find a computer that can read a Canon RAW format in, say, 10 years time."

I would like to disagree strongly. Dcraw -converter is available in free Ansi C source code. Which means that current version of dcraw will compile on any future platform supporting Ansi C.

To my knowledge Ansi C will *not* be obsolete for any foreseeable future. Much too much code has been written in Ansi C to be wasted on next fifty years.. Dcraw is a very stable piece of code and has been used by big names like Adobe. Big thank's for that to Dave Coffin!

Anders Widman, June 17, 2004; 03:12 P.M.

Some clarifications - again :)

1) It is true that the camera can not resolve any details in over- and underexposed parts. Those details are outside the dynamic range that can be recorded and are therefore mapped to pure black or pure white.

2) The only reason you can see more details when you "exposure compensate" in the RAW conversion software is because the monitor has a smaller dynamic range and resolution.

3) Dynamic range and resolution is not the same thing. Dynamic range refers to real-world differences in energy levels. Resolution is how fine steps you can take. It does not tell you anything about the dynamic range. It would be about the same error to say that my image is 300dpi wide as it is to say it has a 16bit dynamic range.

What does this mean then?

It means that when you change the "exposure" in the RAW conversion software is that you shift and compress the original wide dynamic range into the smaller range and lower resolution of the monitor. It does not bring more detail or resolution - it only makes it visible on a low resolution device like your monitor.

A note about the bayer interpolation. It can be combined with the exposure control setting in the RAW conversion software. The interpolation algoritm is adjusted to match the target range. Because of high precision interpolation we will see more than 12bits of data in the final image. The added bits are computed data, but not nessesarily bad or unusable.

Gordon Richardson, June 18, 2004; 01:40 A.M.

A detailed discussion of numeric representation is somewhat off topic, but it is worth re-emphasising one point about gamma-encoding. It increase the dynamic range (almost doubled), at the cost of some loss in precision (fine steps) in parts of the range that are not visually perceptible. (Resolution is not the right word IMO, and conversion spreads the range rather than compressing it.)

The range of monitors is a hardware (physical) limitation, not an inherent property of 8-bit images. I discussed the topic of extended range with Steve Bingham, and point out that a typical Jpeg has a surprising amount of shadow detail that can be brought out by levels changes. The optimum method for getting detail from the camera depends on the software provided.

Dynamic range and precision are linked, in that a fixed number of bits puts an upper bound on the dynamic range, assuming a perceptually uniform space. Using integer representation does have inherent limitations, but has the advantages of compactness and fast calculations. Other methods such as logarithmic bitmaps can increase the dynamic range, or the precision, but there are few advantages to doing so (discussed May '04).

Ajoy Prabhu (www.AjoyPrabhu.com), June 18, 2004; 03:54 P.M.

One aspect, though alluded to by some including Bob, is that with RAW, you can work with the linearity of the sensor response fall-off by keeping the histogram "pushed" to the right, but just so, not blowing off the pixels. I have noticed that what looks overexposed when viewed in the Camera LCD, can be easily "corrected" and the tonality well-preserved with real good shadow detail and less noise.

Ajoy

Sean O'Flaherty, November 22, 2004; 09:27 P.M.

JPEG2000 not only offers better compression, but loss-less compression as well. So it could replace both JPEG and TIFF. Unfortunately fear of paten disputes is keeping it form being widely adopted. See http://en.wikipedia.org/wiki/JPEG2000#Legal_problems_with_the_use_of_JPEG_2000

Anders Widman, January 01, 2005; 11:33 A.M.

JPEG2000 has a lot of potential. There are a few things you should be aware of when using it. First of all the difference in the wave-let compression used in JPEG2000 and new video codecs like MPEG4-Part 10 (H.264).

For normal, smooth images with lots of gradients and low complexity the JPEG2000 is superior than normal JPEG. BUT when you deal with images with very high complexity and lots of high frequency detail, the normal JPEG often gives better results!

This is the result of the wave-let compression. It removes high frequencies, rather than producing the classical JPEG blocking artifacts. But when you deal with high quality images with lots of details it can really ruin the image. For example. Take a picture in a city environment. The JPEG2000 could cause the textures on the walls of the buildings to be lost. Be careful. It deceives you!

However, if you want to post images for the web, then JPEG2000 could be very good. it offers very good quality / size ratio.

The lossless mode is very good though. This is the mode I would use with JPEG200. It is much improved over the lossless mode of normal JPEG and PNG.

thomas seest, January 11, 2006; 12:15 A.M.

hello and thank you for the introduction. i have a question about the root information. The Raw file. Is it possible to view the red/green/blue sensor image, or BAYER PATTERN of a image? When I use Raw in photoshop, it will only allow me to make a transformation of the image. i am curious to see the original root. If anyone has a link to software that would allow this I would be very greatful to see it.

thomas

RAVI KUMAR, January 04, 2008; 09:42 A.M.

hello and thank. you for the introduction Raw,jpeg very greatful and Excellent

vu tran, January 23, 2008; 08:11 P.M.

Hi,

I received some RAW files in DCR format (from a Kodak camera). Is there a free decent software to convert them to TIFF files?

Much appreciated.

Vu

Venugopal Thantry, February 06, 2008; 08:00 A.M.

Bob,thanks at ton for this info.

One trivial query - a)Aperture control is mechanical which is done on lens side, b)ISO control is done by appropriate amplification of signal c)how is the shutter speed control done in these digital cameras? (equivalent to mechanical shutter which used to move in front of the film).

Ian T Davis, February 15, 2008; 04:31 P.M.

Jussi Vakala mentioned CDdraw: I use a conbination of Linux-Opensuse, UFraw, and Gimp software and I believe UFRaw utilises DCDraw. So, with this set up I can, for next to nothing in cash outlay - many thanks to all involved with open source software, shoot RAW; process, and store, my files in non propriety formats. So, I like to believe, they will be accessible in say 10 years time. The only reason I convert to JPEG is to upload files to internet photography sites: e.g photo.net. Even then usually only to keep file sizes within reasonable parameters. Should we then be concerned about propriety/lack of standard RAW file formats.

Frimu Toni, February 16, 2008; 04:24 P.M.

Thanks a lot , interesting and helping post. I`m a beginner and it`s important to get more informations about all this process.


Add a comment