What's "Compressed RAW"??

[Abstract]

Nikon has it. Canon has it. I just want to know how they can remove information when RAW data is supposed to be directly from the sensor. I mean, there's 12 million pixels on my D300, and I expect there to be enough information to explain what "pixel value" was recorded by all 12 MP on the sensor when the photo was taken.

Should I use a "compressed RAW" format, or is there no point in doing so?

My Nikon D300 can record in:

- uncompressed RAW
- lossless compressed RAW (saves 20-40% on file size, but quality stays the
same)
- lossy compressed RAW (saves around 40-60% space, but quality diminishes......WTF?)

I have been shooting Large Fine JPEGs, and lossless compressed RAW. I haven't really tried them all out to see if there's a huge difference while editing, but I certainly do see a difference in file size, so I shoot lossless compressed RAW to save space, but maintain quality. If I'm going to shoot in lossy compressed RAW, I may as well just save the images as an optimal quality JPEG or something, right?

 

[mcarnes]

I imagine it's similar to Apple lossless.

Example:

000111110000000000001111111111

is the same as:

0(3)1(5)0(12)1(10)

That's an over simplification, but you get the idea. No data is lost, but it takes less space.

 

[compuwar]

Nikon has it. Canon has it. I just want to know how they can remove information when RAW data is supposed to be directly from the sensor. I mean, there's 12 million pixels on my D300, and I expect there to be enough information to explain what "pixel value" was recorded by all 12 MP on the sensor when the photo was taken.

They can remove information by processing the file...

Should I use a "compressed RAW" format, or is there no point in doing so?

Depends on if you can accept the processing time and need the extra space.

My Nikon D300 can record in:

- uncompressed RAW

Large files, time to write is determined by file size, buffer space is less useful for long bursts...

- lossless compressed RAW (saves 20-40% on file size, but quality stays the
same)

Lossless compression is like zipping a file up, processing time to do so requires all bits be read and optimized. I expect (but don't know) that time to write is governed by the time taken to compress.

- lossy compressed RAW (saves around 40-60% space, but quality diminishes......WTF?)

Nikon throws out some data in the alpha channel, barely noticeable in prints, quicker to compress, smaller to write.

I have been shooting Large Fine JPEGs, and lossless compressed RAW. I haven't really tried them all out to see if there's a huge difference while editing, but I certainly do see a difference in file size, so I shoot lossless compressed RAW to save space, but maintain quality. If I'm going to shoot in lossy compressed RAW, I may as well just save the images as an optimal quality JPEG or something, right?

JPEG artifacting and white balance is different than just losing some of the alpha channel- for some the trade-off is worth it, for others it isn't. But then most folks don't even print the majority of their work, so JPEG's plenty-fine for them. I know photographers who shoot in the studio and think shooting raw is silly- they can't see waiting for the buffer to clear when they're directing a model. All the settings allow different styles of shooting, and allow differing results.

Personally, I don't know why you'd have the camera spend time buffering/writing a large JPEG if you're shooting raw- you can batch convert in post processing as you save off your NEFs- the only reason I see to save JPEG and RAW is if you want to quickly proof a shot somewhere that prints from the cards, and in that case, medium is generally good enough. But that's me and my workflow- you and yours may be different- good thing the cameras allow everyone some room.

 

[dcr]

Nikon throws out some data in the alpha channel

I'm confused by this statement. Usually alpha channel refers to transparency (like a Photoshop selection) which is not something an ordinary camera could capture from a scene.

http://en.wikipedia.org/wiki/Alpha_channel

My understanding is that lossy-compressed RAW files are exactly that -- lossy compression is applied to the mosiaced RAW sensor data. (Compare this to JPEG which is lossy compression applied to demosaiced/interpreted sensor data)

 

[Abstract]

Personally, I don't know why you'd have the camera spend time buffering/writing a large JPEG if you're shooting raw- you can batch convert in post processing as you save off your NEFs- the only reason I see to save JPEG and RAW is if you want to quickly proof a shot somewhere that prints from the cards, and in that case, medium is generally good enough. But that's me and my workflow- you and yours may be different- good thing the cameras allow everyone some room.

Yes, I'm aware of everything you said except regarding the alpha channel. I just didn't know what you lost through each compression. I don't know what an alpha channel is, but I guess I'll stick with lossless compressed RAW.

And when I said I shoot Lossless RAW and Large Fine JPEGs, I really mean to say that I shoot RAW 95% of the time, and JPEGs just to see if I can live with their result. I can't. I have never really shot JPEG until last month, when I got my D300. I forgot how bad editing a JPEG is.

My real question was whether a "lossy" compressed RAW gave me the same editing abilities as a full-fledged RAW file, or whether the compression would leave the RAW file crippled. I guess I don't need to worry about Lossless compressed RAW, so I'll stick with it.

I imagine it's similar to Apple lossless.

Example:

000111110000000000001111111111

is the same as:

0(3)1(5)0(12)1(10)

That's an over simplification, but you get the idea. No data is lost, but it takes less space.

Ok, this concept makes a lot more sense to me. The above example still doesn't make sense to me (ie: how they'd do that), but I guess I can live with not understanding.

I guess you're trying to say something like

000111110000000000001111111111
= 0(3)1(5)0(12)1(10)
= 0(11)1(101)(1100)1(1010)

....or something lke that. I just don't see how your explanation would work if the brackets weren't there. But again, I can live without really understanding, as long as I know I'm not losing any data.

Thanks.

 

[compuwar]

I'm confused by this statement. Usually alpha channel refers to transparency (like a Photoshop selection) which is not something an ordinary camera could capture from a scene.

http://en.wikipedia.org/wiki/Alpha_channel

My understanding is that lossy-compressed RAW files are exactly that -- lossy compression is applied to the mosiaced RAW sensor data. (Compare this to JPEG which is lossy compression applied to demosaiced/interpreted sensor data)

I was wildly sick yesterday- I meant luminance, but I think I was mistaken, while the differences do show up in the highlights, I think that's an artifact of the way they do the compression.

http://regex.info/blog/photo-tech/nef-compression/

 

[ChrisA]

- lossy compressed RAW (saves around 40-60% space, but quality diminishes......WTF?)

They apply a "gama" type curve to the values so that the same tonal range can be stored in fewer bits. So un-compressed raw is like measuring height with a tape measure and rounding to the nearest foot. "compressed raw" builds a staircase with un-even height steps, tall steps on the end the normal steps in the middle, and then reports hiegh by giving the number of the closest step in the staircase. This takes advantage of the eye not seeing light liberally.

 

[bocomo]

They apply a "gama" type curve to the values so that the same tonal range can be stored in fewer bits. So un-compressed raw is like measuring height with a tape measure and rounding to the nearest foot. "compressed raw" builds a staircase with un-even height steps, tall steps on the end the normal steps in the middle, and then reports hiegh by giving the number of the closest step in the staircase. This takes advantage of the eye not seeing light liberally.

not to split hairs but gamma correction has always been applied to raw files to compensate for the differences in the way the eye/brain sees and the way the camera lens/sensor sees. the camera/sensor sees in a linear fashion (twice as much light on a pixel equals twice as much voltage applied). your eyes/brain see logarithmically, so that you can see detail in a greater range of shadows and highlights. gamma correction is applied to the image your camera creates to make it look more like what you see

[should have said in the first sentence, "...has always been applied to digital camera files to compensate...]