CS6 and RAM

[mawyatt1]

I have a couple year old MBPr that has 8GB ram. Just ordered a Transcend SSD upgrade.

I do some image processing with CS6 and Lightroom 4 (plus Zerene Focus Stacking). The images are from a Nikon D800 either raw ~50MB or TIFF ~110MB. My MBPr only has 8GB ram and I know I need more ram, so doing what I can with the upgradable SSD.

I am not sure if I should get a Mac mini or IMac 27" since it can handle 32GB ram, the mini is limited to 16GB.

Any of you Photoshop/Lightroom experts know if I'll see much improvement with 32GB ram vs. 16GB with these large image files? This may help me decide on the mini vs. iMac.

Thanks

 

[simsaladimbamba]

How many layers do you normally have in your PSD files and how many PSD files do you have open at one time?

Even when I heavily used PSDs with 100s of layers and being 2500 pixel wide or so, I never used more than 2.5 GB of RAM with CS6 on an 8 GB machine.

 

[mawyatt1]

How many layers do you normally have in your PSD files and how many PSD files do you have open at one time?

Even when I heavily used PSDs with 100s of layers and being 2500 pixel wide or so, I never used more than 2.5 GB of RAM with CS6 on an 8 GB machine.

I usually work in raw/TIFF files, sometimes with NIK plugins. I usually don't have 100 of layers, but could have 100's of images open in Lightroom. Usually under 10 images open in CS6 at one time. The images can get really big though, even exceeding 1GB in some cases before I flatten them.

Thanks for the help,

Mike

 

[simsaladimbamba]

Then maybe try the following.

Restart your Mac at the beginning of a normal LR/PS work day, do all the stuff you normally do and at the end of the day open Activity Monitor and look at the MEMORY tab and look at PAGE OUTS or the SWAP file size listed there.

That will give you an inkling, if 8 GB is not enough and if you need 16 GB of RAM or 32 GB of RAM, the latter probably being unlikely.

 

[ChrisA]

I have a couple year old MBPr that has 8GB ram. Just ordered a Transcend SSD upgrade.

I do some image processing with CS6 and Lightroom 4 (plus Zerene Focus Stacking). The images are from a Nikon D800 either raw ~50MB or TIFF ~110MB. My MBPr only has 8GB ram and I know I need more ram, so doing what I can with the upgradable SSD.

I am not sure if I should get a Mac mini or IMac 27" since it can handle 32GB ram, the mini is limited to 16GB.

Any of you Photoshop/Lightroom experts know if I'll see much improvement with 32GB ram vs. 16GB with these large image files? This may help me decide on the mini vs. iMac.

Thanks

Apple's "Activity Monitor" was created to answer these questions. Watch it while you work to monitor memory "pressure" and disk IO.

Those files are not so big. Even if you had 20 of those 110MB files open at once it would still only be 2.2Gb of data. Yes they are large for still images but they are STILL images, not video.

Likely a large and fast storage system would give you the best performance. But do they make SSDs large enough to hold your library? If not building a RAID system out of SSDs gets expensive and then you'd want to connect to it with Thunderbolt which is also expensive.

Activity Monitor should answer your questions although you might have to ask for help on how to interpret what it is saying.

You might find that what you really need a good graphic card. And that would point you to a high-end iMac. 16MB should be enough, but you can verify that with Activity Monitor.

 

[mawyatt1]

Thanks everyone.

I am running Lion OS and I'll look at the Activity Monitor in more detail than I have.

I have a dual RAID Thunderbolt System with twin 4TB LaCie 2 Big drives and a 3TB Firewire and another 3TB Ethernet drive, so HD space I am good.

Thanks again,

Mike

 

[Laird Knox]

I'm running a 2011 MacBook Pro with 16GB RAM and a 500GB SSD. It handles D800 files with ease for the most part. I only have some small issues when I am working with multiple images in one file (star trails and light painting).

If I were able I would upgrade my laptop (16GB, OS/X) and desktop (32GB, Win 7) but I am not unhappy with their performance when working with D800 files.

 

[mawyatt1]

I just ran some of my lower stress applications and watched the Activity Monitor, it created over 2GB of swap, and 3 GB page out. So I think the 8GB ram might be the bottleneck.

I'll try and run one of the more challenging modes (PS & Zerene Stackshot) later this week.

Thanks for all the help and advise, much appreciated

Mike

 

[The Bad Guy]

Close Lightroom while you work in Photoshop. Makes a noticeable difference.

 

[ChrisA]

I just ran some of my lower stress applications and watched the Activity Monitor, it created over 2GB of swap, and 3 GB page out. So I think the 8GB ram might be the bottleneck.

So you see right away that 16MB will work fine. You want to get the Page Out count to zero. It's OK to have high Page In count.

 

[phrehdd]

I work with CS6 PS on a Mac Mini and in the past, have worked with it on an older Mac Pro with 32 gigs of RAM.

If you take a look at various PS sites, you'll see that indeed that the extra RAM does make a difference. If my Mini could handle more RAM I would get it pronto.

From what you describe as your typical run with Lightroom and PS, at least 16 gigs would be a worthy starting point. Also since scratch space is required, that SSD should be really handy to also reduce bottleneck issues.

If you opted for the iMac, consider getting 3rd party RAM and start perhaps around 24 gigs if not the full 32. Part of the challenges is setting up PS to really run smooth as well as understanding that you should always leave some RAM "empty" as applications can go up and down in usage.

My Mac Mini has 16 gigs RAM and SSD. I would consider this the minimum for dealing with files of the sort you mention. My work deals with both RAW files and TIFF (from a scanner) and can get quite cumbersome at times with layers.

The above is just an opinion based on my own experiences. I am sure others will have various opinions so take it for what it is worth.

Presently - I have mail and Firefox open along with Adobe Reader, of my 16 gigs, only 7.29 gigs shows as 'free.' There is a misconception that each app uses finite amount of RAM. Some apps will use more RAM if it is available and some apps are 'bad actors' in that they don't always mind their own store properly and use up memory resources without releasing them (if left open).

 

[The Bad Guy]

Some apps will use more RAM if it is available and some apps are 'bad actors' in that they don't always mind their own store properly and use up memory resources without releasing them (if left open).

Oh **** yeah, Adobe software in particular will literally take as much RAM as you can throw at it.

 

[mawyatt1]

Thanks everyone, this is a big help in my decision process. Closing LR does help.

On another site someone mentioned that using a SSD as any kind of temp storage (swap, temp dir) is a bad idea because the SSD gets slower over time because of the large amounts of write modes temp storage has. I don't know much about SSDs other than they are based upon Flash memory.

Anyone care to comment on this?

Thanks,

Mike

 

[mawyatt1]

Here's an example (very low resolution) of the images I am involved with. These are the silicon chips I/we/others design and work with. The best ones I can't show, so these are the what's available.

To create a final image requires up to 500 individual 36Mpixel (D800E) images stacked. These must be proceeded as 16 bit TIFF files, JPEGs cause subtle problems. The lens is a microscope objective designed for a tube type microscope we use in our probe stations. It's very high quality and requires a "tube lens" equivalent at 200mm, meaning it images onto the tube lens, which then images to the eyepiece. I have replaced the "tube lens" with a Nikon 70-200 F2.8 @ 200mm F2.8. This images to the camera sensor. All this creates a very thin ribbon where the chip is in focus only about 7 microns, its actually close to wave-length/(NA^2), where NA is the numerical lens aperture for you scientists/engineers like me.

This thin in-focus ribbon is then stepped cross the chip and stacked to create the final overall in focus image. Things like wirebonds and chip package create more problems because of the vastly different depth of objects when the chip image is shot at angle (around 30 degrees). The wirebonds, package are "seen" looking down onto the chip below and create stacking issues.

Lighting is another issue, especially for what is called solder bumps as seen in one of the images. These are tiny (less than 100 microns diameter) solder balls that allow the chip to be mounted upside down, called flip chip. These balls are basically spherical mirrors and reflect all the light sources. You need to create a uniform light source without any significant "hot spots". This was done by using dual light tents one inside the other, and double diffused light strobes (4) and multiple double diffused other light sources. The equivalent light power is ~1.5Mwatts!!

All this took years to figure out. The final images are spectacular in detail, you can dive down into the chip and see subsurface details at the sub-micron level.

I now want to go to 1000 or more individual images for some of the newer chips involved. This will tax my computing power even more, so my interest in getting more computer processing capability.

Sorry for the long thread, but though some folks might be interested.

Cheers and thanks for all the help,

Mike

 

[phrehdd]

Thanks everyone, this is a big help in my decision process. Closing LR does help.

On another site someone mentioned that using a SSD as any kind of temp storage (swap, temp dir) is a bad idea because the SSD gets slower over time because of the large amounts of write modes temp storage has. I don't know much about SSDs other than they are based upon Flash memory.

Anyone care to comment on this?

Thanks,

Mike

There is truth that there are finite read/writes/destroys on SSD drives. However, given your tasks at hand, better to still consider an SSD drive and if/when the drive becomes 'noticeably' slower, replace it. I suggest you check out some Photoshop oriented forums to see how others manage their scratch disk challenges.

It seems a better fit would be to get an older traditional Mac Pro 6 or 8 core, and exploit the 4 internal drive bays (plus 2). You might fill one with an SSD for OS and apps and then stripe 2-3 standard drives for scratch or simply have two SSD drives with one used for scratch. Drives like Samsung Evo or Pro come to mind as a pretty good fit.

It seems that some folks purchase the new Mac Pro which is SSD only and use it for tasks that are rather intensive such as video work with no concern for SSD wear. I would worry less about SSD wear as drives are replaceable but lost time (due to bottlenecks) is not. Just an opinion here.

 

[Laird Knox]

To create a final image requires up to 500 individual 36Mpixel (D800E) images stacked.

You have me beat there. My largest stack is about 250 D800 images.

Really interesting images, would be fun to explore a full sized composite. What is the typical final dimensions on your images?

 

[mawyatt1]

You have me beat there. My largest stack is about 250 D800 images.

Really interesting images, would be fun to explore a full sized composite. What is the typical final dimensions on your images?

The high end ones are about 1/4GB after flattening. When I saved the entire workflow it would take about 150GB of HD space to support one image. I usually don't save the workflow anymore as it was taking too much HD space. I got a couple 4TB refurb LaCie Thunderbolt 2Bigs as RAID drives, they work well with Thunderbolt drives and are pretty quick, a have another 3TB Firewire and another 3TB Ethernet as my drive space. The LaCie uses the Seagate 7200rpm Barracuda drives which are reasonable price wise should I need to replace them. So far they have been reliable.

Originally I would import the raw files into LR and convert to 16 bit TIFF for stacking, this took forever. Then I realized I could get TIFF directly from the D800E, so I do that now. I had copied the camera files from the camera to the LaCie where I worked on them, this was slow. I then copied them into a temp directory on my MBPr SSD, this was faster and prompted me to order the Transcend JetDrive SSD upgrade. But reading about "this might not be a good idea" (using temp SSD based file) on another site caused some concern on my part.

I am by no way a computer nor Mac guru, but do understand some of the basics and the help you and the folks here have provided is greatly appreciated. Just don't want to make another mistake like I did with the MBPr (not getting the 16GB ram). BTW in the chip design world where I live mistakes are very costly and generally not allowed...so I have been conditioned to get it right the first time!!


Probably should mention this is all done (mostly) on my own nickel and time, so getting Mac Pro with 128GB ram and 4TB SSD drives is out of the question. Now our chip design CAD computers would certainly have fun with this, but they are off limits!!


Cheers,

Mike

 

[mawyatt1]

Here's the setup at home. I have replaced the tripod with a copy stand modified for this use. The tripod took too long to settle, so I use a copy stand which has a much higher natural frequency, which allowed me to shorten the time between exposures. Also use a rug to increase the damping. I generally need to wait until evening so the pool pump is off, no one walking around, and the AC is off, this produces the least vibrations.

I even tried rear curtain sync for the strobe trigger to help with the shutter vibration. I would really like to get an D810 because of the electronic shutter, but that's not going to happen either!!

Best,

Mike

 

[nburwell]

To the OP, you will notice a difference if you upgrade your RAM. I work off my rMBP and I currently have 8GB of RAM. I use both LR5 and CS6 to process my D800 files, and I do not really have any lag to be honest. The only time I notice a lag is if I'm stacking images in layers for star trails from LR to PS, but I don't do it all that often.

 

[Laird Knox]

Probably should mention this is all done (mostly) on my own nickel and time, so getting Mac Pro with 128GB ram and 4TB SSD drives is out of the question.

I hear that. I built my own machine specifically for image processing. It has striped SSDs for the boot drive, a Ivy Bridge processor (it is a couple of years old now), 32 GB RAM and discreet video cards for each monitor so that I can calibrate them both. It was much less expensive than a Mac Pro. I might consider an upgrade when the next generation of chips becomes available. I'll definitely be looking for a system board that will support more than 32 GB of RAM.

Thanks for all the info on your process. 7 microns is a crazy depth of field. Do you do these images for work or out of personal interest?

 

[phrehdd]

The high end ones are about 1/4GB after flattening. When I saved the entire workflow it would take about 150GB of HD space to support one image. I usually don't save the workflow anymore as it was taking too much HD space. I got a couple 4TB refurb LaCie Thunderbolt 2Bigs as RAID drives, they work well with Thunderbolt drives and are pretty quick, a have another 3TB Firewire and another 3TB Ethernet as my drive space. The LaCie uses the Seagate 7200rpm Barracuda drives which are reasonable price wise should I need to replace them. So far they have been reliable.

Originally I would import the raw files into LR and convert to 16 bit TIFF for stacking, this took forever. Then I realized I could get TIFF directly from the D800E, so I do that now. I had copied the camera files from the camera to the LaCie where I worked on them, this was slow. I then copied them into a temp directory on my MBPr SSD, this was faster and prompted me to order the Transcend JetDrive SSD upgrade. But reading about "this might not be a good idea" (using temp SSD based file) on another site caused some concern on my part.

I am by no way a computer nor Mac guru, but do understand some of the basics and the help you and the folks here have provided is greatly appreciated. Just don't want to make another mistake like I did with the MBPr (not getting the 16GB ram). BTW in the chip design world where I live mistakes are very costly and generally not allowed...so I have been conditioned to get it right the first time!!


Probably should mention this is all done (mostly) on my own nickel and time, so getting Mac Pro with 128GB ram and 4TB SSD drives is out of the question. Now our chip design CAD computers would certainly have fun with this, but they are off limits!!


Cheers,

Mike

Again - consider a used Mac Pro that maybe is a hex core, 64 gigs RAM, only 1 SSD and then some regular drives striped. From what you are saying about your work flow and the size of files, you really do take a hit on lesser machines as far as process times. While I personally am not a fan of the iMac (mostly due to RAM limitations and screen is not a good fit for my graphic needs), it might be a better option for you than the Mini for the ability to use 32 gigs RAM. Also, you might want to check out TB connection to a cheap SSD for scratch space. If an SSD remains fit for 3 years, it has more than paid for itself in terms of speed and production time. Even standard drives should be replaced after a certain amount of time rather than wait for it to start corrupting files or simply fail. I use a Firmtek USB3 enclosure with SSD that performs on par with TB1. I base this on speed test and made the enclosure purchase after reading a review in Barefeats website.

Btw, I admire your skill and patience with the type of work you are doing. It must really make you smile when you get the final image composite and ready for 'public consumption.'

 

[mawyatt1]

I hear that. I built my own machine specifically for image processing. It has striped SSDs for the boot drive, a Ivy Bridge processor (it is a couple of years old now), 32 GB RAM and discreet video cards for each monitor so that I can calibrate them both. It was much less expensive than a Mac Pro. I might consider an upgrade when the next generation of chips becomes available. I'll definitely be looking for a system board that will support more than 32 GB of RAM.

Thanks for all the info on your process. 7 microns is a crazy depth of field. Do you do these images for work or out of personal interest?

I do these for work and personal interest both. I don't get much support from work though, so almost all on my own time and expense. I believe that these images are good advertising for what we do, and this has proven out. Our company & management is operating under a bean counter mentality, so everything is about creating revenue right now and this falls outside that scope.The very reason we are in this situation now is because of the past history management and tomorrow will be worse. It just goes on and on!!

You are welcome, and yes 7 microns is crazy, and when I use the 0.42 NA lens it will be about 3 microns!!

Best,

Mike

----------

Again - consider a used Mac Pro that maybe is a hex core, 64 gigs RAM, only 1 SSD and then some regular drives striped. From what you are saying about your work flow and the size of files, you really do take a hit on lesser machines as far as process times. While I personally am not a fan of the iMac (mostly due to RAM limitations and screen is not a good fit for my graphic needs), it might be a better option for you than the Mini for the ability to use 32 gigs RAM. Also, you might want to check out TB connection to a cheap SSD for scratch space. If an SSD remains fit for 3 years, it has more than paid for itself in terms of speed and production time. Even standard drives should be replaced after a certain amount of time rather than wait for it to start corrupting files or simply fail. I use a Firmtek USB3 enclosure with SSD that performs on par with TB1. I base this on speed test and made the enclosure purchase after reading a review in Barefeats website.

Btw, I admire your skill and patience with the type of work you are doing. It must really make you smile when you get the final image composite and ready for 'public consumption.'

Thanks, patience is something I acquired over time. I am old now and my patience is much better. Yes it is satisfying to see the images emerge from all the individual images that just look horrible by themselves!!

Most of the really good images I can't post or show the general public, they are much better than these. Someday maybe I'll be allowed to post them, we'll see!!

Anyway I am an engineer/scientist (as if you didn't already know :>), and this has been a engineering problem, and working out the details is something we engineering types like to do…so it's been kind of fun.

Please don't tell my wife, she thinks it's all work and no fun!! She was in Ca when I sent the image of the setup on the living room table. Her only email comment was "WTF!!!!" But honestly she has been really good about letting me mess up the house, of course she it's here much which helps!!

 

[yjchua95]

Thanks everyone.

I am running Lion OS and I'll look at the Activity Monitor in more detail than I have.

I have a dual RAID Thunderbolt System with twin 4TB LaCie 2 Big drives and a 3TB Firewire and another 3TB Ethernet drive, so HD space I am good.

Thanks again,

Mike

Upgrade to Mavericks, because RAM management in Mavericks is much more efficient. Mavericks' RAM compression helps too.