Help identifying a bottleneck

[Loa]

Hello,

I shoot a lot of photos with a nikon d500 using raw files. Loading and working with these files can be slow on my 2009 mac pro and I was wondering if the bottleneck was the CPU (3.33 quad) or the drive (sata3 drive working at roughly 350MB/s because of crappy sata3 card).

I can't do much for the cpu, but if the bottleneck is the drive, I could go for a HyperX Predator.

Thanks

Loa

 

[h9826790]

How large your file?

Did you try just connect the SSD (I assume it's a SSD because of 350MB/s) to the native SATA 2 port to check if the PCIe card itself cause the problem?

Slow editing can be lots of reason, but slow loading should be easier to identify the root cause.

Also, if you have enough RAM, you can make a small RAM drive, put the file there, and check if the loading / editing still slow. If yes, that means your problem has nothing to do with your drive.

 

[Loa]

File size = roughly 22-25MB.

I never connected the drive on the sata2 port, given it's ~275MB/s limit.

It's more of a loading problem, not really editing.

I could try making a small ram disk, thanks for the suggestion.

 

[h9826790]

For most operation, SATA 2 and SATA 3 won't make any detectable difference.

I have a Tempo SSD SATA 3 card, I know the difference between 500MB/s and 250MB/s. And 99.99% of time, can't feel any difference.

For a 25MB file, just 0.1s for 250MB/s to finish loading, do you think it really make any difference if you can further push it to 0.05s? Human being can hardly tell the difference.

Of course, if you put them side by side, you should able to tell 0.05s is faster. However, I don't think anyone will call 0.1s is slow.

I know we are only talking one file here. If you deal with 100 files at the same time, then that 0.05s will become 5 seconds, and SATA 3 will start to show its value.

Anyway, don't let the numbers (spec, marketing stuff, not real world usage) control you. That 375MB/s max PCIe card not necessary make your workflow faster than by using the native SATA 2 port. At least give it a try, and I don't think your work flow will slow down by 50% straight away, not even loading time.

And yes, please try the RAM drive test. That's the best way to tell if the slow loading is drive related. e.g. If the loading actually finish in the first 0.1s, but the software need another 3 seconds to finish the process, and display to you, you will feel that you have to wait for 3 seconds, but that's nothing to do with the SATA speed. Even the SSD can finish loading in 0.000001 second, you still have to wait 3 seconds.

 

[Loa]

I'll give the RAM drive a test and see. I understand your point about .1 vs .05.

My main beef is when I load 100 raw files in PS (from Bridge) and have to wait for them to load in the left column. Or when I scroll down and have to wait for the new files to load. It's not a lot of time in actual fact, but it's a lot of small pauses I have to wait until the system is ready for me.

 

[Stefan johansson]

Large files is always a problem,especially when it's photos in raw format. I tried some with my Sony cameras,and I can tell that your desktop Mac is at least faster than my MBP i5,so I don't think yours is so slow.

 

[Loa]

Well, strangely enough it's SLOWER when I use a ram disk. I created a 5 GB ram disk and copied a 1.8GB folder of RAW images to it. I tried opening all those images 3 times on the SSD and 3 times on the ram disk. It was slightly slower on the ram disk.

Any ideas where the bottleneck is then? Is it the old 3.33 nehalem?

Thanks

 

[h9826790]

So, we can basically confirm that faster SSD won't help. Or won't significantly improve the situation.

It's hard to tell what's wrong. But may be we can start with your Mac's config. e.g. How much RAM you have?

 

[Loa]

I have 16GB. I tried closing everything except bridge and photoshop, leaving me with a lot of free ram, and it didn't help.

 

[kohlson]

Loading from the camera, or from files already on the MP filesystem? Into what app?

If from the camera, then the following affect speed:
- The media and filesystem in the Nikon (unless you're removing the card, in which the read speed of that media)
- The interface (USB2?)
- The MP filesystem - how fast it read/writes
- The processing capabilities (CPU, memory)
This is a high-level simplification, and there is more to know. Such as, are the photos processed first, and then written to disk,

Fortunately, you can see how busy the CPU is, if you have memory contention, and to some extent disk performance by using Activity Monitor. Black Magic Disk Speed Test (free) can help you understand your disk performance.

What sort of improvement are you looking for? 10% or 300%?

 

[fhturner]

Yeah, more info about the specifics of the workflow might help us track it down further.

To me, it sounds like you may be exhausting open RAM and resorting to virtual memory/swap. You have 16GB, but with RAM disk, you're dropping nearly 1/3 of your RAM on that, plus you're opening a whole slew of RAW images into Photoshop, which likes a lot of memory itself. And the default amount of working memory for PS is 70% of the available memory, which means even less than the 16GB minus 5GB RAM disk minus system RAM.

Are you monitoring memory usage and swap space w/ Activity Monitor?

Other tests for comparison you could try to see if memory-related: cut your data set in half and test, then increase it to 2x, and compare those results to the the original data set. If it improves w/ half but gets markedly worse w/ 2x, you're probably memory-constrained.

 

[Loa]

Hello,

Ok, here's the set-up. 2009 Mac Pro with a 3.33GHz quad, 16GB RAM, with my sata3 ssd on a sata 3 card that tops out at 350MB/s according to black magic. I'm using Bridge and PS CC 2015.

The files have of course been copied from the camera to the SSD before opening bridge or PS.

The ram disk was just for testing purposes. Usually when I work on PS, I quit all my other apps.

I haven't restarted my Mac in a couple of days, and I did the test this morning. I have 0 page outs. None whatsoever.

Loa

 

[fhturner]

The ram disk was just for testing purposes. Usually when I work on PS, I quit all my other apps.

I haven't restarted my Mac in a couple of days, and I did the test this morning. I have 0 page outs. None whatsoever.

Okay, gotcha. I understood the RAM disk was just for testing...just sounded like it could be a memory limitation when you subtracted ANOTHER 5GB from the available memory to test w/ the RAM disk and the performance got worse. So maybe that's not it then... What about unoptimized Photoshop importing? Perhaps a single core is pegged at 100% during the loading operation and that's as fast as it can go?

My other thought would be in line w/ h9826790: check the SSD on a built-in SATA2 port/bay to see if the "crappy SATA3 card" might be the culprit.