Can I use a 3 Drive internal RAID as a Photoshop scratch disk?

[sngraphics]

Pageouts/swap used are different than scratch disks. They work in a similar manner, but scratch is application specific. Yeah 3-4GB is big. Not many people on here get quite that big, so they can usually fit it in ram. I just emphasize memory as it tends to be faster and often cheaper these days. The RAID could work, but it will slow down on saves if you're using it for storage as well as photoshop will be writing scratch data when it goes to compress the images (enabled by default). Worst case scenario spotlight will slow down the process even further if it's set to keep track of the raid.

I don't work with these file sizes everyday just once in a while.
The main reason I am building the RAID is primarily to hold my working files.
They have been sitting on a single drive till now but since I have a few of the same drives I
thought I would RAID them to get a little more speed out of my setup without spending more money.
I agree with more RAM but my slots are maxed at 4x8GB.
I would have to remove all my chips and buy new 16GB chips.
I have done a few tests with these drives set up in 3 drive RAID and got over 500MB out of them.
Of course that is best case scenario.

 

[thekev]

I don't work with these file sizes everyday just once in a while.
The main reason I am building the RAID is primarily to hold my working files.
They have been sitting on a single drive till now but since I have a few of the same drives I
thought I would RAID them to get a little more speed out of my setup without spending more money.
I agree with more RAM but my slots are maxed at 4x8GB.
I would have to remove all my chips and buy new 16GB chips.
I have done a few tests with these drives set up in 3 drive RAID and got over 500MB out of them.
Of course that is best case scenario.

I think it maxes out at 48 either way. If you're going through that much data, I don't think it would help enough. Just be aware that if you're storing working files on them, it does create a significant bottleneck on saves. I know CS6 seems to save out bigger files, so I'm not sure whether compression is still enabled by default. With CS5 you had to use a disableflatecompression plugin to enable this. In the case of uncompressed data you don't have a scratch disk bottleneck on saves. Spotlight can still slow things down a little on scratch disks, so I usually disable it on whatever volume or folder holds them if possible. The aforementioned bottleneck was that if photoshop is compressing the save, it will write a lot of data to the disk aside from just save data. It also depends on the cpu, but that's a different issue. I can't seem to find enough information on the current behavior.

 

[sngraphics]

I think it maxes out at 48 either way. If you're going through that much data, I don't think it would help enough. Just be aware that if you're storing working files on them, it does create a significant bottleneck on saves. I know CS6 seems to save out bigger files, so I'm not sure whether compression is still enabled by default. With CS5 you had to use a disableflatecompression plugin to enable this. In the case of uncompressed data you don't have a scratch disk bottleneck on saves. Spotlight can still slow things down a little on scratch disks, so I usually disable it on whatever volume or folder holds them if possible. The aforementioned bottleneck was that if photoshop is compressing the save, it will write a lot of data to the disk aside from just save data. It also depends on the cpu, but that's a different issue. I can't seem to find enough information on the current behavior.

Check this website out. It has helped me out immensely.
http://macperformanceguide.com/index_topics.html

Below is a link more specific to what you mention with disableflatecompression.
http://macperformanceguide.com/OptimizingPhotoshopCS6-SavingOpening.html

Yes in CS5 you had to use a plugin for this but it looks like Adobe built "Disable Compression of PSD and PSB files" into CS6 as an option you can check on/off.

----------

Yes, no problem with that at all. I use my 8-drive RAID 6 as scratch for Photoshop, and have used an internal 3-drive RAID 0 in my Mac Pro as well.

You have used these RAID setups for scratch and at the same time for other purposes such as working files?

 

[wonderspark]

You have used these RAID setups for scratch and at the same time for other purposes such as working files?

When I first started out, I only had the Mac Pro, and I used the stock 640GB HDD for OS and Program files, then used the three stock 1TB HDDs in RAID 0 for everything else. Scratch, working files, exports, and so on. I was only working with P2 MXF 1080p files at the time, and it worked fine for editing a feature-length film. Granted, that is "simple" footage.

Over time, I built it up to where it is today. Now I have scratch files broken into two volumes.
- Project Scratch (which appears to be preview files) is located on my very fast eight-drive RAID6.
- Preference Scratch (which is called Media Cache) is on a two-disk software RAID 0 in the Mac Pro. This used to be a three-disk RAID 0, but I needed one bay for my 4TB Hitachi HDD that I use for other things like music and data.

The thinking behind locating previews on the same volume as working media is that when you render files for previews, they're playing back alongside the regular media anyway, so it's all the same thing. It's media, playing from the media volume. On the other hand, the media cache files are not the same thing as preview or media, and therefore should benefit from being on their own fast volume. I don't keep them on the OS/Program volume (the default setting) because I believe in keeping the OS/Programs as clean and pure as possible.

With this setup, I've been able to play any 1080 footage, even DSLR footage (Canon 5D, H.264, 1080p) on the timeline, software acceleration only with my ATI 5870 GPU, without rendering the footage, despite the timeline being red or yellow. I've even thrown certain effects on it, and *still* played it smoothly without rendering. Depending on what and how many effects, it needs rendering to play smoothly, but it seems I'm doing something right in that I rarely render any footage to play it smooth on my dual monitor setup (30" ACD, with another Dell monitor as my main program window.) My RAID 6 is very fast, though. It moves data at over 700MB/sec both read and write.

 

[wonderspark]

If you're interested, my setup is as follows:
2009 Mac Pro (4,1) with firmware update to 5,1
3.33GHz W3680 6-core CPU
32GB 1333Mhz RAM, 8GBx4
ATI 5870 GPU in PCI slot 1 (I have a GTX 285, but it doesn't work as well in Adobe, believe it or not)
Areca 1880ix-12 RAID card w/1GB memory and BBU in PCI slot 2
Caldigit FASTA-6GU3 USB 3.0 and eSATA card in PCI slot 3
Crucial M4 256GB SSD in HDD bay 1, OS X and all programs
2x Apple/Hitachi 1TB HDD in HDD bays 2&3, software RAID 0, for scratch files
Hitachi 4TB HDD in HDD bay 4 for iTunes, Dropbox and all other data not on SSD.
LG 10x Blu-ray burner in ODD bay 1
Superdrive in ODD bay 2
Sans Digital TR8X 8-bay HDD tower with 8x WD2003FYYS 2TB RE4 HDDs in RAID 6, connected via two mini-SAS cables to internal ports on Areca card through space where PCI slot 4 cover is removed. Read=714MB/sec, Write=816MB/sec.

 

[sngraphics]

If a disk is split into two logical volumes (partitions) if you are accessing both partitions then by definition you are not "short stroking" the outer one.
Moving the disk head out of the "outer track" volume and into the inner volume (to read/write ) will impact the access times to data in the "outer track" volumes. It is a physical constraint that short stroking is trying to avoid. A general HDD has the following elements.

Image

or image http://en.wikipedia.org/wiki/File:Hard_drive-en.svg (from article
http://en.wikipedia.org/wiki/Hard_disk_drive )

Moving the head around is what drives up latency. By restricting the head movement only to the outer tracks it doesn't have to move as far. Shorter movements means faster average access times. If access data from anywhere on the disk then the head movements are the same as if there are no partitions/volumes on the same drive.


Blowing away the short stroke advantage by striping over RAID-0 doesn't really buy much back in most general cases on non-trivial sized files.

Partitioning Benefits/Weaknesses

" Benefits
....
Short Stroking", which aims to minimize performance-eating head repositioning delays by reducing the number of tracks used per hard drive
...
Weaknesses
...
Reduces overall disk performance on systems where data is accessed regularly and in parallel on multiple partitions, ... "
http://en.wikipedia.org/wiki/Disk_partitioning#Benefits_of_multiple_partitions

So even if just grabbing some small meta data from the "inner track" volume ( 32K of data) and most of the data from outer tracks ( 10,000K of data) merely diverting for the 32K kills the outer access average times.

Thank you for your detailed description.
I appreciate the time you took to put this all together.
Sorry for the delayed response, have been pulling some long hours with work.

So just to make sure I understand. (sorry, limited intelligence)
I will not see any benefits by short stroking the drives if the 3 drive RAID will be used for both Working Files and Scratch?

Like I said previously I can already get 500+MB (Max) out of this RAID.
How much more could I get anyway if I did short stroke?

So the bottom line here is: RAID the 3 drives and just start enjoying them!

 

[sngraphics]

Update!!!!

As for the RAID0, I'd partition my drives with a small 50GB partition, and RAID0 these partitions for the best performance. If you do it in Disk Utility, make sure that the 50GB partitions are on the top of the list.

Loa

UPDATE!!!!
Was on the phone yesterday with an Apple Senior Advisor.
Called just to double check on setting up my RAID up correctly.
I asked him about partitioning the drives first and then setting up two RAIDs.
A smaller RAID using the short stroked partitions and a larger RAID using the larger partitions.
He told me that technically I can do it but when I actually try to use this setup I could physically damage the drives.
Forgot all the exact explanation but basically he said that the drives would get "Confused" by having 2 different requests coming in and something could happen physically to the drives. (eg. to the heads, locked up etc)

I've never heard anything like this before.
Is he right or wrong?
Anybody have any thoughts or comments on what he said?

 

[wonderspark]

I think he's right, and I'm against partitioning any HDDs used in a RAID.

 

[Loa]

I could physically damage the drives.

First time I ever hear of this strange idea... I've been short stroking my RAID0s since I've been using RAID0s (couple of years now, using different drives) and I've never had a RAID or drive go bad.

Short stroking is a well known and widely used method of getting the most out of our RAID0s...

Don't let the cookie monster scare you.

Loa

 

[sngraphics]

I think he's right, and I'm against partitioning any HDDs used in a RAID.

First time I ever hear of this strange idea... I've been short stroking my RAID0s since I've been using RAID0s (couple of years now, using different drives) and I've never had a RAID or drive go bad.

Short stroking is a well known and widely used method of getting the most out of our RAID0s...

Don't let the cookie monster scare you.

Loa

I believe both of you.
But if by chance this Apple advisor is correct I don't really want to do anything to HDs that are pretty much new.

Also, like I said previously I can get around 550+MB/s (Max) off of this RAID.
(using diglloydTools Disktester, Black Magic & OWC's SpeedTools Utility)
How much more speed could possibly be gained with short stroking the drives before RAIDing?
Is it worth it?

As a side note/question.
When I use these other tools to measure the speed of the RAID I get these max speeds of 550+.
But when I use Apple's Disk Utility to do a Single or Seven Pass erase it is only getting speeds at around 190MB/s(writes).
Basically the speed of only one of the drives & not all three together, which is increasing the time for the erase by 3 times.
Why? Is something wrong?

 

[wonderspark]

I believe both of you.
But if by chance this Apple advisor is correct I don't really want to do anything to HDs that are pretty much new.

Also, like I said previously I can get around 550+MB/s (Max) off of this RAID.
(using diglloydTools Disktester, Black Magic & OWC's SpeedTools Utility)
How much more speed could possibly be gained with short stroking the drives before RAIDing?
Is it worth it?

As a side note/question.
When I use these other tools to measure the speed of the RAID I get these max speeds of 550+.
But when I use Apple's Disk Utility to do a Single or Seven Pass erase it is only getting speeds at around 190MB/s(writes).
Basically the speed of only one of the drives & not all three together, which is increasing the time for the erase by 3 times.
Why? Is something wrong?

Multiply 190 x 3, and there's your 550+... 570. It sounds like the single or seven pass erase is only writing to one disk at a time. I doubt anything is wrong, but rather it's just not coded to write to more than one disk at a time.

I've never bothered to write zeros to disks.

 

[sngraphics]

I've never bothered to write zeros to disks.

I don't really do this to specifically test for speed, more so to write over every block to map out any bad ones.

But it is interesting to watch the speeds in Activity Monitor while it is Reading/Writing.