iMac late 2015: should I get external SSD for media cache?

[ironicles]

Hey everyone, I've already posted a couple of times on different topics about my new iMac, but I promise this is the last one! On my last thread I asked a question towards the bottom but it got buried so I figured I'd make one specifically about this question. I'm going to be receiving a new late 2015 iMac 27 inch soon and have a few questions about storage. I've done some research but there are various answers and I really value the opinions from members of this site.

Here are the specs:

4.0GHz quad-core Intel Core i7, Turbo Boost up to 4.2GHz
AMD Radeon R9 M395X with 4GB video memory
32GB OR 64GB RAM from OWC, still undecided

I am a video editor. I use Premiere, After Effects, Photoshop, and a little bit of Maya as well as Element 3D. Most films I'll be making will involve 4k shots (from Phantom 3 UAV and GH4), most of the time involving creations in After Effects and sometimes involving 3D models in real life scenes, and most will be around 2-10 minutes long.

Now to the question:

1. 512GB internal SSD or 1TB internal SSD? I'm trying to decide how I'm going to go about this and here are my two lines of thinking:
A. Go with 512GB SSD internal for OS, applications, and media cache/go with RAID (something like OWC Thunderbay4 Raid 5 8TB) for media and renders.
B. Go with 1TB SSD internal for OS and applications, as well as currently-being-worked-on project, a cheaper/slower but large (say 12TB) external storage drive just to house older projects, and say a 256GB or 512GB external SSD for caches.
C. other

Is it bad to have an external SSD just for cache? Should it be on the internal?

I'm new to learning about caches, drives, hardware questions, etc. so don't be too hard on me if this stuff seems easy to you! Thanks, and I really appreciate your help!

 

[joema2]

Premiere CC can be quite sluggish editing H264 4K material, even on a top-spec 2015 iMac 27. It is a CPU issue not I/O or GPU so adding more I/O bandwidth won't help.

Your main choice is editing camera native material vs transcoding to a lower-compression codec such as ProRes. If you transcode, the editing performance will be much faster but it takes about 8x the space and I/O bandwidth will be more demanding.

I would not recommend external SSD just for cache or scratch files but maybe the more experienced Premiere editors could comment. Most of my editing is on FCP X, but I have done a lot of editing on CS6 and testing on CC. SSD I/O bandwidth is very high and you are generally not that I/O limited for these tasks so that would be squandering I/O capacity.

The Thunderbay 4 in RAID-5 array is pretty fast. I have edited three-camera H264 4K multicam using Premiere CC on a top-spec 2015 iMac 27 and 8TB Pegasus R4 which is overall similar in performance to the Thunderbay 4. This included material from the Phantom Vision 3 Pro and GH4. There was no problem on I/O but it was pretty sluggish due to high CPU load from Premiere's inefficient playback.

In general you don't want to keep older projects on primary storage because everything should be backed up and that increases backup time and cost for non-used material. From a budget standpoint remember you must back up everything, even if the media is on RAID-5.

The Thunderbay 4 is inexpensive, so I'm not sure going cheaper than that is worthwhile. Just remember it should be backed up.

 

[ironicles]

Premiere CC can be quite sluggish editing H264 4K material, even on a top-spec 2015 iMac 27. It is a CPU issue not I/O or GPU so adding more I/O bandwidth won't help.

Your main choice is editing camera native material vs transcoding to a lower-compression codec such as ProRes. If you transcode, the editing performance will be much faster but it takes about 8x the space and I/O bandwidth will be more demanding.

I would not recommend external SSD just for cache or scratch files but maybe the more experienced Premiere editors could comment. Most of my editing is on FCP X, but I have done a lot of editing on CS6 and testing on CC. SSD I/O bandwidth is very high and you are generally not that I/O limited for these tasks so that would be squandering I/O capacity.

The Thunderbay 4 in RAID-5 array is pretty fast. I have edited three-camera H264 4K multicam using Premiere CC on a top-spec 2015 iMac 27 and 8TB Pegasus R4 which is overall similar in performance to the Thunderbay 4. This included material from the Phantom Vision 3 Pro and GH4. There was no problem on I/O but it was pretty sluggish due to high CPU load from Premiere's inefficient playback.

In general you don't want to keep older projects on primary storage because everything should be backed up and that increases backup time and cost for non-used material. From a budget standpoint remember you must back up everything, even if the media is on RAID-5.

The Thunderbay 4 is inexpensive, so I'm not sure going cheaper than that is worthwhile. Just remember it should be backed up.

Thanks for the great reply! For me, what would your recommendation for hard drive set ups be?

 

[joema2]

Thanks for the great reply! For me, what would your recommendation for hard drive set ups be?

Based on your previous posts, you indicate that around 8TB RAID-5 array is big enough. However note the unformatted "name plate" capacity is less on RAID-5, so this would be around 6TB usable. For the iMac system drive either a 512 or 1TB internal SSD would work.

To back it up you'd need an approx. 8TB actual capacity HDD, and to get it done quickly enough maybe RAID-0. This 8TB G-Tech G-RAID is about $600:
http://amzn.com/B00846Z4YY

For archival backup and misc storage you'll probably need several 1TB, 2TB or 4TB USB bus-powered drives. The 1TB HGST Touro S is among the fastest single-platter bus-powered USB portable HDD: http://tinyurl.com/o4xegwb

The 2TB Seagate Backup Plus Slim is not quite that fast but is 2x the storage: http://amzn.com/B00FRHTSK4

The 4TB Seagate Backup Plus Fast may be the fastest USB bus-powered portable HDD: http://amzn.com/B00HXAV0X6

 

[ironicles]

Thanks So you think the recommended way for me to go would be 512GB for OS apps and cache, and then external RAID for media?

How much slower would things be if I did cache and current media on a 1TB fast internal?

 

[joema2]

Thanks So you think the recommended way for me to go would be 512GB for OS apps and cache, and then external RAID for media?

How much slower would things be if I did cache and current media on a 1TB fast internal?

It would likely not be any slower to put everything on 1TB internal SSD. The I/O bandwidth of the 2015 iMac SSD is extremely high -- 1,500 to 1,800 megabytes/sec. However you only have enough space for a modest amount of 4K. If you have to transcode to ProRes, you will have even less space.

So there is nothing wrong with trying it, and 1TB will give more "elbow room" even if you later use an external RAID array.

 

[ironicles]

Appreciate it joema2! Lots of people are saying it is imperative to put cache on separate drive than media, but lots of those posts are a year or 2 old. So you're saying this current iMac SSD is fast enough for it to not be an issue?

Would it be as fast, faster, or slower if say I put the cache on an external SSD RAID like this: http://www.bhphotovideo.com/c/produ..._J2_256Gb_2X128_SSD_Portable_Thunderbolt.html

 

[joema2]

Appreciate it joema2! Lots of people are saying it is imperative to put cache on separate drive than media, but lots of those posts are a year or 2 old. So you're saying this current iMac SSD is fast enough for it to not be an issue?

Correct. People are not accustomed to a boot drive with the performance if the 2015 iMac SSD, so most of the previous advice does not apply.

It is primarily a space issue not a performance issue. Anyone can see this themselves by simply doing the editing operations in question and observing the I/O rates with a monitoring tool like Activity Monitor or iStat Menus. The I/O rates just aren't that high for H264 material, even at 4K resolution.

If you transcode to a lower compression codec like ProRes, then everything changes -- it's 8x the size so data rates will be 8x higher. But because of the larger size you are less likely to have sufficient room on the system drive, no matter how fast it is. In that case you might need external storage for space reasons but not necessarily I/O bandwidth reasons.

Editing H264 4K on Premiere is most frequently CPU and GPU limited, not I/O limited. The Mac CPU and GPU are fixed commodities -- they cannot be upgraded. Once you get the highest configuration, that's all you can do for those items. The only remaining solutions are transcode at full resolution to ProRes, etc, or transcode to 1/4 res proxy files, which is a big hassle on Premiere since there's no built-in proxy management.

....Would it be as fast, faster, or slower if say I put the cache on an external SSD RAID like this: http://www.bhphotovideo.com/c/produ..._J2_256Gb_2X128_SSD_Portable_Thunderbolt.html

It is unlikely to be significantly faster. A chain is only as strong as the weakest link. For H264 material the weak link is typically CPU and GPU performance, not I/O. Strengthening an already strong link (I/O) will not make the chain any stronger.

Premiere on a top-spec 2015 iMac 27 can generally handle one stream of H264 4K with decent editing performance, if the program monitor is at 1/4 res. It is not lightning fast but it's livable. If you have to edit two 4K streams (e.g, multicam), IMO it's just too slow for my taste but people have difference tolerance levels for responsiveness. That sluggish performance is primarily due to an efficient playback engine code path that results in very high CPU usage. In that case transcoding to ProRes will make it much faster, at the cost of 8x the space and I/O rate. Transcoding to 1/4 res proxy files will make it even faster at the cost of increased complexity managing the proxy files.