Storage Solutions for our OUT-OF-CONTROL iTunes Libraries

[MagnusVonMagnum]

Ok, I've got my two new Seagate 7200RPM 1.5TB Barracudas installed internally in my 2001 Era (but substantially upgraded) PowerMac (which is my whole house audio/video iTunes server to multiple AppleTV and Airport Express units). Well, the primary drive is partitioned and loaded up with the previous 500GB drive software. The 2nd drive still needs partitioned (just a mirror of the first) and then backed up with the primary's contents.

I've been putting the finishing touches on my iTunes library move/update. With the new setup, I moved all my media (photos, iTunes libraries including movie, music video and lossless folders and other various media storage) over to a new "Media" partition so that any restore of my system drives won't involved copying hundreds of Gigabytes of media to fix a simple system crash/problem. Even so, I ended up giving the Leopard partition 300GB (Tiger got 40GB, OS9 got 3GB) so that the Media drive would be exactly 1TB. This will let me take my two old 500GB drives and RAID them for a 1TB network secondary backup (so I have two backups of my media since I NEVER want to have to dump/encode all those DVDs and CDs ever again). I can always cut into the Leopard extra space if needed (I've got extra space to backup on my MBP's external backup drive if required).

Having 4 partitions on the drive went fine. I tested the drive speed with Xbench to compare with the old WD 500GB Green Drives. The old drives topped out around 70MB/sec in Leopard (80MB/sec in Tiger). I haven't tested it in Tiger yet, but in Leopard the new Seagate drives get 100MB/sec sustained reads and 85MB/sec sustained writes. My SATA controller is SATA 1 for the PowerMac, but given this drive doesn't even come close to maxing out the SATA 1 specs (~180MB/sec), I don't know what the big deal about SATA 2 is supposed to be at the present time (unless you RAID 0 some really fast drives or something). The drives seem just as quiet as the Green WD drives (that is to say I can't hear them above the case fans of the PowerMac normally). I did have one freeze this morning after transferring over 650GB of information while transferring the last of my handbrake movie encodes from my PC across the gigabit network. I have no idea what happened there. No panic, just a total freeze of the OS. I've seen Leopard freeze before for reasons unknown so I don't automatically suspect the hard drive is the culprit. Even my Ocotober purchased MBP has had a freeze and even a couple of kernel panics. OS X is anything but perfect. Assuming no more major issues, I'm pretty pleased with the drives' performance and noise level not to mention the price (both drives together cost less than one WD 2TB drive and are considerably faster).

Edit:

I've done the Xbench tests in Tiger with the new drives. I'm getting 93MB/sec write speeds and 102MB/sec read speeds. It's a shame that Leopard lost some ground in those areas. I was hoping Snow Leopard might have restored some of the losses, but it will probably not be PPC compatible so it won't matter for my server regardless, most likely.

 

[emase]

Another storage crazed posting

So I have a similar problem of everyone here with a slight twist.

I have a mac pro (dual quad core Intels) with four 1.5 TB drives. I have concatenated two of them to make a virtual 3TB drive and then mirrored that to the other two concatenated drives. I have an appletv in the living room, however the wifi signal between my mac pro and appletv is spotty. So when I am watching movies, sometimes it is fine, other times it is annoyingly inconsistent. So I bought a Synology NAS box and filled it with two 1.5TB drives to match what I have in the tower. The idea being I could then copy (at least) my itunes library to the NAS box and serve it locally via a swithc to my appleTv.

What I am looking for is some way to easily clone my itunes library automatically to the NAS drives. I tried Time Machine but it doesn't recognize the NAS shares. I could use some backup softwate but most use some form of compression that renders the files unreadable by the NAS itunes server.

Thoughts?

 

[belvdr]

Check out rsync. It will copy the files, and if for some reason the copy gets interrupted, you can start again, and it will only copy the files needed.

Here's a tutorial:

http://www.egg-tech.com/mac_backup/

 

[mpshay]

What I am looking for is some way to easily clone my itunes library automatically to the NAS drives. I tried Time Machine but it doesn't recognize the NAS shares. I could use some backup softwate but most use some form of compression that renders the files unreadable by the NAS itunes server.

Thoughts?

Can you sync an ATV to a NAS? Thought it had to be a computer running iTunes

 

[starcat]

So I have a similar problem of everyone here with a slight twist.

I have a mac pro (dual quad core Intels) with four 1.5 TB drives. I have concatenated two of them to make a virtual 3TB drive and then mirrored that to the other two concatenated drives. I have an appletv in the living room, however the wifi signal between my mac pro and appletv is spotty. So when I am watching movies, sometimes it is fine, other times it is annoyingly inconsistent. So I bought a Synology NAS box and filled it with two 1.5TB drives to match what I have in the tower. The idea being I could then copy (at least) my itunes library to the NAS box and serve it locally via a swithc to my appleTv.

What I am looking for is some way to easily clone my itunes library automatically to the NAS drives. I tried Time Machine but it doesn't recognize the NAS shares. I could use some backup softwate but most use some form of compression that renders the files unreadable by the NAS itunes server.

Thoughts?

First, you better mirror and then concat the drives and not the other way round. Then, you could indeed use NAS shares with Time Machine (this is also done with Leopard Server to backup remote clients). You just need to create a sparsebundle (newer type kind of sparseimage file) which contains the Eth address of the client to be backed up. The mount must be afp and not smb or the like. Google this up. The other thing is you can create a share on your Mac Pro and put all the iTunes lib data files in there, export this share (like on a NAS). Then import the library on all computers (non managed) and replicate only the xml itunes lib description file between all computers (automatically like using cron via rsync or by any other means). You edit your master iTunes library on the Mac Pro, then replicate only the xml file and all other clients see the same changes/library. No need to replicate all the data files as they are available networked to all clients, regardless if you have them on a NAS or a share on your Mac Pro, the share name is the same for all clients, thus the xml description is ok for all of them. Have fun.

 

[starcat]

Can you sync an ATV to a NAS? Thought it had to be a computer running iTunes

Lot of the NAS boxes (like Qnap) have integrated iTunes server, i.e. they present themselves as being an "iTunes computer". This is why a Qnap or equivalent NAS box is way better than a Drobo. Take a look at the TS-509 Pro, the TS-639 Pro, etc. The Irfant Netgear boxes have this functionality too. You can get 70 MB/s sustained out of a single Gigabit port on a TS-509 Pro and it has got two of them. Not so with the DroboShare.

 

[pilotError]

Can you sync an ATV to a NAS? Thought it had to be a computer running iTunes

You can use an NAS as a Data Store (I know it sounds dumb).

Put the Library on the NAS and Alt-Click the iTunes on the Dock. iTunes is pretty good at finding the library (I have mine mounted via AFP).

In addition, like the above poster said, my readyNAS has an iTunes server that shows up as a shared library. It's not useable with an ATV though, since you can't validate using the code that shows up on the ATV when you select a library.

 

[mpshay]

Lot of the NAS boxes (like Qnap) have integrated iTunes server, i.e. they present themselves as being an "iTunes computer". This is why a Qnap or equivalent NAS box is way better than a Drobo. Take a look at the TS-509 Pro, the TS-639 Pro, etc. The Irfant Netgear boxes have this functionality too. You can get 70 MB/s sustained out of a single Gigabit port on a TS-509 Pro and it has got two of them. Not so with the DroboShare.

I know about using for music as an itunes server. I'm specifically asking about the ATV and associated movies

 

[MagnusVonMagnum]

I know about using for music as an itunes server. I'm specifically asking about the ATV and associated movies

For that kind of usage, I'd personally prefer something like a Windows Server based NAS (e.g. HP Media Share) or even using an old Mac as a server (if you want OS X running things). The latter is what I do. I can also use it as a secondary computer for things like secure shopping, banking, etc. It's running 24/7 and has two 1.5TB drives in it that run very fast compared to most NAS units (I'm getting 83MB/sec writes and 100MB/sec reads in Leopard and 93/103 in Tiger). The advantage here with either the Windows or OS X method is that you can run the full blown iTunes application, which WILL let you serve AppleTV units (and iPhones/Touches also). You can easily run a UPNP server as well, Squeeze Center or any other serving software.

Basically, ALL NAS boxes are computers with storage and network interfaces (most minus the video output and keyboard/mouse input capability). Just because the boxes are generally smaller than a typical PC tower (not always if you want lots of storage space), that doesn't really change the fact they are really computers, which is also how they can run things like an iTunes server in the first place. The reason most suck for iTunes + AppleTV is that they are running a variation of Linux which only has reverse engineered iTunes server support and is incomplete to say the least. iTunes itself won't run on Linux (save via Wine, which only works with older versions and VERY VERY slowly) and so you don't get complete support. Thus, a Windows or OS X based server is going to offer the superior experience for such a setup. The latter is BSD/Unix based so Linux really has no advantages there (I can't really say how good Windows Home Server is since I've never personally usd it, but it is still Windows, after all).

 

[emase]

Synology NAS continued

So I used rsync to get all of my itunes library to the NAS box, but the issue I now have is that when I try and link my ATV to the NAS, it does that thing where you have to put in the code to get the ATV to sync to the itunes library. Of course the Synology itunes server does not have this functionality so I now (as I read) need to hack the ATV to get it to support ssh and then I can run a small server that will work around the itunes password thing. Phew!

Anyway, I am (slowly) getting there but if this isn't proof of why ATV needs a major rethink, not sure what is!

 

[Chad H]

Hey guys,
I have a dilemma. I did have a Mac Pro with all hard drive bays filled up. The last 3 were for media of course. I have 600 movies and even more tv shows, and music(2.3TB worth). Now the dilemma, I have had to sell my desk and Mac Pro and have the computer listed in my signature. What can I do to get these movies to my apple tv? I tried to think of every solution.

 

[bohbot16]

Hey guys,
I have a dilemma. I did have a Mac Pro with all hard drive bays filled up. The last 3 were for media of course. I have 600 movies and even more tv shows, and music(2.3TB worth). Now the dilemma, I have had to sell my desk and Mac Pro and have the computer listed in my signature. What can I do to get these movies to my apple tv? I tried to think of every solution.

How about a Drobo connected to your Time Capsule

 

[eegger]

How about a Drobo connected to your Time Capsule

He would lose the data on the drives, when the Drobo initializes them.


Just go get some external cases for the HD and hook them to the Time Capsule

or

Go buy 2 1.5TB HD's and put them in a JBOD 2 hard drive enclosure and use the old HD and a backup.

 

[pjfla]

I have been following this thread for several weeks now and decided to finally chime in with a couple of questions.

My setup: 2009 Mini with two 1TB WD MyBook Studio Edition External HDs.

I have about 600gb of movies/music/photos on one MyBook and use Time Machine to have the 2nd MyBook back up the first. This seems relatively safe to me... is it? Two ATVs connected to this set up.

Also, my wife does a lot of IMovie work. She is frustrated that she has to actually sit at the Mini to do the work (vs using our MBA or MacBook). How can we fix this for her? Could I just plug the two MyBooks directly into my AEBS?

I've ready a lot about DROBO and Firefly but not quite sold. Im open to all suggestions. I guess it really boils down to two issues: Safe backups which I think Im ok, and some sort of NAS. Im not too concerned about cost, but more concerned about reliability and being able to access our "stuff" from anywhere in the house.

THANKS!!

 

[Anonymous Freak]

Heh. I'm down to 2 GB of free space on the external hard drive I keep my iTunes library on. And this is after moving all other data off that drive a couple months ago. It's *JUST* iTunes now; on a 500 GB drive.

FWIW, this may seem a bit over the top, but I have thrown out my CDs. All 1500 of them. These Apple Lossless files are just fine.

I have all mine in boxes in the basement. I ripped 'em all to 192 kb MP3 back in '2003; about two weeks before Apple announced AAC support in iTunes. A hard drive crash in early '05 had me re-rip them to 128 kb AAC, which is what they are still. I don't bother backing up what I own physical media for; so I keep the physical media as the backup. I'm sure I'll end up re-ripping them as Lossless next time I have a hard drive failure.

Likewise, I started ripping my DVDs as 768 kbps 320x240 MPEG-4; now I've moved to 640x480 1.5 Mbps H.264/AVC. Soon I'll likely move to dual-encoded, 720x480 anamorphic AVC at 2.5 Mbps plus 480x320 1.5 Mbps, if I can figure out how to make iTunes store them as 'one' the way it does with purchased HD/SD content.

Next I will start ripping my HD DVDs, once I find a good simple workflow for that. My present workflow is very interactive, and takes a long time; sometimes not finding out until two man-hours of interaction that I have created a bad file. (Audio out of sync is the most common problem.) I'll probably go for 1280x720p/5Mbps AppleTV-compatible rips, unless a 1080p-capable AppleTV comes out soon. (I don't have an HDTV yet, so no AppleTV, but I plan on getting both this Summer or next, after a house remodel.) If a 1080p-capable AppleTV comes out, I'll try to transcode to keep equal quality.

Once I start doing the HD DVDs, it's almost guaranteed I'll end up getting a Drobo. (I have a computer combo Blu-ray burner, HD DVD drive; I've just been getting HD DVDs because they're cheap now. )

 

[srl7741]

I have been following this thread for several weeks now and decided to finally chime in with a couple of questions.

My setup: 2009 Mini with two 1TB WD MyBook Studio Edition External HDs.

I have about 600gb of movies/music/photos on one MyBook and use Time Machine to have the 2nd MyBook back up the first. This seems relatively safe to me... is it? Two ATVs connected to this set up.

Also, my wife does a lot of IMovie work. She is frustrated that she has to actually sit at the Mini to do the work (vs using our MBA or MacBook). How can we fix this for her? Could I just plug the two MyBooks directly into my AEBS?

I've ready a lot about DROBO and Firefly but not quite sold. Im open to all suggestions. I guess it really boils down to two issues: Safe backups which I think Im ok, and some sort of NAS. Im not too concerned about cost, but more concerned about reliability and being able to access our "stuff" from anywhere in the house.

THANKS!!

The quick and easy way without spending any money would be to share the one 1TB HD so she can have access to it from her Notebook and place all the files she needs on that drive.. Putting a Drobo on your network would also be another solution down the road and of course sharing it for anyone else on the network. Keeping it simple is sometimes best.

 

[Advance The Man]

The quick and easy way without spending any money would be to share the one 1TB HD so she can have access to it from her Notebook and place all the files she needs on that drive.. Putting a Drobo on your network would also be another solution down the road and of course sharing it for anyone else on the network. Keeping it simple is sometimes best.

I have a 2TB MyBook (so approximately 1TB to use) - have 350GB of media and this seems to be a perfect setup for me. Can someone lmk if I'm missing something? Keep hearing about drobo - but I don't understand why more wouldn't go with a similar setup as mine b/c of cost. You can get a 2TB for less than $250 now.

 

[srl7741]

Drobo

I have a 2TB MyBook (so approximately 1TB to use) - have 350GB of media and this seems to be a perfect setup for me. Can someone lmk if I'm missing something? Keep hearing about drobo - but I don't understand why more wouldn't go with a similar setup as mine b/c of cost. You can get a 2TB for less than $250 now.

The Drobo is just one of many ideas, it's pretty straight forward and easy to use and has a small foot print over all. You can add drives to it as you need more space and it seems to be appealing to a lot of people needing several TB of HD space. You correct in saying there are many more ideas that are much less $. Visit the Drobo web site and watch some of the videos it will help see why it's appealing for some people.

Drobo

 

[prostuff1]

I have a 2TB MyBook (so approximately 1TB to use) - have 350GB of media and this seems to be a perfect setup for me. Can someone lmk if I'm missing something? Keep hearing about drobo - but I don't understand why more wouldn't go with a similar setup as mine b/c of cost. You can get a 2TB for less than $250 now.

The main appeal of something like the Drobo is that you can add more storage as you need it, instead of daisy-chaining drives one after another on your computer.

I built myself a server based around the unRAID OS and have not looked back. The only external I have is the one that I backup to weekly, and that is my bootable backup. I also have my home folder backed up to my server each week… just in case.

The thing that made me go the unRAID root was that I can pull the drives out and get to the data from any other computer. With Drobo, if the machine dies you are more then likely SOL for getting the data back; you are protected from a drive failure but not a machine failure.

 

[MagnusVonMagnum]

The main appeal of something like the Drobo is that you can add more storage as you need it, instead of daisy-chaining drives one after another on your computer.

How is a Drobo "not" daisy-chaining drives? So it puts them all in one unit. They are still being chained together. The one thing it does do is pool them together as one big virtual drive (although RAID 0 can do something similar internally or externally, you cannot 'add' to it over time). Of course, if Apple ever gets around to offering ZFS for the ordinary user, that point will become moot as well as pooling is a part of its spec and Drobo will be largely moot once again.

My biggest problem with Drobo (other than being slow as molasses even with FW800) is that I don't trust it. I've heard about too many Drobo units breaking and that takes ALL your data with it. I only trust completely separate drives. I back up one 1.5TB drive to another 1.5TB drive and both are bootable (CCC). If one drive dies, I simply replace it and copy over from the other one again. I'm unlikely to lose data. With a 3rd backup externally and the ability to take it off site, I'm virtually guaranteed to not lose data. And I can get all three drives for less than one drobo with 2 drives AND I will get 80MB/sec writes and 100MB/sec reads to boot.

 

[prostuff1]

How is a Drobo "not" daisy-chaining drives? So it puts them all in one unit. They are still being chained together. The one thing it does do is pool them together as one big virtual drive (although RAID 0 can do something similar internally or externally, you cannot 'add' to it over time).

the Drobo does not daisy chain drives together. It creates one big pool for the drives that are in the system. By daisy chain I mean connecting one drive to another via firewire. The drobo does something very similar to a RAID5 in that it spreads its parity (the part that provides protection) over all the drives in the system. The only difference is that it allows you to add drives of any size, unlike RAID5 that needs drives of the same size.

My biggest problem…

I agree with most of your points. I went with unRAID because it allowed me to remove the drives and read them in a different system if need be. They are formatted with the RFS and with almost any linux system can be read and written to. unRAID also allows for data protection and expandability using drives of any size. I have my server (which is a full sized computer) set up in my basement and out of the way. There are 2x500GB drives, 2x750GB drives, 1x1TB, and 2x1.5TB drives in the system. One of the 1.5TB drives I lost because of the parity that is used to protect all the other data, but it is well worth the cost. It gives me a total of 5TB of space. I have an older 160GB drive that I use as a CCC drive and my backup boot drive if something happens.
Now the question is: if you had all those drives in firewire enclosures how much of a pain the arse would that be to plug into a laptop every time you wanted to do something with any given drive? I felt that spending the ~$350 on the components for the server (not including drives) was worth the cost so I did not end up with external drives everywhere.

 

[pprior]

Are you hooking to your "unraid" via ethernet? If so, what kind of read/write speeds do you get in real life?

 

[emase]

Drobo vs. Synology

I have seen a lot of people reccomending the Drobo. Has anyone compared it ot the Synology? I have the Synlogy and besides that it can't register a link with iTunes for sharing (harly it's problem) I really like it so would be curious what the Drobo has that the Synology doesn't. Maybe that the Synology is fairly immatre to teh market.

 

[pprior]

I have owned (and still do) a traditional nas like the synology (I use the readynas+). From a brief overview of the site, it looks like a standard nas appliance with some nice features.

I think the main selling point of the drobo is you can stick whatever (sata) drive you want in it, and it just works. It's much simpler to admin. Doesn't mean it's right for everyone, but personally I've run a file server, used a NAS, and also own a drobo and of all of them, I like the drobo the best by far. I wish it had better file speeds, but for a time machine volume, I can live with it.

 

[Donar]

...
I have the Synlogy and besides that it can't register a link with iTunes for sharing (harly it's problem) ...

The Synology is able to act as a iTunes music server, it should show up in itunes if you turn on the server. Maybe also enable afp protokol.

I have a question to the people who own a WD Share Space: Can anybody tell me how upgrading a 4 TB (4x1TB) Share space to 8 TB (4x2TB) will work. Can you expand the volume, or do you have to erase everything and then start from scratch? The Share space is appealing because of it's low price, the Drobo sounds nice because you can upgrade over time...