Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

[Disaster] Areca ARC-1880LP all RAID-5 volumes destroyed by Mac OS X 10.7 Lion

solaris8x86

solaris8x86

macrumors regular
Original poster
Nov 24, 2007
235
64
Saturn
Today, is a doomsday of my data on my Mac.:(

The most destructive Operating System - Mac OS X 10.7 Lion ​

I got 3 x RAID-5 volumes (12 x 2TB) on my Mac Pro OS X 10.6 formed by a Areca ARC-1880LP HW RAID Controller and they were running fine. I did something stupid to upgrade my Mac to 10.7 Lion Developer Preview-4 beta build on last night for a trial run. After finishing installation. All of my existing RAID-5 volumes crashed. I fallback to 10.6 immediately and realized all of RAID-5 volumes are crash in a disaster manner. Snow Leopard prompts up after boot up: "To back up all your data on all disks immediately, and to reformat them asap", and tried that Disk Utility couldn't fix them. Each RAID-set is having at least 1 disk crashed on RAID configuration shown in Acrea's configuration page; and all arrays are running in "Degraded" mode. If any one more disk is crash. All data would long gone.

I am now spending my best effort to rescue the RAID-sets with Areca Support. I am here to share this doom-feel experience that I have with all Mac Pro users. Do NOT upgrade to 10.7 Lion without a proper RAID card driver installed. Otherwise, disaster like this may come true without warning.

Doomsday man... my doomsday in my I.T. life.....:( 18TB data stored for 10 years now may long gone......
 
Last edited:
solaris8x86 said:
Doomsday man... my doomsday in my I.T. life.....:( 18TB data stored for 10 years now may long gone......
Click to expand...


I feel for you, I really do. I've lost large (for me at the time) RAID sets before.

That being said, RAID is not a substitute for backups or secondary copies. What would happen if your system were stolen or lost in a fire? Or if a ARECA bug happened to crash your data?

Worse yet, "bit rot" occurs on disks, and RAID will not discover or rebuild that for you. Only systems that store CRCs with data (such as high end disk arrays, or ZFS file system for example) can protect against bit rot.

Sitting with several TB of data that is not backed up is a disaster in itself.

Again, I'm not trying to be mean, and I do feel for you. Been there, done that, but not with irreplaceable data.
 
bplein said:
I feel for you, I really do. I've lost large (for me at the time) RAID sets before.

That being said, RAID is not a substitute for backups or secondary copies. What would happen if your system were stolen or lost in a fire? Or if a ARECA bug happened to crash your data?

Worse yet, "bit rot" occurs on disks, and RAID will not discover or rebuild that for you. Only systems that store CRCs with data (such as high end disk arrays, or ZFS file system for example) can protect against bit rot.

Sitting with several TB of data that is not backed up is a disaster in itself.

Again, I'm not trying to be mean, and I do feel for you. Been there, done that, but not with irreplaceable data.
Click to expand...


I would like to commit suiside today. Great depression. My mood is now N times poor than the economics situation of United State of America. Any prefer ways?:( (which way should I die for)
 
DesmoPilot said:
Wait, so you were using RAID as a backup?
Click to expand...

They are Primary data bank. Not all of them having a backup snapshot in Time Machine. My life is expendable. My data is not.:(

**** me to trust HW RAID. **** me!
 
Last edited by a moderator:
Not trying to be mean but why would anyone run beta software on their production or primary computer.
If it were me I'd pile those drives into an external NAS or two.
 
I would add running a raid without backups to beta software on a mission critical workstation...

I'm truly sorry, that's horrible. :(
 
solaris8x86 said:
The most destructive Operating System - Mac OS X 10.7 Lion

Do NOT upgrade to 10.7 Lion without a proper RAID card driver installed
Click to expand...

I wouldn't blame the OS at this point especially since the drivers (Dec 2010) from Areca were obviously never tested on Lion. I think the warning should be
a RAID is not a backup.​
Doesn't matter what the OS is.
 
Why on earth were you running a beta on a system with production data?

Where is that Picard facepalm...
 
solaris8x86 said:
Today, is a doomsday of my data on my Mac.:(
Click to expand...
Ouch. :(

As I and others have stated many times over, running a RAID, no matter how redundant it's designed to be, is foolish without a proper backup system. Life happens, and you're learning this the hard way.

Worse yet, is you tried out a Beta version of an OS. No IT dept. would do this, as it can totally bork a system, which you've discovered first hand.

Even with a backup system in place, getting it all back up and fully operational takes time, and that tends to translate into money (think of a commerce site, or worse, a bank that can't process a single transaction for days). Careers, and even entire companies, can be ruined by something like this.

solaris8x86 said:
Fucx me to trust HW RAID. Fucx me!
Click to expand...
Unfortunately, this was the result of very poor judgment on your part, not the hardware. :(

At least it should be a learning experience (should make you remember to never do it again). ;)

Now given what's happened, let's assume for the moment the disk the card has indicated as Failed is fine. Removing it and replacing it may be enough to bring the array back online (it could begin a rebuild process). Another bit of hope (just in case it doesn't show up as healthy or begin to rebuild), is that Arecas keep a copy of the partition tables in the firmware, which allows them to recover some situations other cards cannot do (cards that don't keep copies of the PTs in it's ROM; just the disks). This requires a command to the card (it's a hidden feature), but Areca will walk you through this if necessary.

Now if the disk actually is bad, then you'll need to replace it. As the array is still operational (currently running in degraded mode), replacing the disk would cause the card to automatically begin rebuilding the array. I'd suggest letting it finish this process before trying to do anything else on the computer (keeps you from adding any further stress to the disks during the rebuild process). If this doesn't work, this is where the hidden recovery feature mentioned above would have to be attempted (last chance to get the data back).
 
DriveSavers

With this much data, though, selling a kidney probably wouldn't cover it.
 
solaris8x86 said:
Each RAID-set is having at least 1 disk crashed on RAID configuration shown in Acrea's configuration page; and all arrays are running in "Degraded" mode. If any one more disk is crash. All data would long gone.

18TB data stored for 10 years now may long gone......
Click to expand...

I'm not sure why nobody has recommended this yet... Get 9 2TB hard drives and start backing up!
 
I do feel for you. I've lost data twice before, not having backups in place. Some of it was very personal, important to me, and irreplaceable.

1) The first time I lost data, I had no back ups. That taught me to back up important data.

2) The second time I lost data, I had actually been backing it up. So I went to the backups only to find that not a damn one of them worked, so nothing could actually be recovered. So that taught me that I'm not really backing up unless I occasionally test the recovery process to verify it works.

We've all seen many stories about #1, but I've also seen a few about #2. TEST YOUR BACKUPS.

P.S. From my own experience as well as years of reading forums, I feel RAID is a bag of hurt. Yes there are some pros, but they come with a lot of cons as well, some of which are unacceptable to me.
 
Another thing to keep in mind is OS upgrades frequently kill disks that were on the verge. An OS upgrade is stressful. The boot disk is writing TONS of data, moving blocks, Spotlight indexes are being rebuilt, etc etc.

In IT I frequently saw disks die right after an upgrade/re-image. People would usually blame it on a bug in the new system, but honestly, it was always on older disks that were likely in borderline condition, and the stress of that I/O was enough to finally do them in.
 
I'm glad I did not upgrade to Lion seeing as it is not even out yet.
I crashed a SAN at work with Lion beta 1. That was the end of my "Hey this is super OK and stable" foolery. I feel your pain but no backups and beta software is pebkac of the highest order.
Telling people on a forum full of geeks takes some balls though.
 
After spending my best effort overnight. I have successfully rescued all data. Since all RAID-5 volumes are in "handicap-condition" but they are still allowing me to copy-out the data. All data are copied to another provisional storage. Then I used to recreate the RAID-5 diskgroup from scratch and restore the data back to the newly created RAID-5 disk groups and volumes. All are ok after a "Die Hard like" overnight rescue operation. Jesus christ!:D

The conclusion is Mac OS X 10.7 Lion Developer Preview 4 will destroy all Areca RAID-5 volumes with its default build-in OS driver (Which means Areca RAID card is now bootable in Lion environment. If they are RAID-disks, Lion will split them off unexpectedly and breaking the RAID configuration in the hard drive sectors - The situation like mine). To rescue these RAID-5 volumes destroyed by Lion. All RAID-5 disk groups MUST BE RECREATED thru Areca's configuration page that to erase all damaged RAID information on the disks. So users who install the Lion with a Acrea RAID volumes attached during the installation process will suffer the same disaster like mine. This is a global disaster because many users have the same hardware configurations like mine and pending to upgrade to Lion. They will step on the same trap during the Lion's installation process. This is unavoidable because it is a build-in OS driver. Who proceed the OS installation. Who will be trapped. Damn it, Steve Job shouldn't distribute the disaster in the WWDC forum this month.:mad:

I am now working with Areca official Support to test and report the bug to Apple priors to Lion release. Otherwise, million of users will suffer this disaster on next month. I am now testing Areca's latest beta driver for Lion (They just provide me this morning).:D

I am reborn now...!!:apple:
 
Last edited:
good to hear you saved it all. I once screwed up the filesystem on one drive and I was ready to commit suicide. I rescued it using a drive rescue in windows thankfully but I cant imagine that happening with 18tb! Im currently backing up to a 1.5tb drive in my system but I might get a 2tb TC and add a 3tb usb drive to it.
 
Good to hear you got the data back.

As foolish (not judging) as using beta software w/ a RAID setup man, I think there's a silver lining here, and that would be that the OP potentially saved potentially thousands of other users of the same potential fate - or worse - and it's one more bug that's being ironed out to ensure Lions release is proper.

Kinda mind blowing to know that the OP is helping Apple and the community on a beta software level getting Lion all ironed out you know like giving Apple an even better, more complete OS to be released, and on an issue that Apple may have not even looked at before.

But yeah I think it's always just risky/dumb move - and I've made plenty of dumb moves in my life etc. - to try pre-release software w/ sensitive data etc. I know I've done risky/dumb moves like that but that's why back ups are important lol :D


Let us know how the drivers work out yeah?
 
Let us know how the drivers work out yeah?
Click to expand...


The latest Areca Beta Driver for Mac OS X 10.7 Lion (not yet available on their website) will be tested a couple hours later after I finish to restore all data into the newly created RAID-5 disk groups (18TB = 2TB x 12 HDDs = RAID-5 volumes x 3 (4 x HDDs per RAID-5 set)). It is top priority. After that, I will test the driver and report the bugs to Apple and Areca.:D

But I do believe users who haven't see this thread will kick into the same trap. It is still a global scale disaster (Doomsday is still coming to look for you:D).
 
Last edited:
Glad you got it back. I'd stick with Snow Leopard till Areca gets their drivers straightened out and Lion is well out of beta. After all what's the rush?
 
Its stories like these that I show my friends who tell me I waste money on having so many backups. I'd be devastated if something happened to my data. I have so much invested in software its unreal.
 
Many, many moons ago, through my own idiocy I over-wrote the IPL volume (for those who know that that is) on a $10M IBM mainframe. I spent the next several hours restoring the volume through a rather painful process ... only to start the over-writing program all over again as I was finishing up.

After another multi-hour recovery, identical to the one I had just finished, I went for a long walk. Lord help us from ourselves.

Lesson here ... I had a backup.
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back