Offsite backup strategy failed - now what?

[rawdawg]

Daily people ask about RAID 5 as backup, and most here including myself wouldn't consider a RAID a backup of anything. A "BACKUP" is a separate disk preferably stored off site. So I've never even bothered with RAID.

Well, I had a HDD go down, but thankfully I had my offsite backup (offsite in case my apartment burns down). Well.... that offsite backup is dead....

I've tried ccc to pull any data I could off it and was going to try all the other disk repair (i.e. disk warrior, etc..) programs out there when I was turned on by an IT friend to SpinRite, the PC BIOS program. After reading reviews it appears to be the most powerful tool available to recover data (please correct me if needed).

SpinRite started crawling through at a snail pace of 2000hours for a 2TB drive. Online it sounds like this can be the case with 6months being the record, but apparently it has restored said hard drives.

At 3.8% the time remaining shot up to 20,000hours and eventually there was a critical error.

Obviously I would love any feedback on how else I could restore this data, but even more importantly this SpinRite process has taught me about the need for a HDD to be "exercised" every so often or else they can develop bad sectors. So what does that say about our offsite backup strategies?

Can we trust HDDs we put away for storage? What should we do to properly store out data. It would be terrible to see all those photos gone someday.

 

[overanalyzer]

I'm not sure what to suggest in terms of recovering the data, but I personally keep an on-site backup via a nightly clone to a hard drive that's in a waterproof and fireproof safe (it has a USB connection built into it), plus automatic online (and therefore offsite) backups via Backblaze. Maybe a strategy like that would be useful for you going forward.

 

[rawdawg]

I just checked out Backblaze and can't believe it says unlimited for $3.96/month. How can that be? Are they seriously offering to back up my 6TB worth of daya for $3.96?

How is this possible? I ask because I must be missing some fine print somewhere...

 

[QuarterSwede]

I just checked out Backblaze and can't believe it says unlimited for $3.96/month. How can that be? Are they seriously offering to back up my 6TB worth of daya for $3.96?

How is this possible? I ask because I must be missing some fine print somewhere...

That's the going rate these days. Crash plan, Carbonite, all have unlimited storage for around that.

To the OP, I'm almost shocked that Spinrite failed. If It can't recover your data nothing will. You might want to contact Steve Gibson (the dev) and let him know. I'm sure it'll surprise him and you might get some free help out of it.

 

[rawdawg]

Yes, with SpinRite failing I am very worried.

I had no idea online storage was that cheap. I literally would be backing up ~6TB of data.

You already answered me, but are you sure I can backup all that data!? I imagine it would take a year to upload anyhow. Is there any catch in the fine print, like they now "own" my stuff, because a lot of it is video which is how I make my living and wouldn't want others to call it their property.

Seriously, 6TB?!

Also, can I get other votes/opinions on which backup site they use? Thanks!

 

[Derpage]

Yes, with SpinRite failing I am very worried.

I had no idea online storage was that cheap. I literally would be backing up ~6TB of data.

You already answered me, but are you sure I can backup all that data!? I imagine it would take a year to upload anyhow. Is there any catch in the fine print, like they now "own" my stuff, because a lot of it is video which is how I make my living and wouldn't want others to call it their property.

Seriously, 6TB?!

Also, can I get other votes/opinions on which backup site they use? Thanks!

Brosef,

Your initial upload may be 6TB. In most scenarios that i'm aware of, your back up is then compared to the drive being backed up and the files which have changed are updated, not the entire 6TB every time. That would be dumb.

 

[nufanec]

RAID isn't a backup if your apartment burns down, but it does provide redundancy. Your best bet is always backup redundancy - that would be an on site backup and an off site backup, perhaps multiple of both. Sorry you lost data, that sucks, but rather than talking about exercising hard drives (that are likely to fail anyway) you should think about implementing a more robust backup solution.

 

[overanalyzer]

As someone else said, all of the online backup services are around the same price for unlimited storage. The major limiting factor is the amount of time it'll take with your connection to upload your files for the first time. But after that, it only uploads changed files. You can have the backups uploaded automatically in the background all the time, or you can schedule them to run at certain times or when you're not using your computer.

The only "fine print" of Backblaze that I know of is on the what they don't backup section (which essentially amounts to your OS files and some files they ignore by default, but that can be backed up too if you prefer):

http://www.backblaze.com/remote-backup-everything.html

What We Don't Backup
Backblaze does not want to waste your bandwidth or Backblaze datacenter disk space. Thus, we do not backup your operating system, application folder, or temporary internet files that are transient and would not be useful in the future. Backblaze also excludes podcasts in iTunes.

Certain Filetypes
You can see these exclusions by clicking on “Settings…” in the Backblaze Control Panel and selecting the Exclusions tab. These exclusions can be removed! Some of these excluded files are:
ISO (Disk Images)
DMG (Mac Disk Image)
VMC VHD VMSN (Virtual Drives)
SYS (System Configuration & Drivers)
EXE (Application Files).
Other Backup Programs
Backblaze also doesn't backup backups like Time Machine and Retrospect RDB.

Deleted Files
Backblaze will keep versions of a file that changes for up to 30 days. However, Backblaze is not designed as an additional storage system when you run out of space. Backblaze mirrors your drive. If you delete your data, it will be deleted from Backblaze after 30 days.

Disconnected External Hard Drives
Your external hard drives need to be connected to your computer and scanned by Backblaze at least once every 30 days in order to keep them backed up.

Generally most online backup services are "per computer", so they also won't back up drives that are attached over a network (so that you can't pay for one computer and then back up several). If your storage is on a NAS that can't run software, you may have to see if you can include it for backup, or may need to see which services will allow that.

It's also worth noting that if you ever have a complete data loss and need to recover, you probably won't want to wait to download 6TB of data. They'll send it to you on hard drives, but you'll pay for that.

That's why having an onsite backup too is a good idea. That way for routine things like hard drive failures you can recover on your own without any cost or delay. And if you have a more serious issue like a fire or flood that destroys both the original drive and the onsite backup, you still have the online backup to fall back to. If you want to really be cautious, you can combine two onsite options - like RAID mirroring plus a nightly clone, or a nightly clone and Time Machine - PLUS the online backup. That way you increase the likelihood that you can do an onsite recovery without the time and cost of recovering from the online backup, except in cases where all the drives are destroyed at once.

 

[monkeybagel]

Daily people ask about RAID 5 as backup, and most here including myself wouldn't consider a RAID a backup of anything. A "BACKUP" is a separate disk preferably stored off site. So I've never even bothered with RAID.

Well, I had a HDD go down, but thankfully I had my offsite backup (offsite in case my apartment burns down). Well.... that offsite backup is dead....

I've tried ccc to pull any data I could off it and was going to try all the other disk repair (i.e. disk warrior, etc..) programs out there when I was turned on by an IT friend to SpinRite, the PC BIOS program. After reading reviews it appears to be the most powerful tool available to recover data (please correct me if needed).

SpinRite started crawling through at a snail pace of 2000hours for a 2TB drive. Online it sounds like this can be the case with 6months being the record, but apparently it has restored said hard drives.

At 3.8% the time remaining shot up to 20,000hours and eventually there was a critical error.

Obviously I would love any feedback on how else I could restore this data, but even more importantly this SpinRite process has taught me about the need for a HDD to be "exercised" every so often or else they can develop bad sectors. So what does that say about our offsite backup strategies?

Can we trust HDDs we put away for storage? What should we do to properly store out data. It would be terrible to see all those photos gone someday.

RAID is not a replacement for backup, but prevents downtime in a situation like this. If you had RAID 1 or 5 you would still be up and running.

If you saved over a file with an blank version, that's where the backup comes in.

I would not trust my data to offsite companies that I don't know - I would do a local backup and place in a safe location.

 

[brentsg]

I use a both a local and an offsite Crashplan backup for mine. You can go crazy with the encryption if you're concerned about that.

You can also seed the initial backup to speed things up, though I'm sure you pay extra. I've only got around 1.6TB archived there, but I did lose my main drive and had my local backup fail (long story).

The only knock is that it took a while to restore my loss remotely. You can request that it be sent on a hard drive for a price, but they were a bit slow to get this out to me. However, I certainly did not lose anything.

 

[bernuli]

I just checked out Backblaze and can't believe it says unlimited for $3.96/month. How can that be? Are they seriously offering to back up my 6TB worth of daya for $3.96?

How is this possible? I ask because I must be missing some fine print somewhere...

Unlimited storage is what they offer. But they most likely limit you on bandwidth per day.

 

[deconstruct60]

Unlimited storage is what they offer. But they most likely limit you on bandwidth per day.

More likely it is an inertia lock-in. After a while you can't pragmatically get the data out. If you have send in a disk for snail mail transfer those fees will make up the difference (e.g., $/GB transferred). Otherwise at about $50 per year over the course of 3 years that $150. 4 $200 .

Most users sitting on asymetric upload speeds pragmatically limit the bandwidth.


If have really agreesive de-dupe it is also likely your bits looks like someone else's bits. ( even encrypted can get overlaps with other stuff ) .

 

[d-m-a-x]

i have two blu ray burners. once a month i burn a spindle of discs, then keep them in a dark drawer (important). Been doing this for 15 years (since cd's). Have never had a problem with lost data.

 

[brentsg]

i have two blu ray burners. once a month i burn a spindle of discs, then keep them in a dark drawer (important). Been doing this for 15 years (since cd's). Have never had a problem with lost data.

Anything that requires this much manual effort will break down for most people.

 

[digitalhen]

I have a server (an old "beefed-up" dell) running Ubuntu and a ZFS storage area of about 12 TB. It rsyncs all my important data every hour from my desktop, and runs ZFS auto snapshots at least every 15 minutes. This server acts as my main place for sharing video and audio around the house.

This then is synced to Crashplan (which allow you to run from Ubuntu, unlike Backblaze), and a second offsite backup at my friends house on an incredibly slow Synology box.

Incredibly geeky I know - it was quite a learning experience - but I have lots of irreplaceable media (pictures of family etc) that make this worth investing in.

That said - the redundancy in ZFS has saved me once, Backblaze when I used it has saved me, and NOT raiding my drives on my desktop has saved me (one drive failed but I didn't lose the other drives).

What I've learnt: diversify your strategies.

 

[d-m-a-x]

Anything that requires this much manual effort will break down for most people.

if you can't reserve a day every month or two for maintanance, you deserve to lose your data. It's like driving without an oil change

 

[chiefroastbeef]

if you can't reserve a day every month or two for maintanance, you deserve to lose your data. It's like driving without an oil change

As much sense as that makes, most people, including me, still do not do it.

 

[brentsg]

if you can't reserve a day every month or two for maintanance, you deserve to lose your data. It's like driving without an oil change

There aren't any alternatives to getting an oil change.

You have a system that works for you that won't work for most people. It's just that simple.

-It requires diligence, time, manual effort, and expensive consumables
-It required a fire proof safe or a trip to a safety deposit box every backup
-The potential exists for you to lose 1-2 months of data due to fire, theft, bad luck, etc.

 

[Tesselator]

I'm late to the thread but:

Yeah, SpinRite is a HDD recovery tool. That is, it's for when you want to save or repair your HDD. It's maybe the worst tool there is for data recovery when the drive is damaged. It's maybe the best tool ever for salvaging your HDD unit tho.

When physical head or platter based errors occur on an HDD you wish to recover data from you want to access it as little as possible! This is critical! If you read around you'll know that the SOP is to clone the drive with a cloner that continues on error. You don't even want it going through its retry procedures. This way you get all the readable data onto a drive that isn't broken. Then after that you can work on trying to recover the bits that didn't make it during the cloning process.

Most of those two-bay drive-cradle USB/eSATA dock things have just such a function too. They sector clone and (the one I have) don't stop for errors. It just continues on and I'm not even sure if it goes through the retry routines typically executed in self checks and when under computer control. It clones everything too... Bad block list, boot sectors, drive signature, the works. This also means that after the cloning process is completed you should never again have both drives connected at the same time to the same machine.

Anyway after that image is created and safe if there is still something you need to recover off the broken drive you will know where it is by examining the clone's map (which I would keep locked BTW - so that your computer doesn't try to write to it or "fix it" while you're studying the structure). SpinRite can then be used on that specific area (of the broken drive) to try and recover what's left.

Turning SpinRite lose on the entire drive is a mistake in this case tho. That would be great as a periodic integrity check but not as a first step in a data recovery process on a broken drive you need to recover from.

Cloning a 2TB drive takes about 6 hours and these cradles don't supply airflow. With heat exacerbating most error conditions it's recommended to connect one of those USB low-speed fans and point it at your drives if you opt for the cradle method.

BTW, the fact that SpinRite failed won't be a surprise to anyone with experience in dealing with broken or troubled drives. Nothing can recover that which isn't recoverable. And relatively speaking, unrecoverable errors are pretty common in both rotational and solid-state media. **** happens...

 

[d-m-a-x]

at $1 per 25 gig, it is 50% cheaper than the average hard drive. I can deal with 1-2 months of lost data -although statistically the chances are low. Probability goes up the longer you leave it on a hard drive i.e. x12 months vs x1 month

----------

yes, it is tedious manual labor. But when a client calls me 9 years later and wants photos, i charge an archive fee

 

[rawdawg]

I'm late to the thread but...

Late is better than never!! This is most helpful for trying to recover this lost data, thank you! What drive do you have - never knew they existed until you brought it up? I was looking at this StarTech one but want to make sure it'll keep going past errors like yours.

Back to those online storage places others have mentioned. Would you trust those sites with your personal photos and information? Paranoia setting in I know, I've just never used any of these sites. Hate the idea of someone else being able to possibly access my stuff and suddenly a picture of mine shows up as a stock photo somewhere or something. Family photos are by far my most valuable thing on my computer and would gladly subscribe to such a service if they would forever be safe (and secure!) there.

I did read fine print that BackBlaze deletes files if it notices the hard drive they were on hasn't been attached to your computer for longer than 30 days. I wish there was a way to lock a folder for permanent archival. They say if you go on vacation for over 30 days you should shut down your computer with all drives attached and then restart it with all drives attached to prevent the 30-day countdown. Sounds like another hold in an otherwise secure backup strategy.

Here are the options you've listed. Anyone have others they trust? Preferably if they have a solution past the 30-day deletion issue

CrashPlan
Carbonite
BackBlaze

 

[goMac]

You need to do both on site and off site if that is your thing. Off site and on site can be used to restore each other, and because the on site is closer it's easier to monitor and replace drives.

Typically, my on site is a full backup, and my offsite is the Pretty Darn Important data only. Because my offsite data set is much smaller (I'm not backing up my music library to it), I can just use a cloud or managed server solution to handle that.

 

[brentsg]

Here are the options you've listed. Anyone have others they trust? Preferably if they have a solution past the 30-day deletion issue

CrashPlan
Carbonite
BackBlaze

You'll just have to read the fine print for each service, particularly regarding retention/deletion and security.

CrashPlan made the cut for me. They don't delete my files, even for drives that don't exist anymore. Also, they have security options available that should suit all but the most paranoid people.

 

[seveej]

OP, sorry to hear about your loss.

Two subsections:
1st off, HD failures:

Hard drive failures come in different guises, and the remedies are (likewise) different. Thus the first step (after noticing that something is wrong) is trying to ascertain what is wrong. Wikipedia's article is long-winded and lacks any how-to's, but may be worth a read. (http://en.wikipedia.org/wiki/Hard_disk_drive_failure)

Basically: If your computer sees the drive (and reports it's capacity correctly), and the drive is not making any frightening noises, the data might be recoverable by using specialist software. If not, the problem is either mechanical (head crash or mechanic failure) or the controller circuitry is screwed. Unless you have a healthy disk from the same batch lying around, both necessitate a third-party specialist. If your data is economically valuable or your business has insurance, this alternative is worth exploring.

2nd, backup&redundancy - policies

The thing with hard drives is that they mostly do not fail, but when they fail everybody's cursing. Back at the change of the millennium I was working at a small IT company, and our main file server was based on a RAID 5 -array of IBM GXP75's - (see http://hexus.net/tech/news/storage/209-ibm-gxp75-failures/ ) - man was that a nightmare, we basically came to the office in the morning and every second day one of the GXP's had failed over night so we went through some 30 drives before getting a totally new solution...

Anyway,
In my experience (both in offices and home offices), I've found that a three tier-solution offers quite a good balance.

1st tier: Local machine (workstations / file servers)
2nd tier: Local high-availability backup
3rd tier: Offsite backup.

Off course, if you're serious about your data you need an offsite backup. But whatever your offsite backup is, it will not be easy to access. That's where the second tier comes in: The purpose of the second tier is to protect against machine disasters, drive failures as well as "dumb users". Thus it needs to used every now and then and must be easy to access...

It also helps if the second tier is in itself both reliable and redundant.

I agree with d-m-a-x, that unless we're talking about data-center volumes, optical disks offer a viable route - they are both cheap and easy to transport and store and allow you to easily offsite-backup in multiples (always burn two disks and store them in separate sets). The caveat is that if you have no "library management" (for the lack of a better word), you'll be in a pinch when you're trying to ascertain which disc contains the most recent backups of files.

I recently posted a poll here, but did not get very many answers... https://forums.macrumors.com/threads/1539302/

My main interest in posting the poll (which I did not want to mention as I was afraid it might be suggestive) is that considering the amounts of data many of us are working with, the concept of internet-based daily backups are hitting the limits of feasibility (data throughput rates, especially on asynchronic lines).

RGDS,

 

[Tesselator]

Late is better than never!! This is most helpful for trying to recover this lost data, thank you! What drive do you have - never knew they existed until you brought it up? I was looking at this StarTech one but want to make sure it'll keep going past errors like yours.

Mine is made by Marshal - of whom I know nothing about. It's the MAL-4535SBKU3 and I think I paid $50 or $60 for it. I wanted one with a memory card reader and a eSATA (besides the USB 3.0) connection but they only had this type in stock for a reasonable price and I needed something that day for backing up a USB 3.0 LapTop. Since then I've learned that the fastest ones are made by Thermaltake - but it looks to me like their dual cradle doesn't have a clone function.

To use the clone function on the Marshal you just plug both drives in, detach all cables, and push the "clone" button twice. It shows it's progress in increments of 25% via blue LEDs.

Back to those online storage places others have mentioned. Would you trust those sites with your personal photos and information? Paranoia setting in I know, I've just never used any of these sites. Hate the idea of someone else being able to possibly access my stuff and suddenly a picture of mine shows up as a stock photo somewhere or something. Family photos are by far my most valuable thing on my computer and would gladly subscribe to such a service if they would forever be safe (and secure!) there.

I did read fine print that BackBlaze deletes files if it notices the hard drive they were on hasn't been attached to your computer for longer than 30 days. I wish there was a way to lock a folder for permanent archival. They say if you go on vacation for over 30 days you should shut down your computer with all drives attached and then restart it with all drives attached to prevent the 30-day countdown. Sounds like another hold in an otherwise secure backup strategy.

Here are the options you've listed. Anyone have others they trust? Preferably if they have a solution past the 30-day deletion issue

CrashPlan
Carbonite
BackBlaze

Me? Personally I would never upload my stuff to those kinds of folks. Like, ever! I figure as soon as I upload it then it no longer belongs to me and becomes community property. And if I wanna "get it back" or enforce my rights, I'll have to go to court which is expensive and time consuming! The way The West has been retroactively enforcing new policy and law all across the board in the past 15 to 20 years is not only unprecedented in recent (past 100 years) American history it's a dirty dirty thing to do - and the way things are headed it WILL be getting a lot worse before it gets any better. So I just figure that if I upload it then it's gone - the end. You're basically handing over all your valuables to someone you'll never see, never have met, who may or may not have close ties with some god forsaken "agency" or other criminals and saying: Here, hold this for me indefinitely... I trust you.

Ha! No way! If I want my stuff on-line I'll make cut versions of it and port forward my NAS or set up my own servers. Even if those kinds of folks aren't crooks **** happens and we've all heard the excuses and explanations before: It was an act of God, It was out of our control, We're truly sorry but there's nothing we can do, We have no control over <enter disaster or malfunction here>. And then there's the issue of speed.

What is your line speed? Mine is 120Mb and that's like 11 or 12 MB/s if all conditions are perfect and I'm right next door with a direct connection. You can maybe get that if you can open 50 or a hundred connections to their servers simultaneously but more likely you'll get between 1 and 5MB/s. At 3MB/s that's about 100 hours to DL or UL one terabyte (4 days of 24/7 I/O without erring out). At 1MB/s that's 300 hours or about 13 days - again if there are no errors. And if there are errors? What, you're going to check the integrity of empty or broken files? How many times have you had to re-download something yourself? For me I get broken or incomplete file downloads about 1% of the time (actually) or maybe a little less. But that's one out of every hundred or two downloads. How many files do you have? And how would I know if it uploaded correctly and perfectly or not? The ONLY way is to download it. What, now we've just doubled our times. Now it's not 4 to 12 days but more like 8 to 24 days of full speed I/O. And of course during that time your browsing and other networking speeds are in the mud.

Then there also may be volume caps. Some ISPs will only allow you so many data packets before they (secretly and unannounced) start capping your speed. I dunno... The only exception I can see to the speed problem would be solved by a very local storage out fit. Driving your LOCKED HDD down to their server site and handing it to them with a contract which specified maintenance and integrity for that physical drive and also have it rotated into their back up scheduling. Then when something went wrong you could drive down and pick up your backup. Of course that doesn't sound a whole lot different than leaving an HDD BU at your Mom's house and/or the office to me.

Anyway, those are my thoughts on the issue... It's not really for me... Maybe it's good for someone who doesn't care or worry about their data and only has about 50GB or less... <shrug>