Best Free Folder Sync as backup app?

[LeandrodaFL]

Im looking for an app that can make copies of my files to an external hard drive, I will use this as sort of backup. Im gonna backup 700GB of files, so it ha to be a great software, I dont want to miss 1 single file. Currently, I have been using "WD Anywhere Backup", but this is not compatible with Lion or Mountain Lion. Here is what I need from the application:

-As soon as I start my computer, the app starts running in the background to monitor the changes in the folders I selected
-Continous and real-time backup. As soon as I make any single change in a file in the folder (rename, or save, or create new thing) it instantly makes the change in the external hdd on the fly! Even deleting!!!!
-I can pickup this exernal hard drive, plug into another computer, and be able to read and manually copy my files to another computer.
-I can run something like "verify items" and it will compare the internal hdd and external hdd to see if anything is missing.
-Has to be Free, user friendly and reliable

Now, If something like this doesnt exist yet, it has to be created, what are developers waiting for? Please, not that I dont need software that performs backups in manner like Time Machine. I already have Time Machine. I am looking for a diferent solution.

Altough I mentioned Free, I will be glad to get a recomendation for a paid software, but only if it performs EXACTLY like I mentioned without flaws, and it better have an insane beautiful user interface

Lol, thansks for the help in advance, now Im gonna pray someone can answer

 

[switon]

RE: git, rsync, TM, ...

Hi LeandrodaFL,

Most of what you want already does exist, and it can be made into exactly what you want with a little more work.

My first question is: "I know you mention you have TM already, but have you investigated Time Machine (TM), as it nearly satisfies your criteria?" It does hourly backups of just what has changed on your disk, and you can recover your data from the TM sparse bundle (the backup file) from any other machine if you wish. TM is also nearly transparent for the User, it just works and few problems arise (although I'm sure that someone who has had TM troubles will disagree with this statement). And even when you are disconnected from your TM backup drive, TM continues making its backups, just that they are now stored locally as "snapshots" until you reconnect to your TM drive. This is nearly ideal for a laptop in that TM backups continue even when you take your laptop on the road. And when you are on the road you can VPN in to your LAN that has the TM backup drive and TM will then reconnect to the TM drive and continue its backups. This sounds quite close to what you require and is nearly as good as it gets for backup.

My second question is: "Have you investigated using iCloud or another of the cloud services for your backup?" For instance, in addition to the hourly TM backups of your entire disk drive, you could also rely on the iCloud or other cloud service to provide nearly instantaneous backups (versions) or your actual work, stored in the cloud. This would provide both an on-site local TM backup as well as an off-site cloud backup of your work.

My third question is: "Have you investigated any of the Version Control Systems (also known as a Revision Control System), such as git or subversion?" A version control system is nice in that it provides all previous versions of your work including records of who performed the modification and who commited the work. Basically, a "repository" is maintained that includes all revisions of the work and the histories of those revisions. You can "roll back" to any revision. For instance, the Mozilla project has been running for over ten years using both subversion and now git and has over 290,000 revisions. People are studying the Mozilla repository in order to understand the history of a large software project. Both git and subversion are free. The git program employs a "distributed repository" concept while the subversion program uses a master repository with checkout copies. Both of these fundamental repository strategies have advantages and disadvantages. If you were to use the git/subversion idea for your backup, all you would need is to write an Automator or Apple Script that executes a single line of bash shell code to commit/update/push/clone your modifications each minute of the day. If you went with the distributed repository of git, then each machine would contain a full backup of your work. And both git and subversion work by using diff and patch so that a minimum amount of data needs to be communicated to and from the repositories.

My fourth question is: "Have you thought about rsync?" The rsync protocol is employed by a number of software projects to distribute new versions of code, such as CTAN's MacTeX project or the various Linux distributions. Running the rsync command every minute from a script would keep your various machines up-to-date with any last minute modifications. For instance, the machine mounting your backup drive would run rsync to your working computer keeping the directory tree structure on the backup drive in sync with your working directory tree. Various backup programs, such as Carbon Copy Cloner, also use the rsync protocol to perform their cloning magic.

In a nutshell: TM provides hourly backups with minimal setup and user interaction. It can also be administered from a single Mac OS X Server instantiation. Cloud services can provide instantaneous backups of versions of your work. Various Revision Control Systems, such as git and subversion, provide repositories of all revisions of your work including the histories of all revisions. And rsync can keep your backup drive in sync with your work drives.

Just a few suggestions to investigate for solving your backup problems...

Regards,
Switon

 

[LeandrodaFL]

Hi LeandrodaFL,

Most of what you want already does exist, and it can be made into exactly what you want with a little more work.
Regards,
Switon

Time Machine doesn't satisfy me cause I don't care about hourly backups, I want SECOND backups, After all 1-hour window leaves all my recently created files at risk.

Lets say I just finished a time MAchine backup, and then download a picture and my Mac crashes. I want to take the external drive and connect it to another Mac and have acess to this new picture.

1 picture seems like nothing, but when its a movie I spend 45 minutes editing, I don't want this work lost. with WD Anywehre backup, the moment the movie gets created, its autmotaiclly backup to the external drive.

Of course, I could hit the "backup now button" on Time Machine, but I don't want to do this everyday all the time. I want automatic real-time continous backup, and I haven't found a solution for Moutain Lion yet.

Regarding iCloud: I haven't looked into it, cause I don't own Mountain Lion yet, Im at Snow Leopard, Im just researching. Nevertheless, as of today, Im not looking for a cloud solution, but I will take a look at it when I get ML. If it also works, I will also use it, but I also what the external hdd instant real-time backup, as getting 700GB of data from cloud is gonna take time.

Rsync seemed too advanced for me, I want a simple easy to use application, and then again, I want this service running on the background making changes as soon as they happen, I don't want even minute by minute backup, I want REAL TIME copy of files

threes gotta be a solution out there, even if paid….

 

[Krazy Bill]

Google continuous backup or real-time backup

Wiki also provides a list of software, some with these capabilities. You'll have to research the listings to see if any are viable - especially for a mac.

 

[switon]

RE: instantaneous backup...

Hi LeandrodaFL,

I understand that you say you want "instantaneous" backups and that "minute" backups are not good enough, but you have to recognize that barring some sort of quantum entanglement solution in future quantum computers, there is no such thing as "instantaneous" backups. Even "mirroring" on a RAID 1 drive is not instantaneous.

Given that fact, then what I suggested above is relatively easy to implement. You just write a script that include a single command line (for rsync or git or subversion) that syncs your recent modifications to a repository, local or distributed. Now I suggested that this script be run every minute, but that was arbitrary, you could run the script every millisecond if you wished. Basically, the OS has to detect that a new request for writing to the disk has even occurred, something that even with interrupts probably takes a microsecond (or whatever the latency of the drive is). Now if "real time" backup is that important to you, you could write your own daemon with interrupts that would interrupt your processor whenever a write to disk is requested, and then simultaneously write a copy to a backup disk. Now this is not "instantaneous", but it is nearly simultaneous and is known as RAID 1 (mirroring).

And lastly, you have to justify your need for instantaneous backup against your risk for requiring that backup. It takes time to both write to a disk as well as to mirror to a RAID 1 disk system. What are the chances of that disk/RAID failing during the write? Extremely slim during your expected lifetime, so small is the probability that it is highly unlikely that you will ever loose your 45 minutes of video editing. You also have to recognize that your WD Anywhere backup was also not truly instantaneous no matter what Western Digital's advertising claims were. There was a delay there too.

So, in summary, I would suggest that you procure a RAID 1 disk system and mirror your work on the RAID drive. This provides you nearly simultaneous backup and protection against a single drive failing (one of the 2 drives of the RAID 1 system can fail without you loosing any of your data). This is as close to "instantaneous" as you are going to get with commercially available equipment. I would then suggest that you continue using your TM as an hourly backup of your work. In practice, I suspect that you will find the TM backups much more useful than the RAID 1 mirrors.

Edit: You might also consider using the "tmutil" terminal commands to modify when TM does its backups. In particular, using the 'startbackup' with '--block' options you can probably cause TM to backup continuously around the clock, once one backup is done it would begin a new backup, essentially giving you continuous backups (but, once again, not instantaneous backups).

Regards,
Switon

 

[LeandrodaFL]

Google continuous backup or real-time backup.

I did search google, but that was unafective, and moreover, there are some many bad choices in the wild nature of apps, that I was hoping I could get users feedback on possible solutions, after all, theres gotta be some Mac user using this type of backup.

Hi LeandrodaFL,

You also have to recognize that your WD Anywhere backup was also not truly instantaneous no matter what Western Digital's advertising claims were. There was a delay there too.

Regards,
Switon

I know, there is a delay, but its around 3 seconds. That I can live with I plan on making a video detailing this in the future and post, as aparenlty, this type of backup is rare. Yes, its sort of like RAID 1, but you specify the folder you want to, its not the entire disk. Moreover, you can set it to mulitple destinations, multiple origins, you can create several backups plans, etc...but what I like is that its "real-time"

 

[switon]

RE: RAID 1, RAID 10, TM to multiple backup disks, ...

I did search google, but that was unafective, and moreover, there are some many bad choices in the wild nature of apps, that I was hoping I could get users feedback on possible solutions, after all, theres gotta be some Mac user using this type of backup.


I know, there is a delay, but its around 3 seconds. That I can live with I plan on making a video detailing this in the future and post, as aparenlty, this type of backup is rare. Yes, its sort of like RAID 1, but you specify the folder you want to, its not the entire disk. Moreover, you can set it to mulitple destinations, multiple origins, you can create several backups plans, etc...but what I like is that its "real-time"

Hi,

I don't believe that "this type of backup is rare", as many server farms are designed with precisely this kind of backup in mind. Businesses that rely on multiple simultaneous transactions (banking, Amazon, etc.) have these kinds of requirements.

As I mentioned above, you can specify a different time schedule for TM backups. You might make it "continuous", that is, once one TM is finished another TM backup starts. Not only that, but with TM you can specify multiple different disks as alternatives, so TM will cycle among the backup disks effectively making multiple backups for you.

Secondly, I think RAID 1 is the easy and commercial solution for you. Yes, RAID 1 will backup your entire disk drive, but after the "initialization" most everything has been mirrored and all future mirroring will only be for what has changed on your disk. Wouldn't it be better to have a mirror clone of your entire disk drive rather than you having to specify what to mirror and what not to mirror? If you do the specification yourself, it is likely that at some point in the future when you actually need the mirror clone you won't have specified the actual file/directory that you need --- thus it is best to mirror everything.

So mirroring RAID 1 gives you nearly simultaneous backup with tolerance to single disk failure (you can even go to a higher RAID system with more disk drives and have tolerance to two simultaneous disk failures, if you are worried about simultaneous disk failures). Not only that, but there are also RAID combinations that give both mirroring for fault tolerance and striping for communication speed. So you can have a faster disk with redundancy - best of both worlds.

...just some suggestions...

Regards,
Switon

 

[LeandrodaFL]

Hi,
Secondly, I think RAID 1 is the easy and commercial solution for you.
Switon

Yeah, but then again, its very simple to specify once "folder home". Im posting a picture of an App name Smartbackup, but I have no idea how good it is, and its paid. It may offer what I want.

May not be rare, but apparently normal users dont use this much to give feedback

 

[switon]

RE: SmartBackup...

Hi,

From the thumbnail that you posted, I could not surmise that SmartBackup satisfies your requirements...at least not any more does it satisfy them than TM or CCC (Carbon Copy Cloner -- which, by the way, uses the rsync protocol). In fact, I suspect that SmartBackup may also use the rsync protocol for its syncing of folders. I don't know this for certain, as I have absolutely no experience with SmartBackup, but many if not most other backup programs of this sort use the rsync protocol. You would have to carefully investigate SmartBackup to see if it indeed satisfies your requirements.

I'll recommend it once more: a RAID 1 drive satisfies your "real time" backup criterion and provides tolerance for a single catastrophic disk failure without loss of any of your data, and TM provides you with revisions to the beginning of time (back to your first TM backup that is) and gives you options for multiple backups. This combination would appear to fully satisfy your criteria for a backup strategy.

Regards,
Switon

 

[switon]

RE: SmartBackup and TM...

Hi,

I just took a quick look at SmartBackup. It states that its principal advantage is that it uses Spotlight to only backup what has changed on your disk. Did you know that this is precisely the same mechanism that Time Machine backup uses to know what needs to be backed up? In addition, the Windows version of SmartBackup makes a point of using what they call "hard links" so that revisions of files are stored. I suspect that this is also an exact copy of the mechanism used by TM for its dated time backups. It appears that SmartBackup was perhaps initially a Windows port of the Time Machine backup strategy, minus the nice GUI. It was then retro-ported back to the Mac so that the same program could be used on both Windows PCs and Macs.

In my hasty research, I couldn't locate any more functionality that SmartBackup has over TM. I do not believe that it satisfies your "real time" backup criteria any more than TM, especially since SmartBackup and TM both use Spotlight to know when to backup what files, so SmartBackup would not "know" any sooner than TM that a file needs to be backed up.

Switon

 

[rhoydotp]

or you can change the tm duration to be every minute by editing

/System/Library/LaunchDaemons/com.apple.backupd-auto.plist

<key>Backup Interval</key>
<dict>
<key>Interval</key>
<integer>3600</integer>
<key>MaintenanceWakeBehavior</key>
<string>Once</string>
</dict>

 

[LeandrodaFL]

Hi,

I'll recommend it once more: a RAID 1 drive satisfies your "real time" backup criterion
Switon

How do I do RAID 1? Im on MacMini 2010


Switon[/QUOTE]

or you can change the tm duration to be every minute by editing

/System/Library/LaunchDaemons/com.apple.backupd-auto.plist

I will use Time MAchine also, but I wil keep it to hourly. I see no point of having to run time machine endeslly, specially cause some of the times, I may spend 2 hours without making changes, watching a movie for example, so why run this by the minute? The good about the app I am used to is that it only starts when I need it to, when I make a change in the specified folder. Moreover it doenst bakcup a lot of things I have no need to backup, like downloaded movies and other temporary work, wich is saved in other folders

 

[rhoydotp]

ok, seems like you already figured out what you want to do

 

[switon]

RE: RAID 1 and TM...

How do I do RAID 1? Im on MacMini 2010


I will use Time MAchine also, but I wil keep it to hourly. I see no point of having to run time machine endeslly, specially cause some of the times, I may spend 2 hours without making changes, watching a movie for example, so why run this by the minute? The good about the app I am used to is that it only starts when I need it to, when I make a change in the specified folder. Moreover it doenst bakcup a lot of things I have no need to backup, like downloaded movies and other temporary work, wich is saved in other folders

Hi LeandrodaFL,

Allow me to repsond to your second paragraph first. First of all, I would also leave TM at an hourly schedule, as for my coding purposes an hourly backup is fine with me. But as I mentioned in an earlier post, "tmutil" can be used to configure this schedule to most anything you want (and rhoydotp explains how to edit the actual plist configure file to obtain the same results). Let's say that you schedule it for every 15 minutes. Thus TM will start every 15 minutes, but this does not mean that it will do anything significant since if your disk hasn't been modified then TM has nothing to backup, so it quickly completes its task. When you say that SmartBackup "only starts" when you need it to, realize that in fact SmartBackup is always "running" and backs up when Spotlight flags a new/modified file; TM can be configured to do the same. You also mention that SmartBackup won't backup downloaded movies and temporary work, I just want to mention that TM also has the same ability. As I stated earlier, I really couldn't find in my hasty research any capability or feature that SmartBackup has that TM does not also have. (There probably is some new feature for SmartBackup, but it is not obvious from their website's list of features and capabilities.)

Now on to making a RAID 1 drive:

RAID 1 requires at least two HDDs (or 2 SSDs, or 1 HDD and 1 SSD). It is best to get two of the same type and size drives, however. Western Digital recently announced their "Red" drives specifically designed for use in server/RAID configurations. Why use an enterprise-class HDD instead of the cheaper commercial grade HDDs? Since you are concerned with "real time" backup and the loss of your work, enterprise-class HDDs are better, have longer warranties, have better vibration shielding, and are designed to run 24/7. They also have a feature that makes them particularly suitable for RAID systems. And that feature is ERC (Error Recovery Control). All HDDs have R/W errors from time-to-time. The commercial grade HDDs immediately report these errors, and when R/W errors are reported to RAID systems, the RAID systems usually respond by marking the drive as "failed", and then moving data off the "failed" drive and requesting that it be replaced. On the other hand, an enterprise-class HDD that supports ERC does not immediately report R/W errors, rather it waits a period of time, something like 7 seconds although this is configurable on most drives, before reporting the error. This allows the drive to "recover" from the error before it is reported, and thus most recoverable errors do not get reported and the HDD continues to work in your RAID. I have seen some individuals use commercial grade HDDs in RAID systems, only to have the RAID system "fail" a perfectly good drive (one that is operating just as it was designed with a low rate of recoverable R/W errors). Enterprise-class HDDs, while more expensive initially, will better keep your RAID out of trouble by not prematurely "failing" good HDDs and thus costing you less in the long run. And since enterprise-class HDDs usually come with longer warranties, we can hope that they are manufactured better and to better tolerances than the commercial grade HDDs. For someone really worried about their work, I'd recommend getting the enterprise-class HDDs for a RAID system.

With the question of drives out of the way, let's address the actual RAID system. The Mac OS X is capable of "doing its own RAID", so to speak. And I think many owners of Mac Pros with 4 drive bays probably do just that, they use Mac OS X to RAID their 4 bays. The Mac mini 2010 has two bays, but one is an HDD and the other is an optical DVD drive, I believe. Unless you have the 2010 Mac mini Server which came with two HDDs? If your Mac mini already has two HDDs, you are all set to configure them as a RAID 1 system. The Disk Utility.app can do this for you. If your Mac mini has 1 HDD and 1 ODD, then you can either swap out the optical drive for a second HDD if you are so inclined to do so. If you don't want to do this, then I'd recommend that you purchase one of the commercial RAID units. These RAID systems are very well designed and work extremely well, and give you many capabilities.

For instance, I'm particularly fond of the Synology DiskStation and Drobo systems. You can get a Synology DiskStation 213+ for a street price of $200, sans drives. You then purchase two enterprise-class HDDs to put in the Synology unit. The software for the Synology unit will help you configure your drives however you wish, say as RAID 1. The latest Drobo 5D has 5 bays for drives (you start with 2 HDDs and can expand at any time to more drives), and Thunderbolt and USB3 ports. (I don't believe your 2010 Mac mini has either Thunderbolt or USB3, however. But the Drobo 5N has ethernet and the 4-bay Drobo has Firewire 800 and USB and so either can be attached to your Mac mini.) Both the Synology and Drobo units support "hot swapping", so if a drive fails, all you have to do is "hot swap" the failed drive for a new one, meaning that you don't loose any uptime for the RAID system.

If I were in your shoes, I think I would seriously consider getting the Synology DiskStation to use as a RAID 1 system. You might "look up" these Synology and Drobo systems on their respecitive websites, as they have a lot more to offer including running many servers/services from the RAID units themselves.

Good luck with your RAID 1 system,
Switon

 

[LeandrodaFL]

Hi LeandrodaFL,
Good luck with your RAID 1 system,
Switon

Thanks for all the help! I forgot that I will set this on a 2012 MacMini with Mountain Lino, wich I plan to get early next year, it has 2 HDD bays

 

[blueroom]

Mac Mini only takes 2.5" drives internally.

I'd also recommend the Synology route.

 

[switon]

Mac Mini only takes 2.5" drives internally.

I'd also recommend the Synology route.

Oh, yes of course. Thanks for reminding me that the Mac mini uses 2.5" drives. My WD "red" HDD recommendation was with the Synology in mind.

Thanks,
Switon

----------

Thanks for all the help! I forgot that I will set this on a 2012 MacMini with Mountain Lino, wich I plan to get early next year, it has 2 HDD bays

Yes, you can order 2 HDDs for the 2012 Mac mini. I seem to remember some Apple advertisements stating that the drives in the Mac mini server were "server quality" HDDs. I don't know what their advertisement means, but I suspect that they are the laptop "blue" drives. These should be fine for configuring in a RAID 1 system. The Disk Utility.app will configure this for you as a software RAID system. And Apple's software RAID is probably not as strident at "failing" a drive for single R/W errors as are some of the "box unit" RAID systems.

Good luck,
Switon

 

[liontamer67]

I just wanted to chime in. I started using Synology 212+ in July with 2 cheaper green drives inside.
I want easy too and Mac has made me very very lazy when it comes to knowing all I need to know about my computer. I have been using ibm compatible computers since 1990 or so. I have built computers. But since 2006 when I started using Mac exclusively (I hated early 90's Macs and their cutie software).
I had never really used a backup drive with different bays before till this year. I also have 2 of the white My World Book harddrives.
Synology was my 3rd system after I tried out 2 other brands...gotta go look in amazon to see what I had purchased. They sucked though.
Anyhoo, Synology was soooo easy to use. I purchased through Amazon and the 2 external drives that you insert into the Synology on amazon. The Synology I received actually had a bad fan. I called Synology and they are in Washington state. They said they could overnight a new one but I would have to pay the shipping. Errr...no. I ordered a new one from amazon for the next day and returned the bad fan one. I got it the next day and works like a charm.
Synology is very mac user friendly too and I found that others are not.
I will go look up my amazon purchases if you want to know what I got. Sorry I have very bad recall memory!

 

[switon]

RE: Synology 212+...

I just wanted to chime in. I started using Synology 212+ in July with 2 cheaper green drives inside.

Cool! So did you set up your 212+ as a RAID 0, RAID 1, or Synology's own RAID system?

Switon

 

[LeandrodaFL]

I also have 2 of the white My World Book harddrives.

Wich WD software they come with, "WD Anywhere Backup" or "WD Smartware"? Have you tried them?