How to sync server files to client for offline use?

[clindst]

I run a small web development company and we recently purchased a Mac Mini Server to use as a central repository for all company files, project assets etc.

We currently have 3 Mac machines (2 macbook pros) running Snow Leopard and we all have our files saved locally to our machines.

Here is what I am trying to accomplish, please chime in if there is a better way to execute:

I would like to have all files saved to the group folder on the server so everyone can access them and so they are all saved in a safe place backed up with Time Machine.

The problem is that 2 of the machines are laptops that we take home at the end of the day. We need to have access to these files at all times without VPNing in.

Is there a way to have all files saved on the server but also synced locally to the client computers? If so, we would like there to be a 2-way sync between the client machines and the server so if any changes are made anywhere, they will be auto synced once the laptops are brought back to the office and connected to the network.

Can someone please point me in the right direction? If there is another/better way to do this, please let me know.

Thanks!

 

[camiloken]

Simple

Activate a mobile account for those users.
You can do that in workgroup manager.

 

[Alrescha]

Can someone please point me in the right direction? If there is another/better way to do this, please let me know.

Actually, upon a second reading, I agree with extrachrispy, below. If you want to stay sane, you are going to want to use tool that gives you history and auditing. CVS or Subversion would be likely candidates.

A.

 

[extrachrispy]

Use an SCM tool (subversion, for example, is built-in). Check in the files that need to be saved, devs can check out a copy onto their laptops. The only time they'd need to VPN in would be to update their copy or to check in changes.

You should be doing this anyway, of course.

 

[belvdr]

Use an SCM tool (subversion, for example, is built-in). Check in the files that need to be saved, devs can check out a copy onto their laptops. The only time they'd need to VPN in would be to update their copy or to check in changes.

You should be doing this anyway, of course.

Even better, use rsync. Then you don't have to download/upload the entire file, only the bits that changed.

 

[clindst]

I found some additional info on enabling Mobile Accounts. Would this accomplish what I am looking for? I already have the main hard drive being backed up with Time Machine which offers a certain level of version control.

EDIT: To clarify, I'm trying to sync a main company workgroup folder to all of the client computers. Upon looking into the Mobile account feature, it sounds like it is intended to sync the individual user's home folder instead of one main folder to all client. Am I missing something here?

 

[extrachrispy]

Even better, use rsync. Then you don't have to download/upload the entire file, only the bits that changed.

rsync is not a version control system, nor does it log an audit trail.

If you're doing development of any kind, or managing changing content of any kind, you NEED an SCM system. Fancy folder-syncing tools are NOT a substitute for revision control.

 

[belvdr]

rsync is not a version control system, nor does it log an audit trail.

If you're doing development of any kind, or managing changing content of any kind, you NEED an SCM system. Fancy folder-syncing tools are NOT a substitute for revision control.

You missed my point. I meant use rsync with subversion. You get the best of both worlds.

 

[calderone]

You missed my point. I meant use rsync with subversion. You get the best of both worlds.

If you didn't say what you meant, he didn't miss anything.

 

[jalagl]

Another vote for a configuration management tool. Subversion (SVN) is relatively easy to setup and use. You can check in the changes to the files next time you are at the office, or enable external access and check in from home.

Check out Version Control with Subversion to get a better idea on this type of setup.

 

[dyn]

You missed my point. I meant use rsync with subversion. You get the best of both worlds.

That sounds ridiculous as you can do a subversion checkout (svn -co). That'll create a working dir which you can keep in sync with the svn repository by using svn. There is absolutely no point in using rsync with subversion.

Btw, if you want to do offline development on a notebook for example you might be better off by using a distributed scm like git or mercurial. Both can interact with subversion but it might complicate things a bit. The problem with scm's like subversion is the client-server model. They need to contact the server to be able to do everything. When offline you can only do a small amount of things making it not very useful. A distributed scm solves these problems as it doesn't need a server, it's local. However, you can use the client-server model with the distributed scm's if you want (in this case you do).

If you want to have version control combined with a wiki, issuetracker, etc. check out trac. There are some more tools that can do the job btw. Trac uses subversion by default but there are mercurial and git (iirc) expansions so you can use trac with those systems.

I'd go for a combination of mercurial and subversion. Use subversion as the main and central repository but use mercurial on the notebooks for offline use instead. Developers can do their development with mercurial the way they want and in offline mode. When they are able to go online they can check in their changes into the main subversion repository. You might want to check if you can do all this with just mercurial (mercurial has some howto's on this).

 

[belvdr]

If you didn't say what you meant, he didn't miss anything.

I did. I'm not sure why you even bothered to reply here. It was a simple miscommunication.

That sounds ridiculous as you can do a subversion checkout (svn -co). That'll create a working dir which you can keep in sync with the svn repository by using svn. There is absolutely no point in using rsync with subversion.

Not necessarily. So long as you are not downloading large files over and over again, you're correct. Let's say you have 1000 4KB files. If one bit changes, svn will then download approximately 4MB of data. With rsync, you only download the bits of the file that's changed, which may only comprise 10-20KB of data. It's a much faster sync.

 

[extrachrispy]

I did. I'm not sure why you even bothered to reply here. It was a simple miscommunication.

Not necessarily. So long as you are not downloading large files over and over again, you're correct. Let's say you have 1000 4KB files. If one bit changes, svn will then download approximately 4MB of data. With rsync, you only download the bits of the file that's changed, which may only comprise 10-20KB of data. It's a much faster sync.

So, please clue us in as to how to make svn use rsync as its transport.

Alternatively, mercurial can download patches to binary files. I only suggested subversion to the OP because it already comes with OS X. You have to get Mercurial from MacPorts.

 

[dyn]

Not necessarily. So long as you are not downloading large files over and over again, you're correct. Let's say you have 1000 4KB files. If one bit changes, svn will then download approximately 4MB of data. With rsync, you only download the bits of the file that's changed, which may only comprise 10-20KB of data. It's a much faster sync.

Not necessarily. This only counts for binary files. Subversion will only store diffs but only if the files are plain text. Binary files are not and thus get synced competely. If those 1000 4kB files get changed one bit and are plain text files than it will only sync 4 bits, not the entire 4 MB.

Rsync works completely differently, it is syncing files on a filesystem level but by default it won't sync changes. You have to setup rsync in such a way it only syncs the changes. If you do it only syncs the diffs regardless whether files are binary or plain text, which is why it's faster if you have lot's of (big) binary files.

If you're using it to save an the amount of data that has to be transferred than you either have a networking problem, you're using the wrong tool for the job or you don't have enough patience If the network is a problem because you're using the 3G network due to being on the road than using subversion is not advisable as is any client-server system. In that case you're using the wrong tool and you need a distributed scm because you can work locally without any network links because the workingdir and the repository are the exact same directory. Using subversion those slow links will always cause some sort of problem and not having any will reduce the amount of things you can do significantly. Suffice to say, large binary files will always be a problem whether you're using rsync or something else so you have to be patient whether you like it or not. In that case you should decide whether you should add a certain file to the repository or not (you may only want to have the source code under version control and leave out the build dr, temp dir and the binary you end up with).

There are other ways of reducing the amount of data that needs to be transported. Rsync is mostly used to backup repositories (you have to stop svnserve for that!), not syncing between workingdir and the repository as rsync does not know of any version control. Rsyncing workingdirs or dirs with the .svn dir inside can lead to some problems. If you search for it you'll come across a lot of howto's advising to exclude those .svn dirs which are necessary for the workingdir (it's where it stores all the version control info, so no .svn, no working subversion). Using rsync is similar to doing a subversion export which is like a checkout but it excludes the .svn dirs. This is also explained in some howto's like the following: rsync helps exporting Subversion’s working copies. The following howto does it a bit differently and also explains why you'd not want to use rsync and why it is prone to error: rsync only files matching pattern,
a Subversion application. The article also explains why they want to use rsync which boils down to circumventing the problems with traditional scm's that use the client-server model and subversion in particular. The correct solution to this problem is not rsync but using a distributed scm like mercurial or git (i.o.w. you're using the wrong tool).