Accessing Big Files Remotely

[KeyBlue]

Hi all,

Apologies if this is quite a basic question!

Our small business currently has an iMac running Server OS connected to some external hard drives. This works great when we're in the office for copying and pasting files, although quite slow if you're working on a file direct from one of the hard drives. The files in question vary, but larger design files can be up to 2GB sometimes, so speed is important.

What we want to do is to be able to work remotely using an internet connection and still have access to all these files.

I guess this splits into two questions:

1. How do we connect to our current setup outside the office via internet? (We have a fixed IP).

2. Is it much easier to setup/will it be easier to work with if we simply bought a Dropbox subscription and stop using our current setup altogether?


My first thoughts are - Dropbox 'should' be quicker working remotely, but would our current setup be quicker when working in the office?


Any advice appreciated!

 

[Mikael H]

It's probably considerably cheaper to change how you work than to make the way you work extremely fast.

1. My recommendation for people who need to access data from outside the office, would be the following:
a) Invest in a good, business-grade firewall, and pay a knowledgeable person to set it up for you. The minimum feature you need for my recommendation to work, is the ability to allow for secure VPN connections to your network.
b) Set up computers on your premises that your road warriors can remote control.
c) Pay the person in step a to set up VPN tunnels from your workers' laptops, so that they can work as if they were on your office network, and have them remote control the computers mentioned in step b. In this way they won't be transferring huge files across iffy network lines, but instead will only be transferring remote desktop traffic (VNC or RDP depending on platform).

This solves the problem of needless investment in huge bandwidth to your office, and of less-than-stellar performance for remote workers. It also avoids introducing gaping security holes in your perimeter defense.

2. Moving a file share to a cloud-based/Dropbox-like solution will kill productivity for everyone, not only for those on the move. Your client computers are likely limited by a) 1 Gbps network connections to the server iMac, and b) the I/O capacity of that same computer, unless it has an SSD.

A 1 Gbps connection means that in the best case a 2 GB file will take about 20 seconds to load over the local network. That's unless anybody else is using the same network share concurrently.

The cheapest way to get better performance for multiple simultaneous users (if the iMac has a regular or Fusion hard drive), would likely be to invest in a small file server or NAS, with multiple drives in a RAID10 configuration. Note that this does NOT solve the bottleneck of a single network connection, but spreads reads and writes over more disks.

Even relatively cheap hardware may support LACP, which allows for aggregating multiple network connections into a single virtual link, but each client connection to the server will be limited by the maximum link speed for a single network connection. The difference here is mainly that several client computers can get a faster connection to the server at a time.

At some point, with files of the size you mention, you will likely want to store working copies of files on each computer, and then regularly copy finished files to a central storage.

No matter which way you choose to go, invest in good backups! Pay someone knowledgeable to set up an on-site + off-site solution that works for your company.

 

[KeyBlue]

It's probably considerably cheaper to change how you work than to make the way you work extremely fast.

1. My recommendation for people who need to access data from outside the office, would be the following:
a) Invest in a good, business-grade firewall, and pay a knowledgeable person to set it up for you. The minimum feature you need for my recommendation to work, is the ability to allow for secure VPN connections to your network.
b) Set up computers on your premises that your road warriors can remote control.
c) Pay the person in step a to set up VPN tunnels from your workers' laptops, so that they can work as if they were on your office network, and have them remote control the computers mentioned in step b. In this way they won't be transferring huge files across iffy network lines, but instead will only be transferring remote desktop traffic (VNC or RDP depending on platform).

This solves the problem of needless investment in huge bandwidth to your office, and of less-than-stellar performance for remote workers. It also avoids introducing gaping security holes in your perimeter defense.

2. Moving a file share to a cloud-based/Dropbox-like solution will kill productivity for everyone, not only for those on the move. Your client computers are likely limited by a) 1 Gbps network connections to the server iMac, and b) the I/O capacity of that same computer, unless it has an SSD.

A 1 Gbps connection means that in the best case a 2 GB file will take about 20 seconds to load over the local network. That's unless anybody else is using the same network share concurrently.

The cheapest way to get better performance for multiple simultaneous users (if the iMac has a regular or Fusion hard drive), would likely be to invest in a small file server or NAS, with multiple drives in a RAID10 configuration. Note that this does NOT solve the bottleneck of a single network connection, but spreads reads and writes over more disks.

Even relatively cheap hardware may support LACP, which allows for aggregating multiple network connections into a single virtual link, but each client connection to the server will be limited by the maximum link speed for a single network connection. The difference here is mainly that several client computers can get a faster connection to the server at a time.

At some point, with files of the size you mention, you will likely want to store working copies of files on each computer, and then regularly copy finished files to a central storage.

No matter which way you choose to go, invest in good backups! Pay someone knowledgeable to set up an on-site + off-site solution that works for your company.

Thank you for your reply.

I see the strategy you mentioned, currently however we only store files on our network hard drive and work on them on our local machines.

If we were to continue this method (as opposed to working from that shared network drive or remote machine), would our current setup, or Dropbox, be suitable?

Thanks

 

[Mikael H]

Thank you for your reply.

I see the strategy you mentioned, currently however we only store files on our network hard drive and work on them on our local machines.

If we were to continue this method (as opposed to working from that shared network drive or remote machine), would our current setup, or Dropbox, be suitable?

Thanks

OK, then I misunderstood.
How fast is your business Internet connection, and what's an acceptable time to wait for copy jobs?
Could a "hybrid" (I know, that word gets used a lot) solution work for you, where people in your offices store/archive their work on the iMac, and the iMac syncs its share(s) to a cloud service; whether that's Dropbox or something else entirely? In that case your office workers would have as speedy a connection to the file store as they have today, but your out-of-office workers could reach a (delayed) copy of the file store from anywhere.

Unless you know what you're up to, opening up a server to the Internet is probably a worse idea than outsourcing the security aspects to a well-known service provider. In other words if it's viable for you from a bandwidth perspective, using Dropbox/Onedrive/[...] to manage Internet-wide access to your files is probably not a bad idea.
Just remember what I said about backups: Make sure you have them, because none of those services actually are backup solutions, even if they do protect you from some kinds of mistakes as long as you notice them within ~30 days.

Now this is what I would advice from a realistic perspective of someone who just wants to go about their business. On the other hand: if you want to know how these things work, you can probably get a good enough understanding of the basic concepts to set up a nice and secure Internet accessible service within a few weeks or so even if you start pretty much from scratch.

 

[KeyBlue]

OK, then I misunderstood.
How fast is your business Internet connection, and what's an acceptable time to wait for copy jobs?
Could a "hybrid" (I know, that word gets used a lot) solution work for you, where people in your offices store/archive their work on the iMac, and the iMac syncs its share(s) to a cloud service; whether that's Dropbox or something else entirely? In that case your office workers would have as speedy a connection to the file store as they have today, but your out-of-office workers could reach a (delayed) copy of the file store from anywhere.

Unless you know what you're up to, opening up a server to the Internet is probably a worse idea than outsourcing the security aspects to a well-known service provider. In other words if it's viable for you from a bandwidth perspective, using Dropbox/Onedrive/[...] to manage Internet-wide access to your files is probably not a bad idea.
Just remember what I said about backups: Make sure you have them, because none of those services actually are backup solutions, even if they do protect you from some kinds of mistakes as long as you notice them within ~30 days.

Now this is what I would advice from a realistic perspective of someone who just wants to go about their business. On the other hand: if you want to know how these things work, you can probably get a good enough understanding of the basic concepts to set up a nice and secure Internet accessible service within a few weeks or so even if you start pretty much from scratch.

Perfect! Thank you for your advice!

Apologies if my original question wasn't clear, I guess what I was after was confirmation that a Dropbox style method would be ideal, and your example of a 'hybrid' way of working is worth us adapting to.

 

[hobowankenobi]

Agreed overall.

Lots of cloud based file sharing services to choose from....Drop Box is one of the leaders for sure.

If you have enough bandwidth, and have so much data that a cloud service is cost prohibitive, one other option:

Synology has lots of choices for NAS boxes that can function as a local file server in place of your current server, and you can run your own Drop Box-like service from it. Pretty slick, lower TCO, and you retain your data 100% of the time. It allows what was called a "hybrid" setup so local users get at least the performance they have now (everything local), but offering efficient and safe WAN access too.

Their included Drive is good; the add-on Presto server is reported to be faster and more feature rich.

I have setup and used Drive, and it nearly as good a Dropbox overall....never used Presto. Based on your file size, it would probably be well worth the cost (compared to buying more bandwidth).

There are other similar products to look at...but Synology is the best/most Mac friendly I have used.

I would recommend not going with a low end model....as the higher end boxes perform better with more CPU and RAM. Also multiple drives (for both redundancy and performance) as well as multiple NICs for more network options.

 

[hobowankenobi]

Synology has a dizzying assortment of storage choices. As a starting point, a model like this might be worth a look: high performance, multiple NICs, enough drive bays for flexibility and performance, etc.

Most NAS boxes can do many things. Don't be intimated by all the things it can be used for...just focus on your requirements: file sharing, remote file syncing, and robust backups. You may well find there are some other useful things you want to do with it, but to keep things as simple and reliable as possible, you don't have to turn on any services you don't need.

Oh, and forgot to mention that they include a great feature for remote access that does not require a static IP or opening dangerous (commonly attacked) ports: QuickConnect.

QNAP is also well liked, but I have not used any of their modern gear, so can't add much other than they are worth checking out.