Fastest File Sharing on a Shoestring Budget

[iansblues]

Hi. Thanks for considering my need.

Synopsis : What is the cheapest way to get great file sharing performance between 10.6-based and 10.5-based Mac Pro and G5 servers?

I am not very knowledgable about networking (retired, but active, C++ programmer). I have a new business that hopes to later utilize internal 10Gb Ethernet connectivity, but at this time, I can't afford more than $400-500 toward my short-term, Mac-to-Mac file sharing solution (the less spent now the better!).

The servers initially involved are 1) a 10.5-based G5 and 2) a 10.6-based Mac Pro. Other 10.5-based G5s are being prepared for this same, internal network. Server purposes will include : 1) file sharing, 2) mysql server, 3) separate Dev/QA/Production Linux servers hosted within Fusion, 4) logging server, etc.

I look forward to any solutions, or existing how-tos, you can provide. Thanks …

 

[Transporteur]

Well, the fastest would certainly be a 10GbE connection or a fibre channel connection.

However, do your machines even offer enough performance to saturate a normal gigabit ethernet connection (100MB/s)? If not, the faster connections will probably be a waste of money.

What you can do (easy and cheap) is to aggregate the two ethernet ports of the server, offering twice the capacity of a single port.
All you need for this (hardware wise) is a managed switch. They are relatively cheap, though.

 

[iansblues]

Thanks Transporteur. In one of your previous threads, tunerX mentioned a value-priced Dlink DGS-1224T switch that can aggregate ports. Since the DGS-1224T is 1) cheap ($50-$200 USD ebay - 2011-02-06 ) and 2) it can aggregate ports, is Gigabit port aggregation probably my best "price vs. performance" bet on the cheap?

My immediate need is to move dozens of 3.5GB video files to a backup server as fast as possible each day, all while staying within my $400-$500 budget. Thanks ...

 

[Transporteur]

Depending on your network size, you could also look at the HP ProCurve 1810G-8. It's the smaller brother of the 1810G-24 and is also fully manageable.
Very good switches and well worth the price.
Keep in mind that when selecting your switch, don't buy one that is too small and offers insufficient trunks if you want to aggregate the ports on your client machines as well. One trunk would be one aggregated port pair in your case, assuming that you run the stock Apple hardware without third party NICs. If you only want to aggregate the ports on your server, a small managed switch would be fine, though.

For relatively low performing hardware (disk IO wise), I'd say port aggregation is the way to go as all that is required is a managed switch. With additional Intel server NICs it should even be possible to aggregate more than just the two build in ports of the Macs.

Unfortunately I haven't set up my hardware to link aggregation yet, so I can't give you real world benchmarks of this.

 

[iansblues]

Thanks Transporteur! My disk I/O performance will not exceed SATA II in the first year, so your suggestion sounds perfect for my need. Phase two could be to migrate up to all Intel-based Mac Pro and Xserve servers, as they become less expensive, then aggregate NIC-based server ports and switch ports to support my expected database server traffic, as that growth occurs. Maybe NAS appliance capacities will "take off" soon and provide the 16TB-20TB of homogeneous file share capacity I'll need between two disk stores by mid-2012.

Thanks for helping me establish a "roadmap" for my network's growth. Once this effort is not a one-man, shoestring-budget shop, I will hire one of the combination "network admin / server admin" friends I worked with for years. They will be able chart the future for the business' network growth.

 

[Transporteur]

I just found the Sonnet 2 port NIC. Goes for $200 and Sonnet claims that it can be aggregated with the internal ports of XServe or Pro, resulting in a 2Gb/s full duplex connection.

-> http://www.sonnettech.com/product/prestogigpcieserver.html

It most likely uses an Intel chip. Original dual port Intel cards (like the Pro/1000 Dual) are cheaper, I don't know how well the cope with OS X, though.

 

[iansblues]

Thanks! I will review the Sonnet card and possibly buy it at the end of the month.

I took your advice and bought a ProCurve 1810G-8. It is to arrive in a few days. I will implement it and run more performance tests, then consider moving on to implement the Sonnet card for even better performance.

I really appreciate your time and your interest in my problem! Following your leadership, I am feeling much more hopeful about my daily, "production processing" windows!

I will update this thread, after I make some progress and complete benchmarks.

 

[paduck]

My immediate need is to move dozens of 3.5GB video files to a backup server as fast as possible each day, all while staying within my $400-$500 budget. Thanks ...

Is that something like 50 GB or more? I have a Time Machine setup on a "server" in my house via Gig-E. It backups pretty quickly, although I imagine 50GB would take it something like 20 minutes or so - I just did a 2.71GB file in 53 seconds to test, don't know what spec you are looking for. It is quicker to backup single large files than lots of little files taking up the same space, so you will get some efficiencies there. Are you just dragging and dropping to the files, or is the process automated?

 

[onecajun]

Network speed

I get about 27gb per 7 minutes on my network. I would get a old g5 xserve and fill it with 3 2tb hitachi drives and a smc gigabit switch

 

[logandzwon]

Apple tends to recommend small-tree, http://www.small-tree.com/ for these kinds of add-on cards. You can port-bond between the small-tree ports and the built-in ports. Sonnet is of-course a well known brand, although I've never used them personally.

With your restrictions and budge I totally agree with the "get a gigabyte switch that supports port-bonding and do that."

One important caveat though... the LOM, (lights out management,) on Xserve does not have it's own port. It has it's own mac address, but shares the built-in Ethernet ports. If you set the ports as bonded you effectively lose your LOM.