Originally posted by dongmin
Can someone enlighten me on the possibilities of this clustering setup? Would it have anything to do with hardware or is it purely a software thing?
clustering is essentially a two part proposition.
Physical:
You need to actually build a cluster of machines. this means they need to be connected together on a network.
For some configurations the network can be slow and high latency. A good example of this would be distributed rendering where CPUs could render alternate frames or they could render chunks of frames (to be assembled when done).
Other tasks need high speed and possibly ultra low latency network connections. Imagine a program that could spawn many many threads (streams of computation) and these threads were occasionally dependent on each other... that is stream "Q" can't start till stream "K" gets done and "Q" better finish because "X" is coming up and it needs that data from "Q".
For the lower bandwidth clusters, a 100BaseT network is fine.
Small clusters (few nodes) can actually use IP over Firewire, which has pretty low latency and good speed.
For high speed, but low cost networks Gig Ethernet is the way to go... but there are proprietary networking technologies that are very fast and super low latency like Myrinet (but they cost a fortune).
Software:
On the other end, you need some sort of Software support to actually cluster.
Some applications have clustering support built in. They often use a client/server approach where one master submits chunks of work to the other nodes then the master gets the returned work and reassembles it. Shake and the new xCode work kind of like this (AFAIK).
OS X also supports things like Suns Grid software and some other related Grid like products.
What I'm waiting for is a real OS Level (Kernel Level) plug and play clustering technology from Apple. Something that can distribute a local thread to any other node designated as a cluster member. Mach should be able to do this, you just need to run code that was written to be highly threaded.