Apple Supercomputer at Virginia Tech

[zachplaysguitar]

Today we got to visit and volunteer to help set up a supercomputer at Virginia Tech comprised of 325 MacPros. When operational, it will be the largest and most powerful power-aware cluster in the world. All the machines are connected using a custom cable, each being capable of speeds over 40Gbps, and all will run some form of Linux.

 

[mkrishnan]

Nice.

So the first generation of the VTech supercomputer used OS X, and used XGrid or whatever it's called to cluster the computers, didn't it? They decided to use Linux? Why use Mac Pros if they're going to use Linux? They don't seem like the most cost efficient option, exactly...

 

[NT1440]

my eyes cannot withstand the beauty of this *weeps*


i see all these posts about you guys in wat seem like colleges with awesome tech departments and it really worries me that I'm not getting into the schools im passionate about.

 

[zachplaysguitar]

I'm not sure why they decided to use Linux but I do know that when they told Apple they weren't using OS X, Apple refused to give the university compatible drivers. The techs had to reverse engineer the Apple drivers to make them compatible. The main thing was to make sure all the temperature monitors functioned properly, so they can remotely monitor performance. The tech there said that he had to manually enter up to 256 different 4 character keys for over 30 drivers to get them to work, and when they didn't at the very least he had to restart, or had a kernel panic.

 

[twoodcc]

dang. i am surprised that they aren't using OS X. what do you mean by 'custom cable'?

 

[zachplaysguitar]

To be honest I'm not really sure. The grad students working there were somewhat hard to understand but they said they had ordered them special and they were over 200 dollars a piece. He gave some technical name but I don't remember. I think it had a T in it...

 

[greg555]

Here's an article about the interconnect they are using (Mellanox Technologies' 40-Gbps InfiniBand technology interconnects): http://www.informationweek.com/news/hardware/grid_cluster/showArticle.jhtml?articleID=209400583

Greg

 

[jodelli]

The last one with the XServes was around 17 teraflops, wasn't it? This one should be faster.

Just looked it up, it will come in around 30 teraflops.

http://www.appleinsider.com/article..._pro_supercomputer_to_crack_29_teraflops.html

edit: The old one was around 12 teraflops.

 

[belvdr]

Not sure why they used Apple either, if they are just using Linux. They could have used a lot less space (not to mention looking a lot better) and had more compatibility if they used some rack mount 1U or 2U HP ProLiant servers. Strange...

 

[yrsonicdeath]

Man I would have liked to see that while I was there. What building is that in?

 

[iParis]

Jeezz!!1
That's a lof of f**** Mac Pros, if I'm not mistaken.
If I was there I would either cry all the way out the door and probably until I got home, or I would never leave.

 

[GroundLoop]

dang. i am surprised that they aren't using OS X.

Me too. Safari would be wicked snappy.

Hickman

 

[DigiCatRedux]

Considering how much money the university was willing to invest on all those MacPros, I'd have thought they'd have bought some better shelves to line them up on - that top row in picture #5 looks like it's getting ready to buckle.
Maybe they'll just use a stack of OSX discs that came with the MacPros to support the middle a bit.

Awesome, never the less.

 

[twoodcc]

Here's an article about the interconnect they are using (Mellanox Technologies' 40-Gbps InfiniBand technology interconnects): http://www.informationweek.com/news/hardware/grid_cluster/showArticle.jhtml?articleID=209400583

Greg

thanks for the link. it still doesn't explain what the interconnect actually is. is it some sort of network connection? or more similar to a firewire connection?

 

[jodelli]

^It's a really fast bi-directional serial link. Apparently channel bonding can be used to increase the throughput or bandwidth, like plugging in two cables between a cpu and i/o device instead of just one.

 

[soberbrain]

I was thinking of getting a new computer. Is a set up like that enough to check email, go on the web, listen to music and watch movies???

 

[yrsonicdeath]

I was thinking of getting a new computer. Is a set up like that enough to check email, go on the web, listen to music and watch movies???

It's what I use and I have no complaints.

 

[jensepollense]

Man I would have liked to see that while I was there. What building is that in?

It's in one of the buildings at the Corporate Research Center.

 

[Phil A.]

Not sure why they used Apple either, if they are just using Linux. They could have used a lot less space (not to mention looking a lot better) and had more compatibility if they used some rack mount 1U or 2U HP ProLiant servers. Strange...

Got to agree with you there - they could have fitted 350 1U servers in less than 9 standard racks. All that shelving looks fugly to me

 

[iSaint]

I was thinking of getting a new computer. Is a set up like that enough to check email, go on the web, listen to music and watch movies???

Many times over...

 

[jodelli]

I was thinking of getting a new computer. Is a set up like that enough to check email, go on the web, listen to music and watch movies???

It's what I use and I have no complaints.

Nah, the c. 2000 cores are overkill. I can make do with a 32 node Apple Workgroup Cluster.

 

[yrsonicdeath]

It's in one of the buildings at the Corporate Research Center.

Makes sense. It seems like there is an endless amount of things back there.

Nah, the c. 2000 cores are overkill. I can make do with a 32 node Apple Workgroup Cluster.

Suit yourself.

 

[belvdr]

I just thought of something... Whoever designed that is going to get smacked if they ever have to open the case on one of those boxes. It'll take 3 times as long as a normal rack mount box.

 

[chris200x9]

hahaha linux pwns mac

sorry like macs but am a linux fanboi at heart

 

[SkyBell]

Darn, the title led to believe it was a "new model" supercomputer or something custom.

Still, that's an epic win.