Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

leonhard

macrumors newbie
Original poster
Feb 4, 2008
6
0
I recently got a research grant of $40k to buy a personal (super)computer, and am anxious about the possibility of a Mac-Pro update. I would prefer a mac, but right now there are no options for memory upgrades above 64GB, or processor upgrades beyond 12 cores.

I know that the mac pro will likely be upgraded or killed in the relatively near future. Assuming that it is upgraded,

does anyone know how likely it would be for an upgraded mac pro to be expandable to >=128GB memory?

are there any hardware-fixes to expand the current generation mac pros beyond 64GB of memory?

also, does anyone know if mac pros (current or possible upgrades thereof) are compatible with NVIDIA's Tesla GPUs?

and finally, I'd appreciate any advice regarding how long I should wait for an upgrade before investing.

Thanks!
 
do your self a favor

if you do have 40k and want a mac pro i highly advise waiting as the current mac pros are from 2010 and it is currently 2012. the new mac pro will probably be a max of 16 core probably 128 gb of ram( maybe 96) and faster ssds and much faster graphics. So if u need a mac pro wait.

on the other hand if u do have 40k to blow then iw udnt even suggest a mac pro i dud suggest a sun server or a super computer. Even though the mac pro is fast it is a PRO computer not a SUPER computer. also i dud never spend 40k on a machine where technology moves so fast in 2 years any purchase u make will be obsolete

i wud not recommend the pro as a super computer as u will realize any job that requires a 40k computer will not be labeled pro but super

hope this helps

- robbie
 
Mac Pros are not compatible with Tesla.

It honestly sounds like you want a Linux PC instead. Probably a quad processor Xeon with a nice Tesla, depending on your software. Software and what you're actually doing plays a lot into this.
 
It's also worth nothing that OS X in its current state will only address up 96GB of RAM, so anything beyond that would be wasted money, at least under that OS.

What type of research are you involved in? $40k is a lot to spend on a single workstation.
 
More information as to the particulars would be extremely helpful, as you may be able to be served properly with a single box PC system (i.e. up to 4x CPU's), or a cluster (either based on CPU's or GPU's, if the software you're using supports GPGPU processing).

As per OS, I'm under the strong impression that you'd be better off with Linux from what information you've provided, particularly with Tesla cards (much better suited to scientific research than OS X due to both hardware and software options not available under OS X). Even the MP's PCIe slot configurations aren't well suited for high I/O requirements vs. other boards running the same CPU and chipset, and I suspect this will be far more the case with Dual Processor based SB boards (i.e. PC versions running as many as 80x PCIe lanes while the MP will likely only be using 40 of them).
 
Also, don't forget: NEVER buy memory upgrades from Apple. Nor SSD's. Do those yourself. It's pointless to overspend on RAM by getting it from Apple.

I also think you might want to look into a custom box. I don't know what kind of processing power you need, but it might be worthwhile to look into getting a Linux-based power machine that you control with a relatively low-end Mac Pro. That might give you some flexibility and power, if it works for you.
 
does anyone know how likely it would be for an upgraded mac pro to be expandable to >=128GB memory?

The processors support up 256GB through 8x32GB quad-ranked DIMMs. It seems likely the Mac Pro will only have 4 per CPU again unless they change the enclosure.

are there any hardware-fixes to expand the current generation mac pros beyond 64GB of memory?

Yeah you can put bigger DIMMs in them. OWC sell 16GB quad ranked RDIMMs and have tested it, but couldn't get the second bank working - so limited to 1 per channel or 96GB total. 32GB DIMMs should work exactly the same way.

also, does anyone know if mac pros (current or possible upgrades thereof) are compatible with NVIDIA's Tesla GPUs?

People have them working on Mac Pros, I don't know if any one is using them within OS X or not.

and finally, I'd appreciate any advice regarding how long I should wait for an upgrade before investing.

Other companies are expecting to ship in March when Intel have released the new processors. Apple have usually shipped within 2 months and no reason to expect any longer than that.

If you aren't going to be using OS X I'd steer clear of a Mac Pro and go with whatever Dell or HP offer as they will likely have twice the memory slots and faster processors available, plus Tesla cards and memory all covered by onsite 3-5 year warranty.
 
I recently got a research grant of $40k to buy a personal (super)computer, and am anxious about the possibility of a Mac-Pro update. I would prefer a mac, but right now there are no options for memory upgrades above 64GB, or processor upgrades beyond 12 cores.

I know that the mac pro will likely be upgraded or killed in the relatively near future. Assuming that it is upgraded,

does anyone know how likely it would be for an upgraded mac pro to be expandable to >=128GB memory?

are there any hardware-fixes to expand the current generation mac pros beyond 64GB of memory?

also, does anyone know if mac pros (current or possible upgrades thereof) are compatible with NVIDIA's Tesla GPUs?

and finally, I'd appreciate any advice regarding how long I should wait for an upgrade before investing.

Thanks!

A couple thoughts from a research science grad student who spends too much time musing about things he would buy with infinitely much money.

1. Don't try to sink all of that money into a single computer. Just don't.

2. The current Mac Pros aren't compatible with the Tesla GPUs. If you're interested in using CUDA, there are other CUDA capable cards available for the Mac Pro. The Quadro 4000 has a Mac version, and there's a newer one as well, as well as the odd GTX cards. See here: https://forums.macrumors.com/threads/1323132/ . Alternately, the ATI cards that come in the current generation of Mac Pros are capable OpenCL cards.

3. With 40 grand, you're starting to sink into smallish cluster territory. I'd do the following:

3a. Buy yourself a nice workstation/client/code mockup machine. A Mac Pro (or MBP if you spend lots of time traveling) with plenty of memory. A nice screen (I suggest a Dell Ultrasharp 23 inch or two). Plenty of storage. A quality mouse and keyboard, etc. Make this machine nice, but don't try to turn it into tens of thousands of dollars powerhouse.

3b. If you're at a university, talk to your university's high performance computing people. Many will allow you to buy "condo" nodes - essentially you pay to put nodes into the main cluster, and you get priority on those nodes, but they manage them, deal with software and hardware, etc. Heck, at my university, folks doing that even get higher access to other nodes. That's a nice, stress free way to give yourself computing power.

3c. If you don't want to do that, build yourself a small, modest cluster. A maybe four nodes, a nice shared storage system, and (hopefully) the space to store it all. If you need more power than this, it's a good place to run your code as a testbed before using something like Amazon EC2 to build a much larger on-demand cluster for running a job.

As much as I hate to admit it, for something like this, I'd probably use Linux machines. The hardware is cheaper, the OS more flexible, you have access to more potential components to use if you're building them yourself, and more vendors if you're buying them, Tesla cards, etc.

But if you're dead set on using Macs, you can make credible compute nodes out of Mac Minis, or use another Mac Pro as a second GPU-based computing machine. I've been musing about specing out and testing hackintosh compute nodes that are more purpose suited than minis, but haven't the time or funding.

Either Pooch or XGrid are decent enough tools to administer an all-Mac cluster, or you can use something like Rocks if you go with a Linux setup. I will tell you that a Mac Pro workstation/client plays rather nicely with a Linux based cluster.
 
If I had 40K for a computing solution and at a University, I'd highly recommend FluJunkie's 3b option. Having someone else manage a cluster will save you a lot of time. Also, with 40K, you could influence the types of nodes being put on the system. Ie. do you want 4 nodes with 12 cores and 24 GB RAM systems, or a monster 32 core, 1 TB RAM SMP?

And I notice you're worried about max RAM, that leads me to think you probably want something like a Dell PowerEdge R910. You could load that up with 4 E7-4850s and 512 RAM and get out the door for around 40K. Like FluJunkie said though, sinking all your money into one computer is probably a bad idea.

A nice place to look around for small clusters and SMPs is silicon mechanics: http://www.siliconmechanics.com/

Some one I've worked with built a small system with them and have been very happy with it and their customer service. Also, AMD offers some nice value in these systems.

So, if it where me and I couldn't spend that money through a university cluster, I'd maybe sink 20-30K into an appropriate solution at silicon mechanics, then buy 1-2 new Mac Pro's when they come out. Throw in a MacBook Pro and I think I'd be done.

Out of curiosity, what are you doing? Is it something that works well with distributed RAM?
 
2. The current Mac Pros aren't compatible with the Tesla GPUs. If you're interested in using CUDA, there are other CUDA capable cards available for the Mac Pro. The Quadro 4000 has a Mac version, and there's a newer one as well, as well as the odd GTX cards. See here: https://forums.macrumors.com/threads/1323132/ . Alternately, the ATI cards that come in the current generation of Mac Pros are capable OpenCL cards.

To be perfectly frank, there are no cards on the Mac that are in the same league as the Tesla. neither the Quadro or the 5870 are at all comparable.

Tesla generally wipes the floor with any graphics focused GPU.
 
To be perfectly frank, there are no cards on the Mac that are in the same league as the Tesla. neither the Quadro or the 5870 are at all comparable.

Tesla generally wipes the floor with any graphics focused GPU.

Agreed. Hence my recommendations. I said "capable" meaning "can run CUDA-type applications." I consider the Quadro (or more accurately probably a cheaper GTX) card to be exactly suited for what I use a Mac Pro for. That is to say, mocking up and testing code before it goes to frolic amongst the cluster nodes.
 
what sort of computation will you be doing?

IMHO if you need a mac environment I would scour ebay for a ton of 2008 xserves as they are going for less than $800 a piece for the 8 core 2.8ghz.
 
Thanks everyone for the very helpful information, advice, and suggestions!

More information as to the particulars would be extremely helpful, as you may be able to be served properly with a single box PC system (i.e. up to 4x CPU's), or a cluster (either based on CPU's or GPU's, if the software you're using supports GPGPU processing).

If I had 40K for a computing solution and at a University, I'd highly recommend FluJunkie's 3b option. Having someone else manage a cluster will save you a lot of time. Also, with 40K, you could influence the types of nodes being put on the system. Ie. do you want 4 nodes with 12 cores and 24 GB RAM systems, or a monster 32 core, 1 TB RAM SMP?

...

Out of curiosity, what are you doing? Is it something that works well with distributed RAM?

My research involves computational physics, and some of the computations I encounter are difficult to parallelize and can occasionally require large memory resources (not long ago, I narrowly squeezed through a bottle-neck requiring ~110GB memory, which was luckily just possible).

Although I already have access to a large cluster, there isn't a single machine (available to me) at my institution with more than 128GB of memory. And so my principle challenge is to outfit a single node with lots of memory.

I have not made much use of GPU computing yet, but it seems well-suited for the problems I work on (which involve a lot of linear algebra), and am eager to see if it can be used to increase efficiency. I'm worse-than a novice regarding GPU computing, but I wanted to keep my options open—and I know that Mathematica 8 supports some degree of GPU utilization.

Actually, on that note: does anyone have any experience with this in Mathematica? The documentation mentions CUDA, but I don't know anyone who has made use of this yet.


A couple thoughts from a research science grad student who spends too much time musing about things he would buy with infinitely much money.

...

3b. If you're at a university, talk to your university's high performance computing people. Many will allow you to buy "condo" nodes - essentially you pay to put nodes into the main cluster, and you get priority on those nodes, but they manage them, deal with software and hardware, etc. Heck, at my university, folks doing that even get higher access to other nodes. That's a nice, stress free way to give yourself computing power.

Thanks FluJunkie for the thoughtful advice. This is also the case at my university, and I've been planning to balance a powerful office mac pro with a cluster-node donation.

The impetus of my question was really to assess the plausibility of a hope that the next-generation Mac Pro would be capable of memory upgrades sufficient to localize the primary machine to my office. Knowing that memory on the order of (even fractions of) terabytes can easily cost $10k's, I am eager to avoid investing in a PowerEdge or similar if a such a Mac Pro becomes available in a few months' time.

While I'm aware that Apple's product line tops-out quite below what is available in linux boxes, am I crazy to hope that this will change significantly with a Mac Pro update?
 
My research involves computational physics, and some of the computations I encounter are difficult to parallelize and can occasionally require large memory resources (not long ago, I narrowly squeezed through a bottle-neck requiring ~110GB memory, which was luckily just possible).

Although I already have access to a large cluster, there isn't a single machine (available to me) at my institution with more than 128GB of memory. And so my principle challenge is to outfit a single node with lots of memory.

The impetus of my question was really to assess the plausibility of a hope that the next-generation Mac Pro would be capable of memory upgrades sufficient to localize the primary machine to my office. Knowing that memory on the order of (even fractions of) terabytes can easily cost $10k's, I am eager to avoid investing in a PowerEdge or similar if a such a Mac Pro becomes available in a few months' time.

While I'm aware that Apple's product line tops-out quite below what is available in linux boxes, am I crazy to hope that this will change significantly with a Mac Pro update?

I wouldn't hold my breath that the Mac Pro will support >128 GB of RAM. And even if the OS will allow it, its probably not going to have more than 8 DIMMs. Then, to get over 128 you'll need 32 GB sticks, which are crazy expensive and probably will be for a while.

If you really want more than 128 GB of RAM, I think you're going to be forced into Dell or HP workstations, or something from Silicon Mechanics. The Dell PowerEdge R910 is really just the most extreme example if you need 512GB-1TB of RAM. If you think something between 128 and 512 would work the R715 may be a good option. But you're right about the RAM price. There is just no getting around the $400 per 16 GB RAM DIMM, so maybe $7-8K for 512 GB of RAM. And then you'll a motherboard with at least 16 DIMM slots, which is not cheap.

As another option, have you thought about XSEDE? A quick start up grant proposal will get you I think 30K CPU hours on Blacklight. Blacklight has a shared memory system (most clusters use distributed), where they allow 128 GB per 16 cores requested. Even if you stuff can't put use to very many cores, you can just request more to get access to the RAM. So if your process will need 512 GB of RAM, you just request 64 cores, even if it will only run on maybe 8 of them. Then if the start up allocation is not enough you can put in a full computational resource grant and specifically ask for hours on Blacklight.

But I think your best option is really to push your local cluster to buy a PowerEdge 910 and integrate it into their system and give you priority. Since clusters are usually always in the process of adding nodes, maybe $20K would be enough to influence them on their next purchase? That would still give you ample room in the budget for a kick-*ss workstation or 2, or a small high RAM cluster. A group of PIs at my university successfully lobbied for just that same thing, without any monetary input even. And now he R910 with 1TB of RAM is just now up and kicking. So might leave the money off the table first and see what they can do for you. It would probably help to round a few other PIs that could really use the increased RAM. Do you happen to have much in the way of Genomics at your university? Those people need RAM.
 
It really sounds like you're looking at either a cluster, or some sort of giant computing backend.

Get a Mac for your desk, and some sort of big iron for running the big simulations remotely, if possible. When I was doing high performance academic work, that's what I did. I had a 8 core Mac Pro at my desk, and if I needed to run a model faster, I'd ship it over to our Tesla/16 Core Xeon box running elsewhere on the network.

That way if you have multiple researchers everyone stays happy too.
 
My research involves computational physics, and some of the computations I encounter are difficult to parallelize and can occasionally require large memory resources (not long ago, I narrowly squeezed through a bottle-neck requiring ~110GB memory, which was luckily just possible).

Although I already have access to a large cluster, there isn't a single machine (available to me) at my institution with more than 128GB of memory. And so my principle challenge is to outfit a single node with lots of memory.

I have not made much use of GPU computing yet, but it seems well-suited for the problems I work on (which involve a lot of linear algebra), and am eager to see if it can be used to increase efficiency. I'm worse-than a novice regarding GPU computing, but I wanted to keep my options open—and I know that Mathematica 8 supports some degree of GPU utilization.

Actually, on that note: does anyone have any experience with this in Mathematica? The documentation mentions CUDA, but I don't know anyone who has made use of this yet.
You could get away with a DP system, but not the MP (actually, it should be able to support the memory so long as it has 1x slot per memory controller = 8x DIMM slots, but it's the PCIe lane configuration that will cause you the biggest problem).

So here's a rough idea:
  • Non-Apple branded DP system using 2x SB LGA2011 Xeons (fast clocks if at all possible as a means of better performance for whatever processes cannot be parallelized or run in GPGPU). The 80 PCIe lanes is what you're truly after with this machine.
  • Even at 1x DIMM per memory controller, you'll be able to get 128GB using 16x DIMM's. BTW, Samsung created 32GB DDR3 RDIMM's, which would allow at least 256GB that way as well.
  • Install 4x Tesla's for GPGPU processing (@16x lanes per card, that still leaves 16 lanes, though the actual configuration would be variable, such as 2 * 8x lanes).
  • Use the remaining slots (hopefully 2 * 8x lanes for a hardware RAID card to feed the I/O rate necessary for a fast system, and high-speed networking in order to connect to the cluster at more than 1G Ethernet).
  • Use Linux for the OS, such as Red Hat (IT Dept. should know their way around that one very well if you need assistance).
It's hard to know if this will exceed your budget as system prices aren't out yet, but it would be possible to get a bare-bones system from a decent vendor, and use 3rd party sources for memory, Tesla's, RAID, and networking gear. Doing this, I suspect you'd be able to stay within budget if it's necessary.

As per writing CUDA code for Mathematica, I suspect you'd be able to find online resources of some sort that will put you on the right track.
 
Don't try and turn your workstation into a compute device.

You'll be far better off by getting a reasonably nice computer as a client to run the compute on specialised hardware.

To pimp out a mac pro you're talking BIG dollars.

For that money you can buy a LOT of computing power to put in a rack. Which you can leave running while you take your laptop home, etc.


If it was me, i'd spec up a nice macbook pro to use as the client machine, and spend the rest on server hardware. The cisco UCS machines can take up to 1TB of ram per node (or say 256gb or so using cheaper DIMMS - due to Cisco's patented memory quadrupling technology - they multiplex the memory address lines, splitting each line into 4, to get 4 times as many memory slots working). The C series below has 64 dimm slots - so 256gb (for example) is easily achievable with cheap 4GB dimms...

Maybe something like this:
http://www.cisco.com/en/US/products/ps11587/index.html

Not sure on pricing on the rack mount C series, but we're looking at a cluster of B series blades at the moment, I suspect a single rack mount machine with a colossal amount of RAM (compared to the measly 96gb max in a mac pro) would be well within your budget - we were quoted about 60k AU for 4 B-series blades with 128gb of RAM each, a 10 gigabit fabric, 8 slot blade chassis, etc.

10 pci-e slots too, for CUDA/Tesla goodness...

edit:
i don't work for cisco. but i'm a networking / virtualization nerd, and they have good gear in that space which would do the job...
 
Last edited:
My research involves computational physics, and some of the computations I encounter are difficult to parallelize and can occasionally require large memory resources (not long ago, I narrowly squeezed through a bottle-neck requiring ~110GB memory, which was luckily just possible).

...

I have not made much use of GPU computing yet, but it seems well-suited for the problems I work on (which involve a lot of linear algebra), and am eager to see if it can be used to increase efficiency. I'm worse-than a novice regarding GPU computing, but I wanted to keep my options open—and I know that Mathematica 8 supports some degree of GPU utilization.

One of these things is not like the other. It's my understanding (someone do please correct me if I'm wrong) that the strength of GPUs lies in operations that can exploit tremendous *numbers* of cores, but have relatively low memory requirements - consider that a Tesla card has a single digit GB amount of RAM at the moment.

If your applications are both difficult to parallelize and high memory, then you are indeed better off with a high powered CPU with ludicrous amounts of RAM. GPU computing isn't the magic cure-all its sometimes being sold as.

Although I already have access to a large cluster, there isn't a single machine (available to me) at my institution with more than 128GB of memory. And so my principle challenge is to outfit a single node with lots of memory.

My suggestion is to make them buy one. With $30,000 to spend - lets say you spend $10,000 of that on a *really* kickass workstation - and piggybacking on the existing cluster so you don't have to worry about infrastructure costs so much, I'm pretty sure you could prompt them to pick up a high memory node where you have priority access in the queue. My university, for example, has two 1 TB RAM nodes.

The impetus of my question was really to assess the plausibility of a hope that the next-generation Mac Pro would be capable of memory upgrades sufficient to localize the primary machine to my office. Knowing that memory on the order of (even fractions of) terabytes can easily cost $10k's, I am eager to avoid investing in a PowerEdge or similar if a such a Mac Pro becomes available in a few months' time.

While I'm aware that Apple's product line tops-out quite below what is available in linux boxes, am I crazy to hope that this will change significantly with a Mac Pro update?

Sadly, I think the answer to this is "yes, you're crazy". Lets say my dreams come true, and all of a sudden the new Mac Pro has a Tesla card to go with it, has doubled the available RAM the OS can address, and comes with a pony.

You're still "only" at 192 gigs of RAM, and that's if Apple *doubles* the amount of RAM the OS can address. I feel like that's something Apple would have told us about with the Mountain Lion beta out.

Basically, I'd love to tell you "yeah, hold on a few months, it'll be fixed soon", but I can't. I'm essentially preparing to abandon the Mac platform for anything other than client machines at this point - I had a blog and some projects in mind to use OS X as the computational engine for my work too, but I quit because it was both depressing and more trouble than it's worth.

Consider that the "Science" section of the Apple site still has a link to the Workgroup Cluster - a product not sold anymore, for quite some time. The "Productivity Lab" has tutorials involving Leopard Server and iWeb.

Macs make spectacular client machines for working in a Linux-cluster environment, which is why I'll continue to use them. But unless the current Mac Pro will suit your purposes, I wouldn't hold out for it to get better.
 
Avoid APple

I was in a similar boat a number of years ago and got burned by Apple's decision to leave the HPC market. I worked with their HPC guys to good performance (MPI) on the Xserve and ended up paying a premium for good performance.

Personally, I would go with a Linux box. There is no need to pay the Apple premium for something that will soon be EOL.


Thanks everyone for the very helpful information, advice, and suggestions!



My research involves computational physics, and some of the computations I encounter are difficult to parallelize and can occasionally require large memory resources (not long ago, I narrowly squeezed through a bottle-neck requiring ~110GB memory, which was luckily just possible).

Although I already have access to a large cluster, there isn't a single machine (available to me) at my institution with more than 128GB of memory. And so my principle challenge is to outfit a single node with lots of memory.

I have not made much use of GPU computing yet, but it seems well-suited for the problems I work on (which involve a lot of linear algebra), and am eager to see if it can be used to increase efficiency. I'm worse-than a novice regarding GPU computing, but I wanted to keep my options open—and I know that Mathematica 8 supports some degree of GPU utilization.

Actually, on that note: does anyone have any experience with this in Mathematica? The documentation mentions CUDA, but I don't know anyone who has made use of this yet.




Thanks FluJunkie for the thoughtful advice. This is also the case at my university, and I've been planning to balance a powerful office mac pro with a cluster-node donation.

The impetus of my question was really to assess the plausibility of a hope that the next-generation Mac Pro would be capable of memory upgrades sufficient to localize the primary machine to my office. Knowing that memory on the order of (even fractions of) terabytes can easily cost $10k's, I am eager to avoid investing in a PowerEdge or similar if a such a Mac Pro becomes available in a few months' time.

While I'm aware that Apple's product line tops-out quite below what is available in linux boxes, am I crazy to hope that this will change significantly with a Mac Pro update?
 
The big things to look forward to with the new Mac Pros are faster graphics, Thunderbolt, and SATA III (those are my main things). The RAM clock will get a boost and it will sport a new processor architecture. The 2010 models will still be plenty fast, though. And yes, 96GB of RAM is *current* limit on a dual-processor 2010 Mac Pro model.
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.