Let's rev up some excitement for the 2013 Mac Pro!

[deconstruct60]

Your assertion is correct. Those Xeons you linked are based on the same chips that are in the imacs. They are based on mainstream desktop chips which are also used in light workstations and micro servers. Haswell will not be right around the corner at that time.

Actually those, the Xeon E3 class offerings, will be "around the corner ta that time". The E3 1200 v3 should be out on the market in the June-August 2013 timeframe if Intel launches the overall Haswell line on time.

But yes those are likely to be used across the whole Mac Pro line if going release machines in the same "horsepower class".

However, if look at the benchmarks in the article the E5 2687W versus the E3 1280 v2 are a bit skewed versus the likely Mac Pro line up. First of all Apple will probably not use anything like the E5 2687W. That the maximum hottest running E5 in Intel's line up. Apple didn't use them in 20008, 2009, 2010, or 2012. It is very unlikely they will use one with those characteristics in 2013. Not only the TDP high, but the price is in the nosebleed zone too. Throwing Apple's 30% markup on top of that would assure that relatively no one bought the box (selling thousands worldwide is a colossal waste of time for Apple).

What may happen though if the Ivy Bridge E5 slide out into Sept-Oct is that top end Haswell E3 are about as fast as the entry-midrange Sandy Bridge E5 . Notice how the i7 990X ( Westmere technology and entire tick-tock cycle behind) lags behind the E3 1280 v2. By the time the E3 1280 v3 is on the market the E5 1600 Sandy bridge models will be an entire tick-tock cycle behind also.

If the clock rates are cranked up a bit and the new transactional memory stuff is effective, then on some parallel loads they'll make up some ground on the Sandy Bridge models with just two cores more.

The ivy xeons appropriate to the mac pro wouldn't be out until the second half of next year. Haswell would be at least a year beyond that assuming no hiccups like we had with Sandy.

There is out and then there is "out and available for workstation vendors". Intel released the Sandy Bridge Xeon E5's to super computer vendors back in Nov 2011. Workstations from system vendors didn't get them until April-May 2012. We'll see this Fall if they pull the same stunt. Ship to select vendors for about 6-8 months before ship to "regular system vendors" again. If Intel latches into that pattern there is a chance Haswell E5 will be in boxes in late 2013. They'll just be boxes that cost over $1M.

That said, one of the potential issues with Haswell E5 models is that it is reported they'll use DDR4 memory.

http://arstechnica.com/business/2012/05/ddr4-memory-is-coming-soonmaybe-too-soon/
[**]


Since the DDR4 standard isn't final yet (real soon now), that is a potential slide out of 2013 for even the limited release systems.

It will be one of the implementation differences between the server class and mainstream class of Haswell implementations. Similar, to how Sandy Bridge E5 have PCI-e v3.0 while it took Ivy Bridge for it to trickle down to the mainstream line.


[** The article mentions an Haswell-EX processor. I suspect there won't be one just like there was no Sandy Bridge-EX processor. It is really the -EP (and -E derivative of the -EP ) that is the basis for the E5 line up that is at issue. Mac Pro's won't use E7 ( -EX) Xeons either. Or E5 4600 series. There are more Xeon subseries that won't be used in a Mac Pro then will be.

-EX will likely have to wait again for the Broadwell process shrink to get its "as many cores as possible" implementation out the door. Ivy -EX is projected at 15-16 cores. The next step is probably 19-20. ]

 

[deconstruct60]

This isn't just about pros.

Perhaps you need to look up and see which forum posting in. It is about the Mac Pro. That means in a $2K and up price range.

This is about having a tower version of the Macintosh for Anybody who wants to have a really powerful Mac that actualizes the state of the art of computing technologies.

This is bunch of FUD and disinformation. "Anybody" isn't going to be able to afford a Mac Pro. This looks far more like an attempt to launch the beaten to death and then feed into a meat grinder topic of the mythical xMac.

Newsflash. Rehashing that topic isn't going to build "excitement" for the Mac Pro. Nor is it particularly relevant to the Mac Pro.

 

[wallysb01]

It would solve the problem of lots of copying of data, but the OS schedules may choke on it too. They sometimes have problems with large number of uniformly clocked CPUs. If they are not homogenous that could cause problems too.

If Intel's newly acquired interconnect technology from Cray works out (http://www.theregister.co.uk/2012/04/25/intel_cray_interconnect_followup/ ) then they may be connected with an Aries interconnect ( assuming that doesn't turn out to be very expensive).
Aries is PCI-e links; at least along the connection to the CPU packages. So perhaps not QPI but at least a link aimed at connecting CPUs.

Interesting article. I was not aware of these Cray interconnects. Sounds like it could be a great option.

 

[deconstruct60]

Interesting article. I was not aware of these Cray interconnects. Sounds like it could be a great option.

If you have that kind of money. They aren't exactly affordable. However, if the interconnect only needed one or two PCI-e x16 sockets could get to designs where a very close proximity cluster expander box could house a good sized cluster of Phi nodes.

It will more naturally fit into Blade and rack server form factors but the deskside, individual user, 1 TFLOP double precision sustained workstation isn't that far away. Wouldn't necessarily need a machine room and nor would it have to sound like a jet engine.

Microway, or some other HPC fouced system vendor, will probably do it first but eventually the components may become commonly accessible enough that Apple will take a stab at it. Probably not in 2013 (or 2014 or 2015) but eventually if still in the Mac Pro market several years out.

It isn't so much that Intel has it. It is more so that Intel is probably interested in making it a "component" that they sell to multiple system implementer vendors. Maybe not to every "chop shop" clone vendor in China. But even just 3-8 vendors is a huge jump from just one.

 

[xgman]

This is bunch of FUD and disinformation. "Anybody" isn't going to be able to afford a Mac Pro.

An "non-pro" enthusiast that happens to have a nice income could easily afford one. More power to them. (literally)

 

[derbothaus]

An "non-pro" enthusiast that happens to have a nice income could easily afford one. More power to them. (literally)

That is a big chunk of computer users period. The 100% income from computing crowd uses what gets them for by as long as possible therefore maximizing income Or they need to maximize right-off's for taxes.
It's really the Video Pro's that are the yearly upgraders in my experience. Audio is staying put as well as new OS is ***** for specialized applications thanks to new sandboxing.

 

[StephenCampbell]

Perhaps you need to look up and see which forum posting in. It is about the Mac Pro. That means in a $2K and up price range.




This is bunch of FUD and disinformation. "Anybody" isn't going to be able to afford a Mac Pro. This looks far more like an attempt to launch the beaten to death and then feed into a meat grinder topic of the mythical xMac.

Newsflash. Rehashing that topic isn't going to build "excitement" for the Mac Pro. Nor is it particularly relevant to the Mac Pro.

What are you talking about? I am technically not a pro, but I only care for the most powerful Mac possible, because I want it to last a long time and to be able to do things really really quickly.. be it opening an application, or rendering a video.

My topic was always and only about the Mac Pro, the only Mac that I am interested in purchasing.

 

[wallysb01]

If you have that kind of money. They aren't exactly affordable. However, if the interconnect only needed one or two PCI-e x16 sockets could get to designs where a very close proximity cluster expander box could house a good sized cluster of Phi nodes.

Well, if you actually have to pay for your time on a cluster, a lot of things will make sense at ~$0.01/CPU*hour. For users like me, it could very well be worth an extra $5000 to have a machine that can have >50 slow cores and a handful of fast ones. There is a lot of over head in moving things to clusters, installing the programs you need on sometimes funky linux distributions where yo don't have root privileges, and waiting through queues beyond just the CPU*hour costs besides. I think a lot will depend on software side of things though. Right now, at least in my work, many things still only scale well to about 16 cores, but many do well with basically infinite cores too. That's how the handful of fast cores + bulk slow ones would balance well. However, if more things start scaling better past 16 cores, I'd rather just see 2x16 core Xeon workstations in say 2015 than these 50-100 atom level processors.

Microway, or some other HPC fouced system vendor, will probably do it first but eventually the components may become commonly accessible enough that Apple will take a stab at it. Probably not in 2013 (or 2014 or 2015) but eventually if still in the Mac Pro market several years out.

And I think Apple would be wise to do so. It would allow for the whole line up to be plugged into this cluster-in-a-box. Maybe that's the direction they'd like to take the Mac Pro in general, as a simple expansion box. I think it would be a shame to lose something similar to a traditional workstation, but it would be potentially more versatile. What it might allow for is a consolidation of the Mac Pro into just the mid-level DP model at $5000+, then our little cluster box sitting in the ~$3000 price range?

 

[avemestr]

I presume that isn't the 2010 but the 2012 12 core 2.4GHz model. The 2010 2.4GHz model has 4 less cores. There would be a larger jump with the entry dual package Sandy Bridge model past 10-20% for a baseline of 8 Westmere cores.

Nope, actually I was heading for the 2010 model with 2x5620. The prices at my locale are the following:

2010 2x2,4x8: 3130 USD (new from retailer with stock in excess)

2012 2x2,4*12: 4430 USD (from Apple with education discount - ordinary price is 5040)

Now, the 2012 model performs about 50% faster than the old model, when it comes to CPU tasks, it seems.

But, I could take the $1300 saved, and direct them towards after-market SSD, a GTX 570 or 680 and more RAM, which would all probably benefit me more every day than the processing power alone.

I'm mostly in the market for a Mac Pro due to gaming, light handbrake and FCP X usage. In other words: I'm more in the "want" than in the "need" camp. Also, tax write-off would be nice this year instead of next. So, money is an issue, but I'm not heading in the Hackintosh direction.

However, illustrative of the Osborne effect directly in action. Canceled sales now for product in the somewhat distance future.

Yep, entirely true. Of course partly because I'm a wanter more than a needer. But I can see why "real needers" jump ship, with the suspense about the future model and the current offering being not-so-compelling compared to Z820 or similar.

 

[deconstruct60]

Well, if you actually have to pay for your time on a cluster, a lot of things will make sense at ~$0.01/CPU*hour. For users like me, it could very well be worth an extra $5000 to have a machine that can have >50 slow cores and a handful of fast ones.

A Phi powered single PCI-e card has 60 cores and 6GB. So there is limited motivations for an interconnect just on that kind of core count. Perhaps for a 10GB memory on the PCI-e card instead of 6GB but not necessarily for a $500-1200 intereconnect. That much more in memory would largely solve the problem.

"... TDP e.g. Thermal Design Power also underwent significant changes. Intel now offers two 245W boards (57C/3GB and 60C/6GB) and three 300W designs (57C/6GB and two 61C/8GB) and is available in two configurations - with or without passive heatsink. Active heatsink (the old Larrabee design) is only available with the 57C/6GB/300W version, while top range configurations (G5756-200 and G5757-200) come either without any thermal solution (probably for testing the liquid cooling and 3rd party designs, such as liquid cooling design from CoolIT Systems, which dates back to 2009), or with a passive heatsink "
http://vr-zone.com/articles/intel-x...ht-in-shining-armor-/16871.html#ixzz24a2euuLu

The more substantive problem if want these inside the Mac Pro case is the 300W TDP issue. This is one issue that motivates an updated case design.

It seems though that Intel is still on a strategy where they are only going to sell these boards to system vendors. So Apple would need to construct a product this fits inside of. It will probably come in under the $5000 budget since it appears that the Tesla K20 is reported to be targeted at around Tesla $3,199-3,999 and the AMD Firepro 9000 around $3,999 . Those two are going to hold the Phi cards back from $5,000 targets. My guess is that Intel will charge more $4,200-4,500 because it is supposedly easier and the cards are running software. Plus the system vendors will have their own integration/support costs to tack on. If it is easier to port established MPI code they can probably get away with it. However, its sustained FLOPS capacity isn't likely to exceed those cards so they can't charge too much.

There is a lot of over head in moving things to clusters, installing the programs you need on sometimes funky linux distributions where yo don't have root privileges, and waiting through queues beyond just the CPU*hour costs besides.

Well Phi cards aren't going to get ride of Linux kernels; the card has its own. But it is single "user" so probably can get ahold of the root password or at the very least sudo privileges. In a shared use envrionment handing out root priv primarily just means handing out trouble later. Someone is bound to tweak things so it is optimal for them rather than the group.

I think a lot will depend on software side of things though. Right now, at least in my work, many things still only scale well to about 16 cores, but many do well with basically infinite cores too.

That is probably where the Phi cards do pretty well. It think using the majority of the cores will be dependent upon some vectorizing compiler split/join only critical sections of the computations. The cap will be that the memory local to the cards won't support anything like 500MB/core data allocations.


The interconnect they got from cray will probably look far more like this in a Xeon E5 + Xeon Phi set up.




[interconnect chip ] < --- x16 Pci-e v3 ---> [ Xeon E5 ] <--- x16 PCI v3 --> Phi card
<__|------------ x16 PCI-e v3 --------------> [ Xeon E5 _] <-- x16 PCI-e v 3 link ---> Phi card


The interconnect would connect another E5 node of the shared memory.

QPI is 25GB/s and HyperTransport 3.1 is 51GB/s you would need two x16 PCI-e v3.0 links to get into the 32GB/s range. For the immediate future Phi would need some major rework so that it had x32 PCI-e v3.0 links coming out. With a pair of E5's the interconnect can work to 'borrow' one from each and be smart about snagging memory from the correct member of the pair that 'owns' that locally shared memory.

If the 2014-2015 Phi change to use DDR4 like the Haswell Xeons then they could probably be equal co-nodes on the interconnect. Until then though it would be something different and I suspect for volume sales sake there would still be a more plain x16 PCI-e Phi model for card usage to compete head-to-head with the GPUs. The x32 PCI-e model would be mounted on special motherboards with the E5s and interconnect. That wouldn't go into Mac Pro class boxes.

And I think Apple would be wise to do so. It would allow for the whole line up to be plugged into this cluster-in-a-box. Maybe that's the direction they'd like to take the Mac Pro in general, as a simple expansion box.

I think it is far more critical that Apple figure out the cluster-inside-a-box first. Dual card set ups where there is one mid-high powered GPU + one mid-high power GPGPU focused card.

That configuration is going to be far more common than the 3-4 GPGPU card configuration.

What it might allow for is a consolidation of the Mac Pro into just the mid-level DP model at $5000+, then our little cluster box sitting in the ~$3000 price range?

The Mac Pro doesn't survive if it only starts out at the mid level dual package price point. It would stagnant due to lack of growth. It is too narrowly scoped.

The empty external cluster box would likely cost $2000. Let alone populating it with multiple Phi cards. What you are talking about is external bandwidth akin to high end Inifinband. Similar bandwidth and latency. The network by itself will cost and the huge power requirements by the GPGPU focused cards also drive up the enclosure cost. It would be closer to being a $10-11K box and that's why Apple will probably stay away from it. The market is going to be small and not so scalable.

The more major upside would be bringing more development and "smaller scaled problem" work to the individual Mac Pros. Not necessarily having a box that suited to be fitted into a multiple box cluster.

 

[StephenCampbell]

Do you guys think that the new tower will aesthetically match the current Thunderbolt display, or will its release see a new aesthetic/color scheme along with a new display to match it?

 

[greenmeanie]

I left for Oracle Linux.
I hate windows and windows 8 is a joke.

I moved on to windows. Good luck waiting.

 

[JesterJJZ]

Do you guys think that the new tower will aesthetically match the current Thunderbolt display, or will its release see a new aesthetic/color scheme along with a new display to match it?

I don't think it will change too drastically.

 

[FluJunkie]

Is it just me, or is discussing the idea of "Cluster in a Box" regarding Apple when they just killed their easy, straight forward cluster/grid creation utility a little absurd?

 

[twietee]

Do you guys think that the new tower will aesthetically match the current Thunderbolt display, or will its release see a new aesthetic/color scheme along with a new display to match it?

I don't think it will change too drastically.

Me, too. Look at the retina MBP; it's the new official flagship and nothing really changed so I'd expect aluminum. One shouldn't be concerned about the form/aesthetics imho, I have full confidence in apple regarding these aspects. One could be much more worried about the internals.

 

[derbothaus]

Is it just me, or is discussing the idea of "Cluster in a Box" regarding Apple when they just killed their easy, straight forward cluster/grid creation utility a little absurd?

No. I think you're on point with that one.

 

[wallysb01]

Is it just me, or is discussing the idea of "Cluster in a Box" regarding Apple when they just killed their easy, straight forward cluster/grid creation utility a little absurd?

But the game has changed a lot since then. The Xserve was meant for a cluster, the cluster-in-a-box is meant for a single user. If in the next couple years the price allows it, a MacBook pro or Mac pro + an "iCluster" (let's call it), could be a nice combo for many users. I think Deconstruct is right, and that the most important thing to do is to get these GPGPU cards working inside a Mac pro first, but with thunderbolt speed increases and potential price drops, the expansion box is worth a serious thought.

 

[FluJunkie]

But the game has changed a lot since then. The Xserve was meant for a cluster, the cluster-in-a-box is meant for a single user. If in the next couple years the price allows it, a MacBook pro or Mac pro + an "iCluster" (let's call it), could be a nice combo for many users. I think Deconstruct is right, and that the most important thing to do is to get these GPGPU cards working inside a Mac pro first, but with thunderbolt speed increases and potential price drops, the expansion box is worth a serious thought.

I'm talking about XGrid, which got terminated when Mountain Lion hit the App Store.

The hardware you could produce swiftly. But for a "Cluster in a Box" to fly in retail computing, it has to work without the usual fuss of most computing packages. Apple had a software package that attempted that - they've just now killed it. I think it's a stretch to say "Oh yeah, tiny clusters are the way they're headed".

 

[deconstruct60]

Is it just me, or is discussing the idea of "Cluster in a Box" regarding Apple when they just killed their easy, straight forward cluster/grid creation utility a little absurd?

No. When the cluster is inside of one box that specific software is obsolete or at best applied to different class of problems. The whole point of Xgrid is to make multiple instances work together because the jobs were "too much" for a single computer. If just have 1 instance, one box, there is no "grid". If just have two talking to each other over single, internal virtual network ( over PCI-e link to classic GPGPU cards or the virtual high speed hiperchannel link link to the Phi card(s) over PCI-e ) again there is no external grid.

When Xgrid first came along there were single core CPUs Macs. Even just duals did appear until relatively recently portion of Xgrid's service lifetime. For instance, a rack of 37 1U XServer G5 with dual processors comes out to 74 cores. Compare that to a dual E5 2670 (8 cores each ) and a 60C Phi card which totals to 76 cores. That is a single box for what a rack of boxes used to do.

The memory footprints comes out roughly the same too. The G5 maxed out at 16GB. So that rack of 37 would max out around 592GB. A z820 maxes out at 512GB and thrown in another 6-8GB for the Phi card. So ballpark 520GB. A revised Mac Pro would probably max at 256GB (with upcoming 16GB DIMMs ) so it would be a factor for very large footprints jobs. But people aren't doing extremely large footprint jobs "stealig cycles" from the normal provisioned computer down the hall either.

The "attack of the killer microprocessors" not only took out the classic vector Supercomputer machines it is moving onto the next victim. Small scale clusters are next. The microprocessors" have kept getting smaller so now what used to be collections of large boxes are now also at risk.

 

[deconstruct60]

But the game has changed a lot since then. The Xserve was meant for a cluster, the cluster-in-a-box is meant for a single user.

XServe wasn't necessarily meant for a cluster. It is oriented to being in a machine room and a rack. That doesn't necessarily mean cluster.

, but with thunderbolt speed increases and potential price drops, the expansion box is worth a serious thought.

I think Thunderbolt would be primarily be useful only in evicting the x4 slots from a Mac Pro: at least clearly in the case of a dual E5 box. 3-4 x16 PCI-e slots along with 0-1 x8 PCI-e slot. If going to be limited to just four then, the fastest four possible. "Mid range" speed stuff can be kicked to Thunderbolt without taking up much internal room.

Only an internal network that is both substantially faster and cheaper than an external very high speed network will reap larger benefits.

The prices won't drop short-intermediate term, but the scope of the workloads can do with a single box will get bigger.

 

[deconstruct60]

I'm talking about XGrid, which got terminated when Mountain Lion hit the App Store.

The hardware you could produce swiftly. But for a "Cluster in a Box" to fly in retail computing, it has to work without the usual fuss of most computing packages. [/quote]

No you don't. With a NUMA SMP box the vast majority of the crufty overhead of a cluster disappears. You are missing the forest for the tress. People use a cluster to get work done. Nobody "wants" to use a cluster. It is used because largely can't afford to buy one box that gets the job(s) done. This core issue is now an increasingly larger number of people can put 1-2TFLOP loads just on the box on/beside their desk.

Even setting up a nominally single user system to talk to a hosted Phi card running a customized Linux instance. There is no need for a general purpose grid software. The number of configurations are fixed. Only hooking to one other instance inside the same box isn't that hard. There is no mutiuser batch queue need that is outside the bounds that the OS X multiuser batch system can't handle. Pragmatically, it is really a single user supercomputer.

So all of the "decision making" that XGrid sets up for "who" to do cycle scavenging from and which/how to copy the jobs to an from remote boxes largely disappears. The data , users , and resources are all local. Localizing the cluster into a fixed container makes it simpler to manage.

 

[FluJunkie]

Nobody "wants" to use a cluster. It is used because largely can't afford to buy one box that gets the job(s) done. This core issue is now an increasingly larger number of people can put 1-2TFLOP loads just on the box on/beside their desk.

I'm well aware of that - it's the core of my assertion.

What I'm saying is that Apple is going in a decidedly "Not That Way" direction with the discontinuation of XGrid.

If normal clusters are a 1, and magic Teraflops On Your Desk That You Don't Have To Configure is a 10, then XGrid is a 5.

Sure, it's possible Apple would hop from 1 to 10, but it's a little odd that they've discontinued support for 5 entirely, aren't working on its development, and have been fairly closed to the community involved. Basically, the company that's going to put a "Cluster in a Box" on your desk isn't the one who sends out an email recommending Condor and SGE.

 

[deconstruct60]

I'm well aware of that - it's the core of my assertion.

Your assertion is disconnected from reality.

If normal clusters are a 1, and magic Teraflops On Your Desk That You Don't Have To Configure is a 10, then XGrid is a 5.

Sure, it's possible Apple would hop from 1 to 10, but it's a little odd that

Apple never was at 1. It is extremely contrived that they are making a transition from 1 to 10. Apple was never in the cluster system selling business. They make smaller systems; not "big iron". They are not moving left-to-right. They are moving right-to-left. XGrid is to the left of "10". Apple is coming from the ">10" range on your number line. The "10"
here is being introduced to partially wipe out the "5 area". There is no intent to complete wipe out that space or to eventually start selling 1's.

Mac Pro's are loosing workload to iMacs and Minis. (and they more general PC competitors). All the Mac Pro needs to do is steal enough from the levels above it to remain viable.

Basically, the company that's going to put a "Cluster in a Box" on your desk isn't the one who sends out an email recommending Condor and SGE.

XGrid is really in a smaller subclass than Condor and SGE. Those are heterogeneous grid management/distribution systems. In the cycle scavenging context that XGrid had most traction in that is a gross limitation over the long term. XGrid's fundamental weakness is being homogenous. That had traction as long confine the solution space of walled garden of just Macs to work with.

Furthermore, they didn't recommend those two. They were listed as alternatives, but the recommended one was Pooch. Even Pooch though isn't homogenous.

The problem of integrating heterogeneous computers isn't Apple's primary focus. They are generally going to hand that sort of integration to 3rd parties as an opportunity.

 

[wallysb01]

XServe wasn't necessarily meant for a cluster. It is oriented to being in a machine room and a rack. That doesn't necessarily mean cluster.

True, XServe's use in legitimate clusters was rare. It was more suited to the "mini-cluster" or rack of servers for individual labs/small businesses.

This has been an interesting conversation. I really do hope Apple has something in store for us regarding a Mac Pro with this kind of GPGPU/Phi type capabilities. Even if its just a $4000 BTO, with a redesigned case to fit, it will certainly help reestablish the Mac Pro as powerhouse workstation.

Speaking of which, in what ways do you think the case could change assuming it might need to cool these ~300W cards. Also, do you think the Mac Pro will expand to 12/16 DIMM slots to make higher memory configuration more easily possible with 16/32 GB DIMMS?

 

[derbothaus]

True, XServe's use in legitimate clusters was rare. It was more suited to the "mini-cluster" or rack of servers for individual labs/small businesses.

This has been an interesting conversation. I really do hope Apple has something in store for us regarding a Mac Pro with this kind of GPGPU/Phi type capabilities. Even if its just a $4000 BTO, with a redesigned case to fit, it will certainly help reestablish the Mac Pro as powerhouse workstation.

Speaking of which, in what ways do you think the case could change assuming it might need to cool these ~300W cards. Also, do you think the Mac Pro will expand to 12/16 DIMM slots to make higher memory configuration more easily possible with 16/32 GB DIMMS?

Lifting the 96GB OS addressable limit would be step 1. Then at least the 128GB currently doable is doable. Then we can start talking more DIMM slots.