Crazy idea. 32GB RAM modules in a cMP. Anyone tried this?

[Kon_Kipa]

@Kon_Kipa

Looking at your 8x32GB setup, your memory timings under windows are 6 6 6 15 which is 800 MT/s and your cycle time is 10 ns.. Your memory is effectively running at DDR3 800 or PC3-6400 speeds.

Stick with 6x32GB (192GB) if you manage to get them to load under your 5,1/X5690 build or as you planned 4x32GB and 2x16GB (160GB). Aim for 1066 MT/s data rate as you cannot achieve 1333 MT/s data rate (which is the default for the 5,1) with that amount of memory installed.

Your 5,1/X5690 benchmarks and memory transfer speeds are going to suffer and be lower.

yup i noticed the drop is memory speed when i installed all 8 modules..

I'll do some more testing & follow up with findings/results

Wont be until later on this week tho (maybe the weekend)

 

[edgerider]

yes ram disk are not faster than nvme raid for most of the task, but for lightroom, première and after effect cache it is definitely more snapier than if you have the cache on the same ssd as the media.
any how we are hitting the ceiling on what those machine can do ...

 

[AidenShaw]

Your 5,1/X5690 benchmarks and memory transfer speeds are going to suffer and be lower.

See https://forums.macrumors.com/thread...nyone-tried-this.2046693/page-3#post-27174174

Ask yourself if you want faster artificial memory benchmark numbers, or faster real world application benchmarks.

If your workflow can use the RAM, the benefits in real world applications far exceed the artificial benchmark "losses".

If your workflow doesn't need the RAM, save money and don't buy it.
[doublepost=1552432665][/doublepost]

yes ram disk are not faster than nvme raid for most of the task

Also note that any real SCSI/SATA/NVMe disk transfers data using DMA without CPU involvement.

A RAMdisk uses CPU to transfer each byte.

If your workflow is CPU-bound, the CPU overheard of a RAMdisk could result in less throughput using a RAMdisk.

 

[handheldgames]

yes ram disk are not faster than nvme raid for most of the task, but for lightroom, première and after effect cache it is definitely more snapier than if you have the cache on the same ssd as the media.
any how we are hitting the ceiling on what those machine can do ...

perhaps, but 4k ramdisk performance can't be touched by any raid or stand alone pcie ssd, nvme or not.

 

[kazkus]

I have two MacPro 2009 (MacPro5,1), one has dual CPU tray and another has single CPU tray.
I had been using 8GB memory modules, 64GB for the dual CPU MacPro and 32GB for the single CPU MacPro.
Since the memory price is recently going down, I upgraded them with 16GB memory modules, 128GB and 64GB.
The dual CPU tray has two X5680, and the single CPU tray has X5660.

Yesterday, I received 8 used 32GB memory modules from an eBay shop.
Samsung M393B4G70BM0-YH9 https://www.samsung.com/semiconductor/dram/module/M393B4G70BM0-YH9/
I put all 8 modules to the dual CPU tray, and it booted Windows 10 Pro and recognizes 256GB memory.

 

[LightBulbFun]

very cool stuff

any luck booting OS X or such?

 

[Soba]

Yesterday, I received 8 used 32GB memory modules from an eBay shop.
Samsung M393B4G70BM0-YH9 https://www.samsung.com/semiconductor/dram/module/M393B4G70BM0-YH9/
I put all 8 modules to the dual CPU tray, and it booted Windows 10 Pro and recognizes 256GB memory.

This is awesome! I notice that you're running BootROM version 144, so I wonder whether this is a very recent change.

 

[handheldgames]

I have two MacPro 2009 (MacPro5,1), one has dual CPU tray and another has single CPU tray.
I had been using 8GB memory modules, 64GB for the dual CPU MacPro and 32GB for the single CPU MacPro.
Since the memory price is recently going down, I upgraded them with 16GB memory modules, 128GB and 64GB.
The dual CPU tray has two X5680, and the single CPU tray has X5660.

Yesterday, I received 8 used 32GB memory modules from an eBay shop.
Samsung M393B4G70BM0-YH9 https://www.samsung.com/semiconductor/dram/module/M393B4G70BM0-YH9/
I put all 8 modules to the dual CPU tray, and it booted Windows 10 Pro and recognizes 256GB memory.

View attachment 843012 View attachment 843013 View attachment 843014

Thanks for sharing.

While the 32gb dimms are running at a 1T command rate and has good timings, it's running at 400/800Mhz, compared to 16GB dimms running with 2T at 666/1333Mhz.

It would be interesting to see the benchmark and timings with 32x3 for each CPU. A test in MacOS would also be appreciated.

 

[LightBulbFun]

Thanks for sharing.

While the 32gb dimms are running at a 1T command rate and has good timings, it's running at 400/800Mhz, compared to 16GB dimms running with 2T at 666/1333Mhz.

It would be interesting to see the benchmark and timings with 32x3 for each CPU. A test in MacOS would also be appreciated.

thats because they are Quad Ranked 4Rx4 DIMMs and he is running 2 of them on the same channel, if he ran with 3 DIMMs before CPU, then they would run at 1066Mhz with 7-7-7 timings

(the CPUs memory controller only supports 1333Mhz on 2Rx4 RAM sticks)

(also the MP generally always runs the RAM at 1T regardless of speed)

 

[kazkus]

It successfully boots Windows 10 Pro and MemTest86 V8.2,
with some minor exceptions of memory configuration especially with single CPU tray,
but no luck for macOS, so it should be macOS issue.

macOS Mojave 10.14.5 does not boot.
It stops just after Apple logo displayed.
With Verbose mode (command+V), "End RandomSeed" is the last message.
It might be related with checking memory.
I have tried several memory configuration as follows.
256GB = 8x32GB
192GB = 6x32GB
160GB = 4x32GB + 2x16GB
160GB = 6x32GB
128GB = 4x32GB
Single CPU tray:
128GB = 4x32GB
96GB = 3x32GB
32GB = 1x32GB



Also, I tried old macOS (or OS X), such as High Sierra and El Capitan, but no success.

However, both High Sierra and El Capitan boot with 160GB (4x32GB + 2x16GB) memory configuration,
but it looks just recognizing 16GM modules, ignoring 32GB modules.
Since macOS Mojave 10.14.5 does not boot with the same configuration,
OS behavior is different between macOS versions.

 

[kazkus]

I tested 192GB = 6x 32GB, booting Windows 10 Pro,
and here are Geekbench 4 results.
256GB https://browser.geekbench.com/v4/cpu/13537497
192GB https://browser.geekbench.com/v4/cpu/13545590

 

[handheldgames]

It successfully boots Windows 10 Pro and MemTest86 V8.2,
with some minor exceptions of memory configuration especially with single CPU tray,
but no luck for macOS, so it should be macOS issue.

macOS Mojave 10.14.5 does not boot.
It stops just after Apple logo displayed.
With Verbose mode (command+V), "End RandomSeed" is the last message.
It might be related with checking memory.
I have tried several memory configuration as follows.
256GB = 8x32GB
192GB = 6x32GB
160GB = 4x32GB + 2x16GB
160GB = 6x32GB
128GB = 4x32GB
Single CPU tray:
128GB = 4x32GB
96GB = 3x32GB
32GB = 1x32GB

View attachment 843069 View attachment 843070

Also, I tried old macOS (or OS X), such as High Sierra and El Capitan, but no success.

However, both High Sierra and El Capitan boot with 160GB (4x32GB + 2x16GB) memory configuration,
but it looks just recognizing 16GM modules, ignoring 32GB modules.
Since macOS Mojave 10.14.5 does not boot with the same configuration,
OS behavior is different between macOS versions.
View attachment 843077 View attachment 843079 View attachment 843080

thanks for sharing. too bad there is software compatibility issue with macos. if there was a hardware issue, windows would not be able to get around the problem.

Have you tried to reset the NVRam and the SMC?

 

[Soba]

However, both High Sierra and El Capitan boot with 160GB (4x32GB + 2x16GB) memory configuration,
but it looks just recognizing 16GM modules, ignoring 32GB modules.
Since macOS Mojave 10.14.5 does not boot with the same configuration,
OS behavior is different between macOS versions.

Thanks for all your testing. I'm very curious whether this will work reliably under Catalina on the cMP.

No other Mac (to my knowledge) has ever supported over 128GB. [Edit: I'm wrong. XServe and iMac Pro can, as pointed out by LightBulbFun below me.] Given the Fall release target of the 2019 Mac Pro, I presume it will require Catalina and won't work with Mojave, thus support for 128GB+ memory should be fully tested in the final Catalina release. Of course, there are other possible factors besides the OS, and it's not clear Catalina will run (even with hacks) on the cMP at release time, but we can hope.

If you have legitimate access to the Catalina beta and are able to test that, that might be really informative.

 

[LightBulbFun]

very interesting results

the Xserve3,1 supports up to 192GB of RAM (and iMac Pro up to 256GB officially and probably 512GB unofficially)

so OS X defo has support for more then 128GB of RAM

under High sierra where its "ignoring" the memory modules, what does hostinfo (a terminal command) say? also does the Diagnostics tab in system profiler say anything? (and what about AHT/ASD?)

its worth noting that Mojave with the 32GB memory sticks, is not even the booting the kernel properly, the "end Random seed" you see is the end of the EFI Boot-loader before the Kernel loads etc

 

[kwikdeth]

according to this page:
https://forums.macrumors.com/threads/mac-pro-cpu-compatibility-list.1954766/

the 4/5,1 mac pro has a maximum OSX support for 160GB. weirdly arbitrary number.

considering the iMac pro is the only machine with official support for anything larger than 128GB, I'm willing to bet its running either some kind of custom kernel, or, its got a different memory controller with support for larger density chips which is absent in other Macs.

 

[h9826790]

It successfully boots Windows 10 Pro and MemTest86 V8.2,
with some minor exceptions of memory configuration especially with single CPU tray,
but no luck for macOS, so it should be macOS issue.

macOS Mojave 10.14.5 does not boot.
It stops just after Apple logo displayed.
With Verbose mode (command+V), "End RandomSeed" is the last message.
It might be related with checking memory.
I have tried several memory configuration as follows.
256GB = 8x32GB
192GB = 6x32GB
160GB = 4x32GB + 2x16GB
160GB = 6x32GB
128GB = 4x32GB
Single CPU tray:
128GB = 4x32GB
96GB = 3x32GB
32GB = 1x32GB

View attachment 843069 View attachment 843070

Also, I tried old macOS (or OS X), such as High Sierra and El Capitan, but no success.

However, both High Sierra and El Capitan boot with 160GB (4x32GB + 2x16GB) memory configuration,
but it looks just recognizing 16GM modules, ignoring 32GB modules.
Since macOS Mojave 10.14.5 does not boot with the same configuration,
OS behavior is different between macOS versions.
View attachment 843077 View attachment 843079 View attachment 843080

Thanks for all the testing and sharing. If you have time, can you confirm that the single processor cMP can boot with 32GB DIMM in Windows?

If yes, how much it can take?

I still believe it’s the memory config in the NVRAM (NOT user deletable) causing the issue. Because only MacOS will read those info, Windows and Linux won’t.

 

[kazkus]

I found that @h9826790 had posted almost the same results on 2018-12-06.
https://forums.macrumors.com/thread...tibility-and-faq.2099103/page-4#post-26883780
Especially, 32GB memory modules cannot be used on macOS (OS X) 10.11 - 10.14.

Today, I have tested the following configurations with OS X Yosemite 10.10.5, macOS Catalina 10.15, and Windows 10.
Dual CPU Tray (X5680 x 2)
* 6x32GB - OS X Yosemite does not boot with panic just after Apple logo.
* 5x32GB - OS X Yosemite boots with 160GB memory. macOS Catalina does not boot, similar to Mojave.
Single CPU Tray (X5660)
* 4x32GB - Both OS X Yosemite and Windows 10 boot with 128GB 800MHz.
* 3x32GB - Both OS X Yosemite and Windows 10 boot with 96GB 1066MHz.
Single CPU Tray (W3690)
* 3x32GB - Windows 10 boots but only with 2GB.
* 4x16GB - Windows 10 boots but only with 2GB.

 

[agejon]

Interesting findings... so probably esxi will support the 32gb dimms!

 

[h9826790]

I found that h9826790 had posted almost the same results on 2018-12-06.
https://forums.macrumors.com/thread...tibility-and-faq.2099103/page-4#post-26883780
Especially, 32GB memory modules cannot be used on macOS (OS X) 10.11 - 10.14.

Today, I have tested the following configurations with OS X Yosemite 10.10.5, macOS Catalina 10.15, and Windows 10.
Dual CPU Tray (X5680 x 2)
* 6x32GB - OS X Yosemite does not boot with panic just after Apple logo.
* 5x32GB - OS X Yosemite boots with 160GB memory. macOS Catalina does not boot, similar to Mojave.
Single CPU Tray (X5660)
* 4x32GB - Both OS X Yosemite and Windows 10 boot with 128GB 800MHz.
* 3x32GB - Both OS X Yosemite and Windows 10 boot with 96GB 1066MHz.
Single CPU Tray (W3690)
* 3x32GB - Windows 10 boots but only with 2GB.
* 4x16GB - Windows 10 boots but only with 2GB.
View attachment 843143 View attachment 843144 View attachment 843145 View attachment 843146 View attachment 843147 View attachment 843148 View attachment 843149 View attachment 843150 View attachment 843151 View attachment 843152

Thank very very much for the tests. You are the first one to confirm single processor cMP can boot 128GB RAM in both OSX and Windows!

 

[startergo]

Today, I have tested the following configurations with OS X Yosemite 10.10.5, macOS Catalina 10.15, and Windows 10.

I don't see Catalina results

 

[h9826790]

I don't see Catalina results

 

[startergo]

It would be nice also to mention the bootrom version at least in the signature. All testing from now on should be performed on the 144 bootrom as it seems there will be no more updates to the cMP firmware.

 

[tsialex]

there will be no more updates to the cMP firmware.

I'm not so sure, Apple updated a lot of BootROMs with High Sierra Security Updates for Macs that didn't have Mojave support. Don't discard new BootROMs, we still have 10.14.6 betas and future security updates.

 

[startergo]

I'm not so sure, Apple updated a lot of BootROMs with High Sierra Security Updates for Macs that didn't have Mojave support. Don't discard new BootROMs, we still have 10.14.6 betas and future security updates.

I am keeping my fingers crossed. I will be very polite to Apple R&D guys. I know they are the best!

 

[kazkus]

Here is the summary of my tests of 32GB modules on Mac Pro 2009 (MacPro5,1 BootROM: 144.0.0.0.0) for two days:
* CPU tray, Dual or Single, does not matter.
* W36xx cannot support 64GB or larger. Windows 10 (and MemTest86 V8.2) can use only 2GB.
* Windows 10 (and MemTest86 V8.2) can boot and use 32GB memory modules.
* OS X Yosemite 10.10.5 boots and uses 32GB modules up to 160GB. If exceeding 160GB, it does not boot.
* macOS (OS X) 10.11 - 10.13 ignores 32GB modules. If it has 16GB modules (maybe smaller module as well), macOS boots, but simply ignores 32GB modules.
* macOS Mojave 10.14 and Catalina 10.15 cannot boot if 32GB modules exist.
** 2019-06-20 update : https://forums.macrumors.com/thread...nyone-tried-this.2046693/page-5#post-27478461
* 6 and 3 modules (192GB and 96GB) runs with 1066MHz.
* 8 and 4 modules (256GB and 128GB) runs with 800MHz.

Since I have not tested all combinations of OS and memory configuration, I might miss something.
I will do some additional tests today in order to confirm a couple of things.