MP 1,1-5,1 cMP 5,1 - AHT won't run- bad backplane?

[Valdaquendë]

Greetings, Friends,

I have been rebuilding a mid-2010 cMP. As part of that rebuild, replaced the SPI with a fresh one programmed with the BootROM file tsialex reconstructed for me (thanks!). I also installed a WiFi upgrade card (BCM94360CD), a Sonnet Allegro Pro USB card and a pair of W5690 (3.46GHz) CPUs. I got the system together this morning and I am wondering whether I've overlooked something or whether I've got a bad backplane.

I got the backplane installed last night and booted up and performed a deep NVRAM reset. I was still running the the old CPUs (W5650s) and wanted to test and compare performance before and after the CPU upgrade. I ran Geekbench 5 and got scores of 558 Single-core and 5274 Multi-core.

Then I installed the W5690s and booted up. When I ran Geekbench 5 the first time I got scores of 669 Single-core and 6069 Multi-core. So far so good.

I noted that the desktop interface seemed slow when drawing selection marquees and selecting desktop items, so I rebooted again, again doing a 5-chime NVRAM reset and I also booted into safe-mode, logged in and then rebooted into normal mode. When I ran Geekbench 5 again, I got scores of 306 Single-core and 3505 Multi-core. These scores were less than half of the first W5690 scores and are actually less than the scores of the W5450s.

Then I tried booting into the Apple Hardware Test thumb drive that I created for 5,1s (and have used on several other 5,1 Mac Pros). It would not complete the preliminary hardware probe, even after 20 minutes.

I rebooted again, ran Geekbench and got scores of 669 Single-core and 6069 Multi-core. I have rebooted and run Geekbench at least five more times and have gotten similar results each time (single-core scores in the mid-600s and multi-core scores in the low-6000s).

But the AHT still cannot complete the hardware probe, so I cannot run the AHT tests. I tried rebuilding the AHT thumb drive and attempting to run it from the rebuilt thumb drive; the result is the same. The hardware probe comes close to completing but will not complete.

I keep a 3.33GHz single-CPU processor tray in the shop for testing purposes. I removed the Sonnet USB card and the dual-CPU tray from the system and installed the 3.33GHz tray. I got the same result; the AHT will not complete the hardware probe.

Any suggestions or advice will be gratefully received and carefully considered. If I've merely overlooked some detail, I'd be relieved. The only other alternative I can currently think of would be to strip the system back down to nothing replace the backplane, re-assemble the system and test it.

Thanks in advance for the time and energy you spent reading (and, perhaps, responding to) this.

Valdaquendë

 

[tsialex]

W3[5,6]xx are the single CPU Xeons, X5[5,6]xx are the dual CPU Xeons, seems you are mixing the two all over.

Did you get the correct AHT release for MP5,1?

 

[Valdaquendë]

They are X5690s, just as the old ones were X5650s; I don't know what I was thinking.

I downloaded the AHT (filename: 022-4831-A and listed as MacPro5,1 - Mac-F221BEC8) from upekkha's Apple Hardware Test page on GitHub. The test's version is 3A207.

 

[tsialex]

Did you tried ASD3S149?

Apple Service Diagnostics, Apple Hardware Test, and Mac Test Pro 2001 - 2010 - loremipsum.info

January 2001 – Apple Hardware Test for Power Mac G4 Cube version 1.1 – Download G4 Cube, will not work with any other computer. January 2002 – Apple Service Diagnostic 062602 1.0 XServe – Download XServe July 2002 – Apple Hardware Test for iMac version 1.2.2 – Download Apple iMac, Apple eMac...

www.loremipsum.info

 

[Valdaquendë]

EXCELLENT!! I've accumulated a lot of tools, over the years, and though I've found and used AHTs, I had not run across the ASDs. Thanks for this excellent source! I'm currently downloading ASD3S149 and reading this article to learn more about it. Any tips or advice about it?

 

[tsialex]

EXCELLENT!! I've accumulated a lot of tools, over the years, and though I've found and used AHTs, I had not run across the ASDs. Thanks for this excellent source! I'm currently downloading ASD3S149 and reading this article to learn more about it. Any tips or advice about it?

The EFI version gets hardware problems faster, but for nasty intermittent memory errors that happen eventually with temperature or memory pressure, the macOS one is better. You need to install your AppleOEM GPU to run ASD.

 

[Valdaquendë]

Thanks, tsialex! I'll run both tests; I've not run either of them before and I want the experience.

I actually have two largely-rebuilt cMPs at this point but neither one has a graphics card of its own. I am switching the Radeon HD 5870 between them as needed. GPU prices are so "out there" at this point that I am holding off on buying and I am basically "fishing" for the occasional bargain ... without success so far. The idea of paying more than MSRP for a used card (that can't be returned if it goes wrong) rankles me. It's just not right; like everyone else, I'm hoping for some sea-change that will bring prices down to something approaching reality.

But per your very excellent advice back when we first met, I now work on them only with solid test components like the known-good single-CPU tray and the HD 5870. I'll report back once I've run them.

 

[tsialex]

Thanks, tsialex! I'll run both tests; I've not run either of them before and I want the experience.

I actually have two largely-rebuilt cMPs at this point but neither one has a graphics card of its own. I am switching the Radeon HD 5870 between them as needed. GPU prices are so "out there" at this point that I am holding off on buying and I am basically "fishing" for the occasional bargain ... without success so far. The idea of paying more than MSRP for a used card (that can't be returned if it goes wrong) rankles me. It's just not right; like everyone else, I'm hoping for some sea-change that will bring prices down to something approaching reality.

But per your very excellent advice back when we first met, I now work on them only with solid test components like the known-good single-CPU tray and the HD 5870. I'll report back once I've run them.

Buy the HP GT630 that I've wrote you about, it's a solid low-end card, with a public Mac EFI and with METAL support. I bet that you can find it easily from recyclers, it's a very common card from HP corporate desktops.

 

[Valdaquendë]

OK; I created the ASD3S149 boot USB drive in two partitions just as instructed and restored each part (OS and EFI) to its appropriate partition. Then I booted into the EFI partition ... 40 minutes later, it is still "searching for hardware to run tests". It is much the same as the behavior of the AHT; queries hardware and never gets past it.

This is probably irrelevant but since the BootROM contains information about hardware, I double-checked it. When I programmed the SPI, I checked to make sure that Boot Block BIOS version was right and so on. I just checked it again and it looks fine.

I am going to try to run the ASD OS tool now and will report back. (NOTE: It is running now and reported no errors in startup/SMBIOS, from which I surmise that the BootROM is, indeed, OK).

 

[Valdaquendë]

I ran the OS version and found that all tests passed except for the AirPort tests. I assume that this failed because I had upgraded the AirPort card to the BCM94360CD and so the stock AirPort card was not found.

So I tried disassembling the system, disconnecting the BCM94360CD, reconnecting the original AirPort card and reran the OS test. This time all tests passed; runtime was about 17 minutes.

I then tried running the ASD EFI test again. Again, it hung on the hardware detection phase. I also re-ran AHT and got the same result; the process hangs very close to the end of the hardware probe.

I am, by the way, allowing approximately 15 minutes, each time, before aborting the hardware detection phase. I can't believe that it could ever take longer than a minute or two under normal circumstances but wanted to give the routine every possible chance to succeed.

I still hope for some solution other than a "bad backplane" conclusion but I must say that I feel some relief in having a couple of spare backplanes on hand, just in case.

 

[Valdaquendë]

Buy the HP GT630 that I've wrote you about, it's a solid low-end card, with a public Mac EFI and with METAL support. I bet that you can find it easily from recyclers, it's a very common card from HP corporate desktops.

I will definitely check these cards out. Thanks for the tip.

 

[Valdaquendë]

FYI, tsialex - I looked for and bought 2 of those cards; one for $40 and one for $50. They'll give me graphic functionality on both of the cMPs I'm currently working on and they'll free up the Radeon HD5870 for other diagnostic uses.

I'm still offering sacrifices to the gods of graphic goodness in the hopes that Ethereum will curtail crypto-currency mining or, better yet, that AMD will alter their GPU firmware to limit cryptocurrency mining on certain product lines. There is bitter irony in the fact that nVidia has begun releasing cards with firmware that reduces the mining efficency of their GPUs while AMD (basically, Apple's sole supported GPU mfr) has not. Admittedly, nerfing the mining capabilities of GPUs will almost certainly result in another running battle, with miners trying circumvent any measures taken by mfrs but at least it's an attempt to extricate users from a world in which no ordinary user can buy even a well-used GPU for less than the MSRP of a new card.

Thanks, again, for the tip.

 

[Valdaquendë]

I was prepared to tear this system down to the backplane and swap out the backplane for another one and then test it to see if the ASD EFI would work with that (figuring that in this way I could determine whether the backplane was at fault.)

Then I had an idea. As I mentioned above, I have another cMP that I completely restored before starting on this one, going over the backplane, completely going through the CPU tray, refurbishing the Northbridge cooler, installing a reconstructed BootROM file (thanks to tsialex), resetting and re-greasing the CPUs, etc. It is stock; that is to say I have not installed any upgrades or modifications (yet) except for the SSD that is its boot drive. So I thought that, rather than tear cMP #2 down to nothing, replace the backplane, put it all together and retest, I would set up cMP #1 and run the ASD on it to see how it behaved.

It is doing the exact same thing. Further, the AHT also behaves in the same way as with cMP #2.

I have carefully looked up and re-downloaded the ASD and AHT files, rebuilt the AHT and ASD volumes over again from scratch and have carefully followed the appropriate directions. But to no avail. In short, neither machine will run either the AHT for this model or the EFI ASD. In both cases, the hardware probe never seems to complete.

Both systems, however, run the ASD OS tests without problem and both systems pass all tests (as long as the original AirPort card is installed).

I am going to tear down cMP#2 replace its backplane with a spare and see if this behavior repeats itself. I'll post my findings afterward. In the meantime, if anyone can think if a reason that these tests won't run or suggest a way of resolving this problem, I would be very grateful.

I admit that this is probably somewhat of a fine point, as both systems seem to run quite well. But if there are problems or limitations I wish to know about them and why they exist and, if possible, I want to try to correct them.

Thanks for your time and attention; it is much appreciated.

 

[Valdaquendë]

This morning, I shut down my office cMP (Mid-2010 5,1 - 3.33GHz single-CPU), swapped its Radeon Rx580 Pulse for the OEM Radeon HD 5870 card that I use for diagnostic work, and attempted to run the EFI ASD. The result was the same as with the other two cMPs; the diagnostic never gets past the hardware probe.

All three of these systems are, at this point, "stock"; that is there are no hardware or OS modifications (other than the SSDs that each uses as a boot volume). All three are Mid-2010 cMPs, each has at least 16GB RAM. Two are dual-CPU systems (one 2.44GHz, one 3.46GHz); these two have had their BootROM files reconstructed and flashed back on to their SPIs. The 3.33GHz single-CPU is (AFAIK) running on its original EFI.

It's hard to imagine that all three systems have some problem that prevents the ASD from running; particularly one that affects all three systems in the same way. There are two possibilities that I can see, currently, that might cause this to happen; either there is something wrong in the creation of the ASD media or there is something in hardware or firmware that is preventing the probe from completing.

To create the ASD media, I have:
• Downloaded the appropriate ASD file: ASD 3S149 according to this page (and other sources) and unzipped it
• Double-clicked the first segment (ASD_ssps_004-0499-A.001.dmg) to mount it
• Mounted the two enclosed disk images ("ASD EFI 3S149.dmg" and "ASD OS 3S149.dmg")
• Erased a 64GB USB drive, using GUID as its partitioning scheme and MacOS Extended (Journaled) as its format
• Partitioned the drive, creating two partitions, one ≈30GB (EFI) in size and one ≈30GB (OS)
• Clicked on the USB drive's 'EFI' partition in Disk Utility
• Clicked 'Restore'
• Dragged the mounted disk image (ASD EFI 3S149) to the 'Restore from' field
• Clicked 'Restore' and allowed the restore process to complete
• Clicked on the USB drive's 'OS' partition in Disk Utility
• Clicked 'Restore'
• Dragged the mounted disk image (ASD OS 3S149) to the 'Restore from' field
• Clicked 'Restore' and allowed the restore process to complete
• Ejected the USB drive and closed Disk Utility

To run the ASD, I have:
• Disconnected everything except the mouse, keyboard and display
• Inserted the USB drive in the subject system
• Booted up, choosing 'ASD EFI 3S149' as the boot volume

If you can see anything incorrect in these steps please don't hesitate to correct me. I've gone back and repeated them several times, now, to ensure that I am not committing some mistake along the way. But the results have been uniformly disappointing.

All the processes above complete without error; nonetheless, the EFI ASD will not run past the hardware probe. And, in fact, neither will the AHT (which I installed on another USB drive). The OS ASD, however, runs properly and all three systems pass without error. I'm mystified as to why this is happening.

Could this be caused by some obscure problem with the GPU (a used Radeon HD 5870 which seems to work fine) or because I am using an old Dell display? Or could there be some other reason?

Thanks for reading and considering this problem. Part of me says, "forget about it and move on", since the OS ASD passes and the systems seem to work well. But I find it troubling that a diagnostic tool Apple made specifically for this system will not run properly and I should, at least, like to find out why.

 

[tsialex]

I'll have sometime Sunday and I'll check if the same happens with my Mac Pros.

 

[Valdaquendë]

Thanks, tsialex; it's a most perplexing problem. I really appreciate it. I'm going to try doing this on another USB drive (a brand-new 128GB) on the off-chance that the problem is with the drive I used. (The AHT didn't complete either, though that was installed on a different USB drive - just grasping at straws, here).

 

[Valdaquendë]

I have now done many hours of research trying to figure/find out why the EFI test won't run. The only real clue I have run across was a posting by BDAqua on this Apple Communities Thread, in which he stated that:

The never released to the public ASD 3s149 should run GUI HW test but not the EFI test.

I have found and messaged him to see if we can confirm the test's non-functionality and, if possible, find out why it occurs and whether the explanation applies to the AHT's failure to complete a hardware probe. I'll report any answers/explanations here.

 

[tsialex]

I have now done many hours of research trying to figure/find out why the EFI test won't run. The only real clue I have run across was a posting by BDAqua on this Apple Communities Thread, in which he stated that:


I have found and messaged him to see if we can confirm the test's non-functionality and, if possible, find out why it occurs and whether the explanation applies to the AHT's failure to complete a hardware probe. I'll report any answers/explanations here.

I’ve run the EFI test several times in the past successfully, but I don’t remember running with the 1[3,4].0.0.0.0 EFI releases. AHT I’ve run recently and successfully.

 

[Valdaquendë]

OK; good to know.

I've been wondering whether the EFI version has something to do with it. Clearly, it is the EFI version of the ASD that won't run and it occurs to me that the AHT probably uses or interacts with the EFI as well, which would account for its small size and quick runtime.

Then, in response to my query, this morning, I got the following reply from BDAqua on his forum (BDAqua's XYMpire):

Only additional info is from long gone German site, so maybe EFI version is why?...

ASD 3s149 EFI, 10.6.4 USB, NBI

Plus the 2 Twelve core cMP 2010s I had ran the GUI HW test but not the EFI test.

That doesn't give us much more to work with but it occurs to me that, if this is a compatibility problem between the current firmware version and the ASD EFI test (and, perhaps AHT), we might ferret it out by reflashing the firmware back to an original (or close to original) version and seeing if the ASD would then run.

The original firmware for a Mid-2010 Mac Pro (EFI v. 1.5) is available at this page on Apple's website. I am willing to try this in the cause of figuring out what's going on here; it could be helpful to others as well as satisfying my own insatiable curiosity. But I will totally understand if you think this is a fool's errand.

Running the firmware updater results in a "cannot be run with this version of software" or some such message, as I expected it would.

In that case, since the SPI can't be reprogrammed while it is on the backplane, I would need to unsolder it, reprogram it using the firmware (.fd) file in the firmware updater package and resolder it on the board. Does this strike you as a workable path forward? (Actually, I would just program a new SPI and save the 'original' for replacement afterward.)

Also, I noted that the firmware file, in the Mid-2010 Mac Pro (EFI v. 1.5) updater is named:

MP51_007F_03B_LOCKED.fd

I'm not sure what 'LOCKED' means, in this context. I wouldn't want to go through all of this just to find that I cannot use the file to flash the SPI.

What do you think?

 

[tsialex]

MP51_007F_03B_LOCKED.fd is a standard binary image, exactly the same as the 144.0.0.0.0 MP51.fd is.

You can reconstruct it following my instructions without any issues, as any other MacPro5,1 generic binary image.

 

[Valdaquendë]

Thanks, tsialex; I think I'm just nutty enough to go ahead and do it. I'll keep you posted.

 

[tsialex]

Thanks, tsialex; I think I'm just nutty enough to go ahead and do it. I'll keep you posted.

Don't forget that anything before MP51.0083.B00 will not support APFS disks, so boot Sierra (or High Sierra with JHFS+).

Prepare your test disk beforehand, also keep your 144.0.0.0.0 ready to flash and ROMTool on your test disk.

 

[Valdaquendë]

Done. The behavior was the same; the original EFI made no difference. I went ahead and switched back to v.144.0.0.0.0 but it was worth the effort to check it out.

BTW, I actually replaced the EFI chip both times; I wanted the practice soldering. When I first started soldering these chips, I used the chisel point to heat all 4 pins on each side at once and that is one good technique for soldering and desoldering a DIP but I wanted to practice the method you described, of soldering pin 1, positioning the chip, soldering pin 5 and then soldering the remaining pins.

When I was done, I cleaned the pins on the fresh chip I had programmed with the original EFI, erased it and put it back in my stock of new SPIs.

 

[GoJohnGo]

For what it's worth, I'm encountering a similar issue with both of my 5,1 cMPs. The only difference is that they do run AHT, but ASD EFI 3S149 always gets stuck.

The one I've set up for testing:
ROM 140.0.0.0.0
CPU 2x3.46
RAM 128GB
Disks: none
Graphics: AMD 4870 (stock Mac card)
The only items plugged in are the monitor, keyboard, and USB drive holding the ASD.

Is the image bad?
Is there another version that would work? I read that 149 was an update of 143, but haven't run across an image for 143.

I just bought some used RAM and was hoping to be able to run this to test it, as using TechTool generated a "stuck address" error and I'd like to gather some more info.

 

[h9826790]

For what it's worth, I'm encountering a similar issue with both of my 5,1 cMPs. The only difference is that they do run AHT, but ASD EFI 3S149 always gets stuck.

The one I've set up for testing:
ROM 140.0.0.0.0
CPU 2x3.46
RAM 128GB
Disks: none
Graphics: AMD 4870 (stock Mac card)
The only items plugged in are the monitor, keyboard, and USB drive holding the ASD.

Is the image bad?
Is there another version that would work? I read that 149 was an update of 143, but haven't run across an image for 143.

I just bought some used RAM and was hoping to be able to run this to test it, as using TechTool generated a "stuck address" error and I'd like to gather some more info.


View attachment 2010275

It looks like your cMP is a 2009 flashed 5,1.

ASD and AHT is very very hardware specific. Even though 3S149 is for 5,1, but you still need to have the whole set of native 5,1 hardware to make it able to diagnose properly.

e.g. No hard drive may be the issue. Or 3S149 doesn't expect HD4870 (this is a 4,1 graphic card, not 5,1 graphic card).

Anyway, for your info, I can run ASD 3S149 on my 2009 flashed 5,1

https://forums.macrumors.com/threads/opencore-on-the-mac-pro.2207814/page-127?post=28690275#post-28690275

But this is with a Radeon VII that can only display via UEFI GOP in ASD, and by using OpenCore.