MP 1,1-5,1 Cannot install or use Nvidia Webdrivers anymore!

[flyproductions]

I checked the console and looks like trustd is retrieving information about signature revocation associated with this pkg from somewhere locally.

I doubt that something stored locally can cause the problems. In this case disconnecting from the internet wouldn't make any difference.

 

[flyproductions]

I've just finished unblocking all of the connections I blocked, one at a time, and my computer is still working. So yeah... I don't know if any of my blocking actually made a difference, something else may be keeping my computer from breaking right now...

Well, now i did just liek you did and blocked litterally everything that tried to make contact to the net within the first minutes after connecting.

Anyway, no success! After two reboots with internet connection everything is messed up again!

So can you give us some details to make it possible to reproduce your scenario:

1. Wich version of the package are you using (WebDriver and CUD, i. e. 387.10.10.10.40.140 and 418.163)

2. On which security update level is your machine (i. e. 10.13.6 (17G14042), shows when you click on the version number in "About your Mac")

3. Did you reactivate SIP or gatekeeper after installing the drivers?

4. Did you set your machine to any different/earlier date?

Any of this information could be very helpful to sort out teh reason for this mess.

 

[startergo]

Maybe this is somehow relevant?

Stolen Nvidia certificates used to sign malware—here’s what to do - ThreatDown by Malwarebytes

Two Nvidia code signing certificates have been leaked by the LAPSUS$ ransomware group. We explain what it means and what you can do about it.

blog.malwarebytes.com

signature-base/yara/gen_nvidia_leaked_cert.yar at master · Neo23x0/signature-base

YARA signature and IOC database for my scanners and tools - Neo23x0/signature-base

github.com

 

[originaldotexe]

Maybe this is somehow relevant?

Stolen Nvidia certificates used to sign malware—here’s what to do - ThreatDown by Malwarebytes

Two Nvidia code signing certificates have been leaked by the LAPSUS$ ransomware group. We explain what it means and what you can do about it.

blog.malwarebytes.com

signature-base/yara/gen_nvidia_leaked_cert.yar at master · Neo23x0/signature-base

YARA signature and IOC database for my scanners and tools - Neo23x0/signature-base

github.com

kexts dont have to be signed on macos, you just have to disable SIP and allow them to load in system preferences.
well, drivers dont have to be signed on windows either but its more of a pain to load unsigned drivers because you have to go through this annoying process each time you boot the computer to disable driver signature verification (it cant be permanently disabled)

 

[startergo]

kexts dont have to be signed on macos, you just have to disable SIP and allow them to load in system preferences.
well, drivers dont have to be signed on windows either but its more of a pain to load unsigned drivers because you have to go through this annoying process each time you boot the computer to disable driver signature verification (it cant be permanently disabled)

But you have to remove the revoked certificate and resign to run locally

 

[flyproductions]

It's enough of an annoying mess too. But some "sort of" solution seems to be, to unplug the internet connection just while booting (and may be a few minutes after). Reconnectig after does not seem to cause any problems.

I repeated this a few times by now and everything is working fine so far.

 

[Dayo]

While this procedure still generally works, it has been superseded by an improved and easier scripted process available in POST 461

Aug 2022 Update: Use this option only if the information in Post 1 does not work for you

Right,

It seems the answer is to simply completely disable all certificate revocation checks.
Not typically something recommended but I suppose it makes little difference when running HiSierra.

First, you need to boot into Safe Mode. This will enable a basic GPU driver that will, while not accelerated, allow you to operate your Mac. To do this, turn on or restart your Mac, then immediately press and hold the Shift key until you see the login window and log in to your Mac. You might be asked to log in a second time.

You can verify you are in Safe Mode as follows

• Go to About This Mac >> System Report >> Software
• In the System Software Overview, look at the value listed next to the item labeled Boot Mode.
  • Safe: The Mac is using safe mode.
  • Normal: The Mac is not using safe mode.

Once in Safe Mode, carry out the following steps:

1. Download the web drivers in case you need to reinstall them
   • Only directly from the Nvidia website
   • If you do not have a usable backup from before 31 May 2022, download MacOS as well as you may need to reinstall
2. Fully disconnect your Mac from the web
3. Run sudo sh -c 'echo "0.0.0.0 ocsp.apple.com" >> /etc/hosts' && sudo sh -c 'echo "0.0.0.0 ocsp2.apple.com" >> /etc/hosts' && sudo killall -HUP mDNSResponder in Terminal
4. Run crlrefresh rp && sudo rm -f /var/db/crls/* && sudo sqlite3 ~/Library/Keychains/*/ocspcache.sqlite3 'DELETE FROM ocsp;' (AKA Purge Command) in Terminal to purge the current cached Certificate Revocation List
   • Ignore any error messages on running the above
5. Run sudo date -u 120200002021 && sudo reboot (AKA Date Command) in Terminal
6. Reconnect to the web and you should be good
   • If not good, disconnect from the web, rerun the purge and date commands above (in that order) and then reinstall the drivers before reconnecting
     • You may want to go into Safe Mode first
     • Disregard any cert stripping steps you may have come across
   • If not good, disconnect from the web, rerun the purge and date commands above (in that order) and then restore a backup from before 31 May 2022 before reconnecting
     • If you do not have a usable backup from before 31 May 2022, install MacOS to a separate disk, migrate your data across using Migrate Assistant, then execute Steps 3 to 5 before reconnecting instead.
   • If still not good, rerun the purge and date commands above (in that order) and stay offline
     • You may want to go into Safe Mode

EDIT:
To revert the changes (if/when a proper fix is available), you will need to...

1. Open Terminal, type sudo nano /private/etc/hosts and press "Enter"
   • I use Nano and forget what the default editor in Terminal is
   • If you don't have Nano, I think vim is most likely the default.
2. Delete the ocsp lines; then save and close
3. Run sudo killall -HUP mDNSResponder && sudo reboot in Terminal to refresh the DNS cache and reboot

Alternatively, just run the script in Post 461 and select the option to revert the workaround

 

[startergo]

It's enough of an annoying mess too. But some "sort of" solution seems to be, to unplug the internet connection just while booting (and may be a few minutes after). Reconnectig after does not seem to cause any problems.

I repeated this a few times by now and everything is working fine so far.

Right but any impersonator using revoked certificates can hijack your system this way

 

[startergo]

Probably a safer approach:

Resigning OS X / Mac OS application · Issue #23 · devinhalladay/spotio

On OS X / Mac OS we need a way to resign the Spotify.app, so we don't have to keep telling users to disable their GateKeeper (which is just generally a bad idea). This issue relies on the outcome o...

github.com

 

[DJenkins]

What the hell this is outrageous! Running 10.13.6 here with 3 x GTX1080Ti's which I really don't want to go to waste.

Thanks to everyone who's helped, going to try these steps tomorrow but I am a bit suss on the security vulnerabilities.

For anyone on hackintosh getting black screens on boot because the drivers won't load, punch in the boot flag:
nv_disable=1
You will have no GPU acceleration but at least you can get back in to your machine to try these steps. Chrome was bugging out but Safari works ok with minimal graphics tearing and errors.

Going to keep an eye on this thread to see if any other solutions come up...

 

[startergo]

UPDATE MARCH 8, 2022:​

The two NVIDIA code-signing certificates that were reported to be leaked in this cybersecurity incident are expired:

subject CN: NVIDIA Corporation
issuer CN: VeriSign Class 3 Code Signing 2010 CA
serial: 43bb437d609866286dd839e1d00309f5
valid from: ‎September ‎1, ‎2011
valid to: ‎September ‎1, ‎2014
And:

subject CN: NVIDIA Corporation
issuer CN: VeriSign Class 3 Code Signing 2010 CA
serial: 14781bc862e8dc503a559346f5dcc518
valid from: ‎July ‎27, ‎2015
valid to: ‎‎July ‎26, ‎2018
Criminal actors might include these expired certificates in malicious code that has been fraudulently signed, creating the misimpression that the code came from NVIDIA. We recommend that customers run NVIDIA software provided only from our trusted, legitimate sources. Also, NVIDIA recommends following the industry best practice of not trusting any certificates beyond their expiration date.

NVIDIA Support

nvidia.custhelp.com

 

[The Buggman]

It's enough of an annoying mess too. But some "sort of" solution seems to be, to unplug the internet connection just while booting (and may be a few minutes after). Reconnectig after does not seem to cause any problems.

I repeated this a few times by now and everything is working fine so far.

I booted up today with this exact thing in mind. I unplugged the ethernet while booting and let everything load up before plugging it back in. Everything is still intact and running smoothly.

 

[Stieber]

You do not have LittleSnitch

• Download the web drivers in case you need to reinstall them
• Fully disconnect your Mac from the web
• Run sudo sh -c 'echo "127.0.0.1 ocsp.apple.com" >> /etc/hosts' && sudo sh -c 'echo "127.0.0.1 ocsp2.apple.com" >> /etc/hosts' && sudo killall -HUP mDNSResponder in Terminal
• Run sudo date -u 020200002020 && sudo reboot in Terminal
• Reconnect to the internet and you should be good
  • If not good, disconnect from the web and rerun the date command above but reinstall the drivers before reconnecting
  • If still not good, then you need to use offline

Actually worked for me. But after installing the webdriver, the system is only booting with "nv_disable=1".
Otherwise it gives me this.

 

[The Buggman]

NVIDIA Support

nvidia.custhelp.com

I was personally aware of this breach from the day it happened, but at no point since have I installed anything from NVIDIA nor from anywhere else on this computer until yesterday, when we managed to reinstall the webdrivers (downloaded from NVIDIA).

 

[flyproductions]

Right but any impersonator using revoked certificates can hijack your system this way

Yes, this is valid for any way of bypassing security mechanisms, as it is in most cases not possible selectively. But taking a little care of what one installs might help. Especially for thigs that reqire authetification.

 

[Dayo]

the system is only booting with "nv_disable=1".

Are you on a Mac or on a Hack?

Regardless, the thing to do is to contact Nvidia and ask for a reissue of the drivers with updated certs.
You can then revert the changes (updated the post with instructions) and be okay for another 5 or so years.

 

[grad]

I can confirm that setting back the date worked (at least for now and I haven't even dared to connect to the Internet yet). I tried it on a OpenCore-booting High Sierra hackintosh system. Had to insert "nv-disable=1" first in the boot-args in order to avoid the black screen issue and the "IOConsoleUsers: gIOScreenLockState 3" loop. It always helps if you have OpenCore EFI on a USB stick, this way you can edit config.plist on a different working computer. I then disconnected from Internet and booted in to 10.13.6 recovery partition (luckily had one) and set back the date (maybe BIOS could have been enough), also blocking OCSP at /etc/hosts, as @Dayo suggested here. I rebooted to 10.13.6 and reinstalled the 387.10.10.10.40.140 Nvidia WebDrivers I had kept for safekeeping on top of what was already installed in the past. Removed the "nv-disable=1" argument in OpenCore. After a reboot I can log in to GUI and have GPU acceleration as before.

Well, it was too good to be true. It doesn't work now. All I can do now is wait for Nvidia to release new drivers.

 

[flyproductions]

Right,

It seems the answer is to simply completely disable all certificate revocation checks. Not typically something recommended but I suppose it makes little difference when running HiSierra.

Two ways

1. You have LittleSnitch
   • Download the web drivers in case you need to reinstall them
   • Look for trustd in LittleSnitch and expand until you find ocsp and/or ocsp2
   • Block this/those process(es)
     • You may need to do this separately for user and for system
   • Fully disconnect your Mac from the web
   • Run sudo date -u 020200002020 && sudo reboot in Terminal
   • Reconnect to the internet and you should be good
     • If not good, disconnect from the web and rerun the date command above but reinstall the drivers before reconnecting
     • If still not good, try the option below instead

This sadly didn't work for me.

I followd all the steps, except i was unable to find ocsp or ocsp2 but but only ocspd. So i disabled any outgoing or incoming connection for that. For both, user and system.

The disabled "Allow any outgoing connection" could not be deleted as it's a system rule which cannot be modified.

Regardless of that, after two reboots with internet connection everything was dumped again!

Is there any setting to be made, making LS present durin bootup or at least load before the drivers? Otherwise i will have to try the „No Little Snitch“-Method.

?

 

[Dayo]

unable to find ocsp or ocsp2, only ocspd

ocspd(1) [osx man page]

ocspd performs caching and network fetching of Certificate Revocation Lists (CRLs) and Online Certificate Status Protocol (OCSP) responses. It is used by Security.framework during certificate verification. Security.framework commu

www.unix.com

The disabled "Allow any outgoing connection" could not be deleted as it's a system rule which cannot be modified.

Means it is still working, fetching and caching the revocation stuff.
BTW, as the revocation status is apparently cached, it means that if you disconnect after a failure, it will use the locally cached result in the interim. Fits in with what someone posted earlier about something being stored locally.

will have to try the "No Little Snitch" Method.

You may have to amend that to the ocspd domain (or add this) if you can find that out from LittleSnitch.

Send a message to Nividia ... the more they get, the more serious they take it.

 

[flyproductions]

Means it is still working, fetching and caching the revocation stuff.

Even with the Rule „Deny any outgoing connection“ for system as well as user just above and itself disabled?

This is what LS comes up with when trying to delete:

Sounds to me as if the protected rule is no longer valid if disabled.

BTW, as the revocation status is apparently cached, it means that if you disconnect after a failure, it will use the locally cached result in the interim. Fits in with what someone posted earlier about something being stored locally.

I have just deleted any cache on my machine that i could find. User, System and Library. Will see if this helps.

You may have to amend that to the ocspd domain (or add this) if you can find that out from LittleSnitch.

Sadly could not find any domain it tries to connect to.

Send a message to Nividia ... the more they get, the more serious they take it.

Yes, this might be a good idea.

So now i try the terminal method.

 

[drifeppelin]

I've just finished unblocking all of the connections I blocked, one at a time, and my computer is still working. So yeah... I don't know if any of my blocking actually made a difference, something else may be keeping my computer from breaking right now...

I suppose I'll see if things are still working tomorrow.

So just a little update: After unblocking everything last night and seeing my system was still working after several reboots, I booted up my computer today, and it started happening again. So it seems like whatever I did (whether it was the blocking or something else) took some time to 'wear off'...

So can you give us some details to make it possible to reproduce your scenario:

1. Wich version of the package are you using (WebDriver and CUD, i. e. 387.10.10.10.40.140 and 418.163)

2. On which security update level is your machine (i. e. 10.13.6 (17G14042), shows when you click on the version number in "About your Mac")

3. Did you reactivate SIP or gatekeeper after installing the drivers?

4. Did you set your machine to any different/earlier date?

Any of this information could be very helpful to sort out teh reason for this mess.

1 - The driver version I'm using is 387.10.10.10.40.140 and CUDA is 418.163.

2 - System version is macOS 10.13.6 (17G14042).

3 & 4 - The state of my computer at the time it broke was as follows: SIP enabled, gatekeeper enabled, web driver already installed. I did not have the CUDA driver installed.
When it broke, I followed Buggman's steps like this: I went into recovery mode, unplugged my ethernet cable, opened terminal, disabled SIP, disabled gatekeeper, set the date back to 2021, and reboot. As it rebooted, I held shift to enter safe mode. In safe mode I reinstalled the web driver (same version as always - 387.10.10.10.40.140), and then my computer rebooted normally, with everything working once again.

Now that I was back into my system, I opened Radio Silence's network monitor, and then plugged my ethernet back in to check what connections were being made. I blocked all of them, and restarted. It rebooted successfully, so I then tried unblocking each of connections, one at a time, rebooting in between each to see if the system would break like before - it never did, even with everything unblocked.

The next thing I did was just go about my usual computer activities for a while, then I went to bed, leaving everything unblocked still. Now I boot up my computer this morning, and it's broken once again like before.

So now what I've done is go through Buggman's steps again to bring my system back to a working state, but with a few differences this time:
- I set the date to 2020 rather than 2021.
- In addition to reinstalling the web driver, I installed the CUDA driver (418.163) for the first time.
- Once I was back into a fully functioning offline system, before plugging my ethernet back in, I blocked trustd (/usr/libexec/trustd) and ocsp (/usr/sbin/ocspd) in Radio Silence. (I could not find ocsp2 in my system files).
- In addition to blocking trustd and ocsp in Radio Silence, I followed Dayo's steps for the non-little snitch method exactly as they instructed.
- After thoroughly blocking trustd and ocsp I plugged my ethernet back in, and that's where I'm at presently.

So now I'm going to just wait and see what happens... I've restarted a couple times already and so far things are working, but like before it's hard to say if it will remain that way.


EDIT: Just to further clarify what happens to my system when it "breaks"... For me I get stuck in a boot loop. I cannot get into macOS without going through recovery mode. The boot loop in verbose mode says the exact same sort of thing as in Stieber's post further up this page ("AppleKeyStore : operation failed", etc.). I expect adding "nv_disable=1" to my boot-args might get me back in as it did with Stieber, albeit without GPU acceleration.

 

[Demigod Mac]

Update:
I contacted Nvidia support (live chat) about this and they are aware of the issue.
The support rep provided me a screenshot of the macOS web driver installer refusing to launch and I confirmed it was the problem, and linked them to this thread. The rep reported it to Tier 2 support who may follow up with me.

 

[garibaldo]

unable to reinstall webdrivers after executing @Dayo 2nd procedure (without LS), installation show a message of expired drivers , and keeps failling... cMP 3, 1, High Sierra, without network connections...

 

[flyproductions]

Thanks for taking the time to completely answering all the questions. So i have one more:

- I set the date to 2020 rather than 2021.

Do you keep the date set to an earlier state or let automatic time and date step in after the (successful) installation.

EDIT: Just to further clarify what happens to my system when it "breaks"... For me I get stuck in a boot loop. I cannot get into macOS without going through recovery mode. The boot loop in verbose mode says the exact same sort of thing as in Stieber's post further up this page ("AppleKeyStore : operation failed", etc.).

So for this i'm in a better situation. I keep SIP disabled and if things fail, a simple reboot with holding shift takes me to a safeboot state from where i can reinstall. No bootloop, black screen or anything...

 

[drifeppelin]

Do you keep the date set to an earlier state or let automatic time and date step in after the (successful) installation.

Every time I've connected back to the internet, I've let the automatic time and date step in.

Also, another thing to add... I've just noticed I'm able to run the web driver install package now. Even though the date is set to the present, the internet is on, no safe mode, etc... I'm not getting a prompt saying the certificate has expired. The package installs, and after rebooting, the package still works when I go to run it again. I haven't changed anything in my system since my last post.

EDIT - screenshot: