Catalina - *all* web-browsers fail to open new pages after a few hours use.

[johngwheeler]

I am experiencing the weirdest problem that I've seen in using Macs for nearly 20 years.



I have a late-2019 MacBook Pro 16 with the latest version of macOS Catalina (10.15.4).



The problem is this, which started about 3 weeks ago.



1) After a reboot, everything works as normal, but after "a few hours to a couple a days" (i.e. 4 - 48 hours), all of my web-browsers (Safari, Chrome & Firefox) fail to connect to "new" web pages, but pages already open continue to work normally including links to other pages with the same web site.



2) The problem seems limited to the actual web-browsers themselves, and not other internet connected clients. So Mail still works, Slack and other chat applications work, Spotify works OK. Other devices on the same network are all OK.



3) Command line network utilities still work and connect to websites, e.g.

curl, telnet, ping, wget, netstat, nslookup - all work perfectly with DNS names or IP addresses.



4) I have disabled firewalls, anti-virus web protection, and checked by any router controls - everything works as normal



5) I have tried different networks and ISPs, i.e. changing to different wi-fi networks and connecting to the internet via mobile data rather than fixed line. No difference.



6) I've reset PRAM



The browser failure is curious because it is only partial. Let me explain...



If I have a web page open, for example cnn.com, it continues to work, and links within the page to the same domain (i.e. cnn.com) continue to work as usual.



However, if I try to open a new tab and go to *any* web site, using DNS name or IP address, then it just hangs.



If I try to navigate to another page from a tab that is still working, e.g. go from cnn.com to bbc.com, then it fails. So it looks like "new" web domains are rejected, but ones that are working on an open tab continue to work.



I thought this might be a DNS issue, so flushed the DNS cache with:

sudo dscacheutil -flushcache; sudo killall -HUP mDNSResponder



No change.



7) The browser problem eventually affects system stability and causing multiple applications to hang. Generally I have to do a hard reboot (holding down the power button).



8) I have run two different virus checkers doing a deep scan.



9) I have reinstalled MacOS, but kept my data, so the only next step would be a complete new install which is very time-consuming given the number of applications I need to re-install and configure - and with no guarantee that this problem won't happen again.



The only applications that I recall installing before seeing these problems were Skype For Business (now un-installed) and GoToMeeting.



The Fact that command line and non-browser internet tools still working normally, but *all* browsers have the same problem is really weird. Are Safari, Chrome and Firefox using a common MacOS component (WebKit?). And how would this affect other non-browser client over a matter of hours or days?



I'd really welcome any suggestions of what to try...



Thank you!



John.

 

[casperes1996]

This sounds a bit to me like the kind of behavior you’d get if you updated a dylib (dynamically linked library) that all the browsers link to while the browsers were still running.

Any file descriptors still pointing to the old version will continue working, but trying to open a file descriptor to the new version will fail until a system reboot

If you run softwareupgrade that normally requires a reboot without doing a reboot, you’ll see something similar, if it updates a library used by Safari or another browser. The behaviour you’d usually see there, is that trying to open something new will initiate an infinite loop with Safari saying it encountered an issue in a small bar at the top that keeps looping until you close the site, pinning a full CPU thread.

If that’s not what you’re seeing just disregard this

I also have no clue why you might be seeing this issue, but it sounds very similar. - What I describe only happens if I run topgrade or softwareupgrade And ignore the message to reboot immediately though.

 

[Kier-XF]

I've had this with a brand new MBP 16. A complete OS reinstall did not fix the problem - after a time, the browsers have exactly the same issue, even when attempting to connect to an Apache installation on the MBP itself, either via http://localhost or http://{hostname.local}.

Rebooting fixes the issue, albeit temporarily - which is not really a fix at all.

 

[hmkabaker]

I believe Firefox is the culprit. Or Catalina is letting Firefox use memory in a way that destabilizes the system. Try running all browsers except Firefox. I've been running Chrome exclusively, with lots of tabs open, and allowing the system to remain idle for hours while I'm away from the computer. No problems. Previously, this was guaranteed to hang page connections for Firefox (and Chrome, if both were running). iMac, late 2015, 8GB.

 

[Kier-XF]

I’m afraid Firefox can’t be the culprit, as I’ve had this problem on a new MBP 16 with a fresh copy of Catalina and only Safari installed.

 

[TiggrToo]

I’m afraid Firefox can’t be the culprit, as I’ve had this problem on a new MBP 16 with a fresh copy of Catalina and only Safari installed.

When this happens can you do pings and traceroutes on the command line to the requisite hosts?

 

[Kier-XF]

Next time it happens I’ll do more testing and report back - to be honest thus far I’ve just got annoyed with it and rebooted rather than doing extensive investigation. I’ll do a full suite of diagnostics next time it hits me.

 

[throAU]

try reboot your router instead? could be some state problem in your wifi access point or router.

 

[hmkabaker]

Kier-XF,
Thanks. So it should hit me again sometime, running Chrome, I guess. Another problem was waking up from screen-saver sleep, which I decided probably was not a cause. Also, Time Machine getting blocked when trying to start a backup while Firefox seemed to be hogging memory; not a problem with Chrome so far.

 

[johngwheeler]

This sounds a bit to me like the kind of behavior you’d get if you updated a dylib (dynamically linked library) that all the browsers link to while the browsers were still running.

Any file descriptors still pointing to the old version will continue working, but trying to open a file descriptor to the new version will fail until a system reboot

If you run softwareupgrade that normally requires a reboot without doing a reboot, you’ll see something similar, if it updates a library used by Safari or another browser. The behaviour you’d usually see there, is that trying to open something new will initiate an infinite loop with Safari saying it encountered an issue in a small bar at the top that keeps looping until you close the site, pinning a full CPU thread.

If that’s not what you’re seeing just disregard this

I also have no clue why you might be seeing this issue, but it sounds very similar. - What I describe only happens if I run topgrade or softwareupgrade And ignore the message to reboot immediately though.

Thank you; this seems like a solid response and quite possible. The fact that only my web-browsers are initially affected does lead me to think that they are using a shared library.

What is odd is that happens after a significant delay after having rebooted the machine - usually several hours. It's almost as if the library (or some other component in the browser) goes "off" - very unscientific language I know!

Are these libraries unloaded / modified during normal usage without explicitly upgraded anything?

I do notice the problem after waking from sleep, but can't say this is the deciding factor - it may have happened during the same period of usage. For example, I've been using Slack or GoToMeeting, then try to refer to a web-page, only to find it doesn't respond.

I suspect that the issue started after I installed some remote conferencing apps for work: Skype, Skype For Business, and GoToMeeting.

I updated to Catalina 10.15.4 after this in the hope it would fix the issue, but no luck.

I installed LittleSnitch to try to diagnose, and can see that the browser doesn't even get as far as making the outbound HTTP(s) request and so is not captured.

Thanks for your input!

John
[automerge]1586230815[/automerge]

try reboot your router instead? could be some state problem in your wifi access point or router.

Definitely worth a try! I did find that my router acts as a DNS server (nslookup shows the DNS server IP address as 192.168.0.1). Some addresses were not resolving correctly, indicating that maybe the router is not pulling down up-to-date DNS records from the ISP.

I've now set my network interfaces to use Google's DNS to see if this helps, but it didn't appear to make any difference.

The fact that command line utilities still work (e.g. curl https://www.google.com returns the Google home page) makes me think that the basic network infrastructure is working OK.

Thanks for the suggestion!

 

[johngwheeler]

I've had this with a brand new MBP 16. A complete OS reinstall did not fix the problem - after a time, the browsers have exactly the same issue, even when attempting to connect to an Apache installation on the MBP itself, either via http://localhost or http://{hostname.local}.

Rebooting fixes the issue, albeit temporarily - which is not really a fix at all.

Thanks; I'm glad to know that I'm not alone....although it doesn't really help us find a solution.

As I said, I did a re-installation of MacOS, but kept my apps & data. I don't know whether this kind of reinstallation does a fresh installation of Safari, but I wouldn't expect it to change Chrome or Firefox, but might reinstall the system libraries they depend on.

It's maddening because 90% of my work involves using web-browsers (remote Cloud services).

I'm wondering whether there is any OS process that could be restarted to avoid having to re-boot.

It looks like a time-dependent bug that causing some deeply embedded component to fail after a period of use. Maybe a memory leak or similar?

 

[casperes1996]

Are these libraries unloaded / modified during normal usage without explicitly upgraded anything?

Shouldn’t be, no. I’ve only seen it happen after using the command-line Softwareupdater and ignoring its message about rebooting
[automerge]1586232348[/automerge]

Thanks; I'm glad to know that I'm not alone....although it doesn't really help us find a solution.

As I said, I did a re-installation of MacOS, but kept my apps & data. I don't know whether this kind of reinstallation does a fresh installation of Safari, but I wouldn't expect it to change Chrome or Firefox, but might reinstall the system libraries they depend on.

It's maddening because 90% of my work involves using web-browsers (remote Cloud services).

I'm wondering whether there is any OS process that could be restarted to avoid having to re-boot.

It looks like a time-dependent bug that causing some deeply embedded component to fail after a period of use. Maybe a memory leak or similar?

Doesn’t sound like a memory leak. That’d make the processes take up tons of memory and slow down the rest of the system. A pointer that accidentally gets freed would be more like the symptoms you describe, not being able to load the information it needs, but then I would expect a segmentation fault normally, but I guess the browsers could catch the error and unsuccessfully keep trying to resolve it themselves.

Though that still doesn’t explain why it happens to you and not everyone.

As for restarting a system process that’s hard without knowing what could be the culprit. You could try doing an lsof on all active browsers when it happens and compare to see if there are any interesting files they all hold a file descriptor for.

 

[Martyimac]

This answer wont help any but on my 2019 iMac with 40GB of ram and the 500GB SSD, I am not having this problem. During this Covid isolation I have had my Firefox open constantly from 6AM to 10 PM and not once has this happened. I have had up to 8 tabs open and no issues.
Sorry but maybe that will give somebody some help.

 

[hmkabaker]

Added, odd behaviors sometimes:

Forced reboot with the power button left something lying around after showing the error log. Needs another shutdown and restart. When I did a shutdown from the Apple menu, it did an automatic restart, saying the shutdown was abnormal and the system restarted because of a "problem." In some of these really hard crashes, Bluetooth would stop, so I had to plug in a wired mouse and attach the keyboard with a USB cable to get the error report and proceed. I do not understand the error log. If anyone thinks it will help, I'll go back to Firefox. (Maybe Safari has the same problem, or Catalina has the same problem with Safari and Firefox but not with Chrome?)

 

[johngwheeler]

As for restarting a system process that’s hard without knowing what could be the culprit. You could try doing an lsof on all active browsers when it happens and compare to see if there are any interesting files they all hold a file descriptor for.

Thanks for the lsof suggestion. Worth a go.

I've created a new MacOS user and have been using this for 2 days now. I was just about to report that this had fixed the problem, but within the last 5 minutes the problem showed up again....

 

[johngwheeler]

Thanks for the lsof suggestion. Worth a go.

I've created a new MacOS user and have been using this for 2 days now. I was just about to report that this had fixed the problem, but within the last 5 minutes the problem showed up again....

Hmmm...just tried lsof /Applications/Safari.app/Contents/MacOS/Safari and it hangs...

I wonder if there is a deeper problem with the file system?

 

[casperes1996]

Hmmm...just tried lsof /Applications/Safari.app/Contents/MacOS/Safari and it hangs...

I wonder if there is a deeper problem with the file system?

How do you know it's hanging and not still processing?

- But also I should've been clearer. lsof [path] will show processes that have a file descriptor for that path.
What you want is
lsof -p [pid]
where pid is the process ID of one of the browsers.

I gotta say though, I can't recommend doing this manually. There are a lot of file descriptors and more than one pid for processes relating to Safari - i.e webkit, safari support bundle, safari safe browser.

If you wanted to compare it all you should probably automate it.

Also to show all file descriptors you need root, since you can otherwise only see the ones you own.

Frankly it's probably more effort to go through lsof output than it's worth, I more pointed it out as a theoretical option, though you could automate comparisons and have only the output that's the same for all the browsers which would likely significantly reduce the output to siff through. - There's also a nifty little tool you can brew install called pidof that gets the process ID of whatever you want. So pidof Safari would give you all the PIDs for Safari and processes named something with Safari in it, like the support bundles and such I mentioned before

 

[johngwheeler]

How do you know it's hanging and not still processing?

- But also I should've been clearer. lsof [path] will show processes that have a file descriptor for that path.
What you want is
lsof -p [pid]
where pid is the process ID of one of the browsers.

I gotta say though, I can't recommend doing this manually. There are a lot of file descriptors and more than one pid for processes relating to Safari - i.e webkit, safari support bundle, safari safe browser.

If you wanted to compare it all you should probably automate it.

Also to show all file descriptors you need root, since you can otherwise only see the ones you own.

Frankly it's probably more effort to go through lsof output than it's worth, I more pointed it out as a theoretical option, though you could automate comparisons and have only the output that's the same for all the browsers which would likely significantly reduce the output to siff through. - There's also a nifty little tool you can brew install called pidof that gets the process ID of whatever you want. So pidof Safari would give you all the PIDs for Safari and processes named something with Safari in it, like the support bundles and such I mentioned before

I left lsof running for over an hour with no change. After a reboot it responds immediately....weird.

Good point about using the PID with lsof. As you say, Safari has a lot of processes so your suggestion of using "pidof" with some automation is a good one. Thank you!

 

[hmkabaker]

FWIW, updated yesterday to OS 10.15.4, reverted to Firefox browser, loaded it up with open tabs, ran mail and other programs and let it sit idle overnight with no trouble. Screensaver sleep off and on occasionally. Time Machine updated normally with no hangs. Launched Chrome, which ran slowly; maybe running into a memory problem with just 8 GB in this late 2015 iMac. But no crashes, and browsers find new links in reasonable time.

 

[johngwheeler]

Here's an update to my "browser not opening new pages" problem...

I'm now confident that the issue goes further than my web browsers. The fact that core Unix utilities like "lsof" (list open files) also stopped working indicated that the problem was extensive.

I also found that AVG antivirus and EtreCheck (system diagnostics tool) also stopped responding.

Malwarebytes (another security application) did run OK and reported no suspicious files.

This tends to indicate an issue with the file system, or at least the OS libraries that write to it, and I started to prepare for a clean install by taking a full TimeMachine backup and creating a bootable backup disk with Carbon Copy Cloner (CCC). BTW, MacOS Disk Utility shows no errors on my internal disk.

I booted from my CCC backup without issue (this does require changing the default MacOS Catalina protection to prevent booting from external drives), and have been using this for about 3 days now - without error!

I will continue using the external disk (a 500GB SSD via USB-C) for at least another 3 days to see if any issues occur.

This does start to indicate that the problem may not be a corruption of the OS or applications - these are just copies of the internal disk - but may be something wrong with the internal disk itself.

I am hoping that there is no fundamental hardware error (system diagnostic test passed with no errors), and that simply reformatting the internal SSD and restoring from the CCC backup will fix the issue.

If not, then the next step would be to do a complete clean install from scratch - i.e. reinstall everything again from fresh downloads which is a real bore....

I'm not sure if there is some 3rd party low level disk checking tool that I could use to validate the health of the Mac internal disk. What does the Apple Genuis Bar use to test disks? Unfortunately Apple stores outside China are still in lockdown, so this would have to wait until, who knows when!

 

[johngwheeler]

Latest update: I had to admit defeat and did a clean install of MacOS 10.15.4 after erasing the internal SSD.

My problems didn't end there because *both* my latest Time Machine backup *and* my Carbon Copy Cloner backups failed to restore. My CCC backup disk (an external SSD) became unreadable; I suspect the SSD or the enclosure is faulty because I've seen other problems booting or running MacOS from this disk. In retrospect, this was a very bad choice as a backup disk! I should have discarded it as soon as I saw unreliable behaviour.

No idea why my TimeMachine backup couldn't restore to the new MacOS installation. It went through the whole recovery process with apparent success, but I was unable to boot the Mac afterwards - it just got stuck in an endless reboot loop.

Fortunately I had a second TimeMachine backup on a network drive, which I can mount to use the migration assistant. However, I have decided not to migrate all my previous apps or user profile because I just don't know if these contain any problematic configurations.

So I going to reinstall everything manually, and create disk images at various points during the installation process.

So far, there's no repeat of the weird disk access / browser tab error.

 

[daneintx]

This issue has been driving me crazy lately - and given me a convenient excuse to consider getting the new macbook pro.

But I think I have found a possible culprit: AVG - my antivirus software. I stopped it and voila, everything is suddenly loading fine again. No time outs. At least for now

Anybody else on the thread tried disabling their virus software and see if it had any impact?

 

[casperes1996]

This issue has been driving me crazy lately - and given me a convenient excuse to consider getting the new macbook pro.

But I think I have found a possible culprit: AVG - my antivirus software. I stopped it and voila, everything is suddenly loading fine again. No time outs. At least for now

Anybody else on the thread tried disabling their virus software and see if it had any impact?

AVG? That isn't anti-virus, that's malware in and of itself. An absolute ad-fest. Have a Google around about some of the ***** they've pulled.
Just follow best practices and you don't need to worry about anti-virus; Especially not AVG. I used them for a decade when I was a Windows user btw, so I'm not just talking out my arse.
If you absolutely want an anti-virus tool on macOS, I recommend Sophus or MalwareBytes

 

[americantechieinspain]

This has been driving me crazy too.

The problem: the DNS resolver stops working.
The rest of the TCP stack is working correctly. You can ping IP addrsses on the LAN and on the Internet. Open web pages (DNS names have already been resolved to IP addreses) will continue to work fine.

If check the resolver, I can ping each of the nameservers
%cat /etc/resolv.conf
nameserver 1.1.1.1
nameserver 8.8.8.8

Right now I cant reproduce the problem but I believe it has to to with the mDNSResponder process.

I've noticed that it happens when I connect via AirPlay to my living room TV.
[automerge]1589567753[/automerge]

This issue has been driving me crazy lately - and given me a convenient excuse to consider getting the new macbook pro.

But I think I have found a possible culprit: AVG - my antivirus software. I stopped it and voila, everything is suddenly loading fine again. No time outs. At least for now

Anybody else on the thread tried disabling their virus software and see if it had any impact?

Not in my case. I don't use an AV.
It could be that the all or part of the TCP stack resets when you disable your AV. AVG might be hooking into the DNS resolver.

 

[americantechieinspain]

AVG definitely changes the DNS resolver config to use Secure DNS

AVG Antivirus: Secure DNS

help.avg.com

It seems AVG is either reconfiguring the DNS resolver, hooking into it. That explains why your problem goes away when you disable AVG.