Hello,
I think I've got the same problem : sometimes these icons load, sometimes not. deleting ~/Library/Safari/Touch Icons Cache content when Safari is closed sometimes works, sometimes not. Quitting Safari and re-opening it sometimes is enough to make the icons reappear. Check the two screenshots below, one before quitting Safari, the other after. And sometimes some are loading, some not sometimes a folder only partially loads. When I open it, icons are fine inside, but from the front page only a few appear in the folder. completely random.
Note : Favicons are for the sidebar, TouchIcons are for the front page.
No problem with Safari on iOS, 100% consistently working.
I've already tried closing Safari, disabling iCloud sync for it, removing all Safari folders in ~/Library, caches, etc. and re-enabling sync, it sometimes works for a couple of hours, then fails again. Not a big problem but still makes not sense to me. It was working fine on Catalina. I regret updating... so many problems.
Is it the issue you're experiencing ?
Safari 14.0.1 on Big Sur.