Sounds annoying and I don't see this on my computer. Have you tried to reset Safari?
From the Apple forum:
Safari/Preferences/Advanced - enable the Develop menu, then go there and Empty Caches. Quit/reopen Safari and test. Then try Safari/History/Show History and delete all history items. Quit/reopen Safari and test. You can also try try Safari/Clear History…. The down side is it clears all cookies. Doing this may cause some sites to no longer recognize your computer as one that has visited the web site. Go to Finder and select your user/home folder. With that Finder window as the front window, either select Finder/View/Show View options or go command - J. When the View options opens, check ’Show Library Folder’. That should make your user library folder visible in your user/home folder. Select Library/Caches/com.apple.Safari/Cache.db and move it to the trash.
Do a backup, preferably 2 backups on 2 separate drives.
Quit the application.
Go to Finder and select your user/home folder. With that Finder window as the front window, either select Finder/View/Show View options or go command - J. When the View options opens, check ’Show Library Folder’. That should make your user library folder visible in your user/home folder. Select Library. Then go to Preferences/com.apple.Safari.plist. Delete the .plist.
Restart the computer, open the application, and test.