Hello,
You really do not need to worry about GoFetch as an average user. It is not about stealing passwords; It's about peeking into isolated data from beyond the process isolation boundary, i.e. data in flight.
It requires that you are running malicious software that can invoke your cryptography function to its heart's content with chosen inputs, or if you speak cryptography, a CPA (Chosen Plaintext Attack). More ideally it can mess around with intermediate values or something but it needs some form of guidance over what it's trying to exploit. It is primarily a risk if you run a server on your machine. In that case malicious software on said machine can snoop on the prefetching of data while spamming the public endpoint with connections with chosen encryption challenges and extract the private key from it even if it should only live in the process space of the server daemon and not be accessible to the program in question.
The issue is entirely orthogonal to the merits of password managers, but I will say this. Password Managers are, in almost every circumstance, more secure and safe than not using them. It is way more likely that a server you've interacted with gets hacked than that your personal machine gets taken over. In the case of a password hash list breach, if you use a password manager, they will get a nearly uncrackable hash that has only been used once. If not using a password manager you will have picked a memorable string that you have probably used several other places, and it's now considered compromised on all those sites not just the one that had a data breach.
But GoFetch does not relate to storage, and as the unlocking of your password vault is a one time operation per use without inputs owned by other processes, it would take a billion years for a GoFetch malware, running on your computer, to get anything out of your keychain.
Also note that Intel chips also have hardware vulnerabilities. Some mitigated, and countless others yet undiscovered no doubt.