Never have loyalty to any company - especially a bank. They are not loyal to you and will change policies, remove features, increase prices, or sell their business to a competitor on a whim.
This is especially true of banks and banking products. Always go with the lowest fees/highest savings rate. Most HISAs are fee-free these days anyway, so you can keep multiple accounts open and just move money into the one that gives the best return.
Companies make money by taking it away from you. Do whatever gets you the best deal, because if you don't, no company will ever do it for you.
While I agree 100% with the sentiment, I'd say that you also have to factor in the hassle & time it takes to switch & monitor accounts.
I'm not loyal to any bank, but won't be moving $ out of my Apple Savings Account to chase a few 1/10ths of a % of interest. Not out of loyalty to apple, but because the rate is close enough & the benefits of moving for that little don't outweigh the cost.
A savings account, in my mind, is there to give you access to liquid cash when you need it. You don't want to keep a ton of $ in there as you're losing $ on it over time, relative to other investment opportunities. And when you do the math on even $100,000, a 1% difference (4% to 5%) only works out to $1,000/year. At $10,000, it works out to $90/year. While that's not nothing, it is also small enough that I'm not going to waste time moving my $ around & setting up accounts to chase that. Apple is at 4.1%, I think some of the others mentioned here are 4.7%, so that 0.6% would earn $600/year, or $50/month if you had $100,000 in the account. At $10,000, it's $5/month.
But I do agree with wiki, don't ever base your savings on loyalty to any bank (or company). Find ones that work for you & when they no longer work, move on to one that does.