It's faster on wifi sometimes on cellular takes a little bit try turning off and back on
I've experienced this as well... I thought this was Apple servers, but option to turn "music library" on and off worked first time, every-time, i can only guess its just due to cellular.
same thing with "loading library"... has to be on wi-fi... (maybe its just a requires "better" cellular connection ??, no idea), but once library is loaded on Wi-Fi, i can switch to cellular and play ok without buffering, which means it can't always be cellular. Though still an issue, outside of Apple.]
I have 4G and its actually faster than my home DSL. Been though many iPhone's and all of them experienced the same, so its defiantly provider.
Not saying that it "requires wifi" but this is my experience... Yours may be different.