While I agree with you that iPadOS really needs some significant upgrades, I think - if this rumour turns out to be true - that it is not about the CPU or GPU. As you said, even the M1 chip is perfectly capable for 90% of all tasks an iPad does.M4, when M1 is overpowered for iPads?
At this point, its just spec-chasing until the software catches up to the hardware.
There are a lot of rumours on Apple going all-in on AI functionality. Just like you generally don't use the CPU to create images (that's a GPU task), you want the AI tasks to be done by dedicated cores which are enhanced for this task.
If Apple does go all-in on AI functionality, well, the M3 chip actually lags quite a bit behind the A17 Pro chip - while the M3 came out later! The M4 chip would, presumably, offer a significant boost in terms of processing cores enhanced for AI tasks - over the M3 chip.