On the PC and consumer side of things, Intel required a new motherboard and chipset if you wanted to upgrade your CPU. In other words, the CPUs from 1 generation wasn't compatible with the motherboard and chipset from another generation. Every subsequent generation (at least from Skylake onwards) required a new motherboard and chipset.
Not necessarily, no. The motherboard in my kids' PC has an 8th gen Core i3 processor in a FCLGA1151 socket. Flashed with latest firmware it will support any 9th generation Core processor. And I would not be surprised if this support was extended to 10th generation desktop processors when they are released.
Similarly, flashing the MP4,1 with a 5,1 firmware allows one to use the later hex core processors.
Now, even if there was no upgrade path to newer generation CPUs, it could still be interesting to do a mid life upgrade of a machine with a higher-end but used CPU.