No, you're right.
Maybe others will jump in and say otherwise but the sort of thing that would happen typically, would be you'd want to upgrade the graphics card as that would see a huge increase in performance, but you'd find your power block wasn't powerful enough or it physically wouldn't fit.
Or you'd want to upgrade to thunderbolt ports or whatever, but the BIOS needed to be upgraded, or they'd be incompatible anyway. Or you'd want to put in a new HD and find the BIOS was physically limited to a certain size.
Or you'd do loads of things and find there was still a bottleneck somewhere and as the system is limited by its slowest part, it didn't make any difference (Bus speed for instance).
I'm sure there are people who successfully managed it- and I did have limited success too- but generally you had to really be into it, start with something that could be upgraded for size, power, heat dispersal etc, and really understand what you were getting yourself into.
Very often, you'd get 6 years into a machine and find so many things had improved or changed anyway (Wifi, Bluetooth, USB, bus speed, physical RAM connectors) that spending ages to get some part of it to work better didn't really make a lot of sense and that's basically how 99% of the population ended up.
Very few people ever really want to get into their machine, and besides, newer machines are very often actually cheaper than tinkering with an old machine and everything is up to date.