I agree with your theory. I believe that if Apple could raise performance reasonably by increasing power consumption, they would have done it.I think power consumption is the key consideration here. Apple cores have been steadily increasing their power consumption but are still way below the typical enthusiast desktop levels. The way I understand @theorist9 question is why not use the higher thermal ceiling available on a large desktop and get some more performance along the way? I have been asking the same question myself. It's just that, looking at the power curves, I am not sure it is that much of an option with the current designs. This is what leads me to wonder whether the scaling is limited by the chip design itself to reach the high performance we have now and a more scalable core would not be able to have that same performance at the current power consumption targets. At the end of the day, it's like Germans like to say: everyone is cooking with water. The technology available to everyone is fundamentally the same, and while Apple has the edge of being on N3, it doesn't give them any magical scalability.
I think they found out that there is a point of dimnishing returns. Unfortunately, that point is likely not competitive with other desktop offerings.