I often wonder why Dell, HP, etc can make very good Intel laptops, yet Apple seems to always be "waiting for" or "disappointed in" Intel. Could you please help me understand why that is?
Pretty much what @
kschendel said - some of other companies' upgrades have been of questionable value if you look carefully at the specs - they may have downgraded from a 'premium' iGPU to a regular one, or switched to a lower-powered (i.e. slower) processor model just to get the "New 9th Gen Processor" label. This is aided and abetted by Intel's "i3/i5/i7" branding which is almost pure marketing and tells you very little about the specs. Also, since
all of Apple's Mac products are ultra-slim or small-form-factor their designs tend to be very dependent on the performance and thermal/power characteristics of a particular range of processor models.
...also, the MacOS GUI is quite GPU intensive (esp. with non-integer scaling, which is the default in some models) so it can't really make do with a sub-par iGPU.
On the other hand... Intel's release cycle is neither a new phenomena or a closely guarded secret, and Apple have huge resources. They could probably try a bit harder - not wait 4-5 years between major re-designs, maybe diversify their Mac range to include some models that prioritised power over thinness. I can't see any reason - for example - why the 2018 Mac Mini got 8th-gen chips 6 months before the iMac. I suspect time/money investment in Mac takes a back seat to the more profitable iPhone and that Apple are heavily relying on MacOS lock-in to keep the Mac customers on board.
Also, wouldn't Intel NUCs and other mini-PCs be superior alternatives (OS aside) to the Mac mini?
Many of the NUCs and similar machines are a bit "fake" in that they look very neat in the photos - where you can't see the humungous external power brick or hear the fan going insane. The Mac Mini is almost unique in having a built-in PSU.
However, yeah, on the face of it, Apple skipped an opportunity to build a "Skull Canyon" Mac Mini, which would have been a major boost c.f. the 2014 model, and the Hades Canyon - with semi-discrete AMD graphics (i.e. there's a CPU and separate GPU chip combined into a single physical package) would have seemed like the obvious choice for the 2018 Mac Mini.
If the OS was not an issue, if I was looking for a Mini PC I'd certainly consider either the NUCs or one of the many Mini-ITX or smaller PCs that offer a more sensible balance of size and expandability than the Mini. Plus, all joking apart, if you're happy with Linux and don't need high performance check out the new Raspberry Pi (about $70 by the time you add a case, PSU and SD card).
I hate to say it, but the 2018 Mini is a blatant example of cutting the cost while increasing the price and convincing the faithful that it is somehow an "upgrade". However you try to rationalise it, a quad-core
desktop-class i3 is a substantially cheaper component than the contemporary
mobile-class i5 processor that would be the natural progression from the older models.
Finally, I've always wondered what's so great about integrated GPUs, ARM or otherwise? Can they really compete with discreet GPUs from AMD and Nvidia?
Cost, space - it's one less chip on the motherboard - and power consumption/heat. The Intel ones, certainly, can't compete with discrete AMD/NVIDIA GPUs but the better ones are
good enough for general use. They're certainly the sensible choice for something like a MacBook Air.
...but MacOS really needs the better Iris/Iris Pro (or whatever Intel is calling them today) integrated GPUs - which Intel only builds into laptop chips. The iGPU in Intel
desktop chips is only really fit for basic "business graphics" - not running multiple 4k or 5k displays in GPU-intensive scaled mode.