Another way to look at it is to compare Apple to HP, Dell, etc. HP, Dell, etc, use big contracts with Microsoft to get lower volume prices for Windows. They also shove in a lot of trialware, shareware, and other forms of licensed crapware to subsidize the cost. Get rid of those aspects and their prices would go up as well.
As far as all the crapware, it's definitely true the consumer systems get loaded up with it (particularly the budget boxes).
But if you order a business machine, it's not there. They know businesses don't want it, nor want to spend the time removing it. So vendors cater to this market ("extras" in terms of hardware may be missing as well, such as SD card readers for example, as businesses tend to see that as a potential security risk). So the profit margin works out.
As per Apple's pricing though, most of it is due to their love of high margins from what I've seen (i.e. Gross Margin of ~41% last I saw). Any discounts on software don't translate to a PC so much in terms of the user, as is applied to the vendor's margins (Dell, HP, ...).
Looking inside a Mac Pro chassis and a $1400 DIY desktop PC that has the same types of internal components (Xeon, et al), the Mac Pro is much better designed - virtually no cables, great cooling... it's wonderful from a technical standpoint and I came from a PC environment, both boxed and DIY setups... even the Dell Precision line - cables aplenty and a poor interior design layout in general. Yuck.
In terms of workstations and servers on the PC side, I've seen both extremes (rats nest to extremely clean and well laid out, even for the same model). So I've always wondered if the sloppy units (not a result of bad design) were the result of the system being assembled on either a Monday or Friday (spaced out/not interested in what they're doing due to weekend activities - daydreaming if you will, and possibly hung over on Monday

).
When I think of vendors such as Silicon Graphics, DEC, and Sun of systems' past, I don't ever recall seeing a rats nest that wasn't the result of an IT person. Those would blow the MP away in terms of how well they were laid out, cooled, and ease of replacing components in a hurry. I really miss systems like these....
I've yet to look at a new HP all-in-one device, but I suspect its screen will be a TN panel. Definitely not an H-IPS that Apple routinely uses for the iMacs.
Not sure if this would be the case unilaterally (you've not indicated a model or series), but for the low cost versions, TN would be the way they'd have to go to meet the manufacturing budget. Upper scale versions however, may use the same panel (i.e. Dell uses the same H-IPS panel in the iMac in one of their monitors, so it's not impossible that HP could use it in a product or two either).
Not to mention, fewer Mac owners means they have to pay any difference on top of the rest.
There is truth to this (lower economy of scale), but in the case of an MP, they follow Intel's reference designs for the most part (added FW controller is one exception).
There is no real "Apple tax" as far as I'm concerned. There are, however, subsidies, crapware, Windows OS and its problems (the registry, fragmenting file systems, no native support, etc), et cetera. Windows might be faster at gaming, but that's only as long as the games are DirectX-optimized and the ports made to OpenGL, etc, just remain "good enough".
There's problems with OS X and firmware as well on the Mac side, so nothing is perfect. Macs can even get infected with malware if the user isn't careful, so they're not even immune to this.
I say let the user figure out which platform is best fit for them (system + software + any training that would be involved), and get on with doing whatever they need to do.
