The PCH is a a lot like the older ICH. It is a super I/O chip (You might get a few more PCI-Express lanes for things like another Ethernet port, FireWire, USB 3.0 at this time.) The PCH just takes on SATA, USB, and RAID functions and not much else.
I realize how you're looking at it, and I don't entirely disagree (definitely can be viewed as a Super ICH of sorts).
But where it gets more complicated, is how many PCIe lanes it has.
If it's just a few, say 8 for example, then that's only enough for attaching 3rd party chips to add features that either aren't in Intel's parts, or the board designers chose to use another manufacturer (i.e. may go for Marvell controllers for SATA, SAS, or even hardware RAID rather than the X79 that includes the hardware RAID 5 controller). In such a case, Super ICH would apply.
Now lets consider the PCIe lanes on the CPU. Those 40 lanes of Gen 3.0 on the LGA2011 can produce say 2x 16 lane slots + 1x 8 lane slots (16/16/8) = 3 slots (5, if each group of 16x lanes is attached to 2x slots for an 16/0 or 8/8 configuration). Though fast, it's a bit limiting for that much CPU (need fast I/O to keep the cores fed), and 2x GPU cards and a RAID card would consume all the lanes. Nice and fast.
But what if that system also needs an FC, 10G Ethernet, or even an Infiniband card?
There's no slot to put it in.
In the case of Ethernet, users are stuck with whatever controller is on the board. Granted, this is more of a unique case, as it's possible enterprise boards could include a 10G Ethernet chip via PCIe lanes to the X79. But they're on the expensive side (~$70 just for a 1G chip in quantity; 10G was all "Call" - go figure

), and won't leave any lanes if the X79 only has 8 lanes available.
Example of a 10Gb/s converged controller that uses 8x lanes (Gen 3.0 compliant). So it's more likely IMO, that another card will be needed.
But if there are more lanes, say up to 40, then that's more than should be necessary for 3rd party chips. Thus allowing for additional slots @ PCIe Gen 2.0 spec.
Hence why I see it as having the potential of being more than just a turbo charged ICH.
Now which is correct, I'm not entirely certain yet. Need more corroboration than what I've seen so far to give this additional merit, or ideally, clear confirmation from Intel. But what is available is compelling enough to let the drooling begin.
I have seen this mentioned before as well. The number of lanes beyond the DMI 2.0 x4 ones provided still are not concrete. The standard bidirectional 20 Gbps that DMI 2.0 provides screams lacking when you are talking about up to 14 hard drives off the PCH alone.
Nothing is, as it's premature leaks (even the Intel slides leave at least as many questions as it does answers).
But in cases like this, corroboration helps to sort it all out (i.e. multiple people that have access to information covered by NDA make their own Block Diagrams that find their way on the web).
Dropping PCI is worth noting but not terribly important. In most of the X58/P67/Z68 block diagrams I have looked over, most peripheral controllers are already on PCI-Express. The few PCI slots you do see are bridged over. There are many more boards going exclusively PCI-Express as well.
I wasn't talking about slots, but rather that designers were still using the PCI bus for 3rd party chips soldered to the board (how they could do this and leave all of the slots PCIe). PCI was still usable for things like FW800, SATA (still usable for mechanical disks), 1G Ethernet, ... sorts of things.
I'm not sad to see PCI go (tired old bus that was ready to retire), but there does need to be a way for 3rd party chips to be added so board manufacturers can create a a product line for multiple price points/needs. With PCI gone, the logical successor is PCIe.
Now if the CPU is the only portion that will offer PCIe lanes (for use with 3rd party chips or slots), it won't turn out too well. Either there won't be enough slots on the board, or they won't have enough lanes available for full bandwidth for the cards located in them (i.e. trying to run an 8x lane card in a 1x or 4x slot that can actually exceed the throughput possible in a 4x lane slot).
We will have to get more information on that. Internal QPI is not something new.
No it's nothing new. Nor do we currently have that information from Intel.
But given their use of systems engineering (recycling/expanding existing tech), particularly in a subsequent new architecture (not going to re-invent every single aspect from the ground up if they can help it), internal QPI is highly probable IMO.
It is sadly, solely based on what I know. I am still expecting much more than a single DMI 2.0 interface from the processor to the X79 PCH.
I'd be amazed if this is all they did, but even Intel's own slide on the X79 that leaked clearly indicates PCIe lanes (QPI) for chip to chip communications.
This is why I'm convinced that there is at least more than DMI 2.0, which though faster, would still saturate (still has the SATA limitation, even though it's been increased to a larger value; figure its in the ~1660MB/s range for SATA, assuming the same ~340MB/s or so is reserved for the Ethernet, USB, Audio,... controllers). I hope they didn't just double everything (reduce SATA throughput limit ~1320MB/s).
The mention of PCIe lanes for SATA controller throughput would help this immensely if it's configured (seems this will only be available on the -D <Performance> and -T <RAID> variants).
This may help (reposted for convenience).
From what I know, X79 is also backward compatible with Nehalem/LGA 1366 processors as well.
I would suspect so as well.
I doubt they'll discontinue the X58/5520 immediately though, as enterprise parts are typically supported for 5 years from socket debut. So it should be around about another 3 years.
There's also a financial side to this as well, as the X79 likely isn't using the same component package. Board makers won't want to use the X79 as it would mean a new board design on a design they only want to coast on (keeps it profitable vs. new design over fewer sales).
And then there's the IT side, as they want identical systems for management purposes to reduce their man hours (more people, or existing people putting in overtime).
It's possible that there will also be LGA 1356 chips for low-end DP market. nanofrog has already explained the complexity issue of LGA 2011 chips which adds costs so LGA 1356 would make sense. LGA 1356 would most likely have less PCIe lines and memory bandwidth though. AMD's upcoming Opteron lineup is looking pretty strong as well so maybe Intel will finally try to gain some market share in the lower-end server market.
Whether these will find their way into Mac Pro is another question though.
Yes, the LGA2011 is more complex. But it's also the right target to keep up with what the MP currently is, rather than taking a step backward.
The specific reason for this, is the LGA1356 parts only have 24 lanes. Fine for a mid-level server that won't need a lot of PCIe cards (primary need for slots is fast networking and/or storage; slots likely to be configured in 3 * 8x lane slots as devices like RAID, FC, and 10G Ethernet controller cards are usually 8x lane).
Not so much for a high-end workstation that users are likely to stuff to the gills, and will want at least one 16x lane slot. I can see it now... Apple announces "Brand new MP with a whopping 2 slots! Yeah!". I suspect at least a few users would want to string Steve up by the family jewels over something like that.
I still feel that the jury is still out on LGA 1356. We still get reports that the socket exists in some sense but X79 appears to be tied to LGA 2011, with quad channel RAM no less, and possibly to the older Nehalem LGA 1366 processors as well.
Granted, the X79 is usually mentioned with the LGA2011, but with the systems engineering that Intel seems to be employing, I actually suspect it will work with both the LGA2011 and LGA1356 parts (same interconnect implementation for the different socket CPU's).
Which translates into financial sense. One chipset that works on multiple socket CPU's, rather than a chipset per socket. Given the fact the socket count has increased, this seems the logical approach IMO to counter R&D and increased production costs (ability to better utilize existing production facilities due to fewer SKU #'s = controls production costs, such as eliminates additional tooling <converting another facility to the correct process> or need for new facility construction).
I know this reflects the consumer side of things but Intel used the Core i7 9xx line-up as the dress rehearsal for the later Xeon lineup.
Again, it's systems engineering. The consumer parts you're speaking of will almost certainly have a direct Xeon equivalent (ECC is left Enabled, otherwise no difference).
Another thing I've noticed that could be causing additional confusion with the X79, is there are 4 variants (different features, and distinguished by separate SKU's <-A, -B, -D, -T>). And the information being released, is likely focused on the Premium variant (-T SKU), which has the additional SAS/SATA controller (additional 8 ports; this portion is also in the -D variant) as well as the hardware RAID 5 controller (hardware RAID 5 = -T variant only).