Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Thunderbolt 2.0: capable of PCIE 3.0 16x cards in external enclosures?
- Thread starter darkgoob
- Start date
- Sort by reaction score
2 hoses will fill a bucket twice as fast as 1 hose
What if you only wanted water from Hose A, as that was the file you're looking for? Wouldn't it be better to just have a single hose that's twice as big (or in the case of PCIe, 4 times as big?)
I hope we're talking about the same thing, it's hard to figure out where the comment tree came from.
Apologize in advance if I got it wrong.
What if you only wanted water from Hose A, as that was the file you're looking for? Wouldn't it be better to just have a single hose that's twice as big (or in the case of PCIe, 4 times as big?)
I hope we're talking about the same thing, it's hard to figure out where the comment tree came from.
Well I guess you better wait for TB3 or TB4 then.
It was a theoretical question I posed. I was wondering if it were possible to aggregate the TB ports for more throughput. The water would already be mixed and require water from both Hoses for a complete payload.
I would want things to be way different too, but that wasn't the question I was pondering.
4K video is not low-bandwidth, especially when you're dealing with multiple streams of 4K video. Most 3D graphics apps will transfer their assets to video memory, and then interact with them from there (i.e. the only bandwidth-intensive part is the initial transfer). There might be certain classes of 3D graphics workloads that require a lot of data streaming, but this certainly doesn't apply to games etc.
Quick back-of-napkin math:
4K video = 3840 x 2160 x 4 = ~32MB per frame * 30 FPS = ~958MB/sec
So, each 4K stream at 30 FPS is nearly 1GB/sec.
You know, I wonder how this will affect benchmarks on TB2 vs 2GBps PCIe with external GPUs.
As far as running modern GPUs over TB2, I'm done debating with one-sided benchmarks. The Techpowerup article (which is the closest thing we have to actual "data") says that there's wide variation between GPUs when using lower bandwidth and that as CPUs get more powerful, the bottleneck will become more apparent. Other sites have pointed out how game-specific it can be--that some games show a loss of ~40%.
What I'm predicting is that when the new MP comes out, fanboys will declare anything that's not than an IvyBridge-E PC to be an invalid testing station; and therefore can live in their blissful ignorance thinking that 2GBps is just as good as 8GBps when it comes to video cards.
You know what, fanboys: Have fun with your $500 PCIe Chassis and inadequate bandwidth, I'm building a PC.
As I understand it then under OSX then the drivers need to be Thunderbolt aware. You cannot just stick any PCI-E card in a PCI-E Chassis and it works. You need specific driver support, hence why people such as sonnet publish a list of cards know to work in the PCI-Express/Thunderbolt chassis.
Only GPU card seen supported is a Matrox Card, which isn't exactly high end.
Windows ironically doesn't have this issue as it apparently see's the Thunderbolt Chip as a PCI-Express Switch so works out of the box. ( I will leave aside that if running Windows why are you looking at a Mac Pro )
As such even if you can physically fit the GPU in and transfer the data, you still need a Thunderbolt aware driver.
Hopefully more companies will actually do the driver development necessary.
Only GPU card seen supported is a Matrox Card, which isn't exactly high end.
Windows ironically doesn't have this issue as it apparently see's the Thunderbolt Chip as a PCI-Express Switch so works out of the box. ( I will leave aside that if running Windows why are you looking at a Mac Pro )
As such even if you can physically fit the GPU in and transfer the data, you still need a Thunderbolt aware driver.
Hopefully more companies will actually do the driver development necessary.
He should have explicitly said 'relatively low bandwidth' . It is relative to the context of requiring x16 PCI-e. For PCI-e v2.0 that is 8GB/s . So even 120 FPS is half that.
The card does work with x4 but just "real time" for all shooting contexts. The x16 physical seems more so driven to keep folks from trying to put it into shorter x4 electrical (or less because really sitting on PCI-e bandwidth expansion (dilution) switches ).
On a mutliple active device TB network probably would see those kinds of issues ( presuming mutiple devices with traffic going in same direction. )
----------
OS X sees the chip the same way. And no Windows doesn't work correctly "out of the box" , unless definition if 'works' is behaves badly when unplugged. There may be more hooks to get something to generically 'wink in' when it appears. But I think OS X 'knows' that just adding TB devices that don't appear to have savy drivers is just a bad idea that isn't really going to work. It will just appear to work.
This is not true. The Thunderbolt controller isolates the Thunderbolt specific parts from the rest of the system. From intel's brief:
To be clear, I'm agreeing with you -- I expect TB (even TB2) to have a noticeable performance impact, and as that article says, it's only going to get worse as CPUs and GPUs get faster. For certain workloads, it'll probably be enough to make it unusable.
TB1 is not 16 lanes of PCIe 2.0 though, it's 4 lanes (so call it ~2GB/sec). Assuming that TB2 does actually double that to 8 lanes of PCIe 2, you're still only looking at ~4GB/sec of bandwidth to and from the card. As I said, if you're rendering a 4K video project that combines multiple streams of video, it's not going to take much before PCIe becomes the bottleneck. Do you really want your 4K video renders to only happen in real time? I'm not a video guy, but I would've thought you'd want a background render to happen faster than real time.
Now, having said that, there are a ton of other workloads outside of 4K video that probably won't run into the same limitations. I'm happy to see the enclosures coming down in price, there was one announced recently that was targeted at $200 or so. We'll see, I guess. Pretty sure my next system is going to be a Hackintosh though.
TB2 is still 4 lanes of PCIe 2.0 but it combines the two 10Gbps channels into a single 20Gbps channel to support the transmission of 4K video over Displayport 1.2. The PCIe and DP are multiplexed onto that 20Gbps link so you'll probably want to run any bandwidth intensive PCIe devices off a separate TB controller from the one your using to drive your 4K display. Thankfully the new Mac Pro has 3 TB controllers so this shouldn't be a problem.
Ahh, makes sense. So in that case, you're still basically limited to the ~2GB/sec from 4 lanes of PCIe Gen2, which would make working with bandwidth-intensive workloads like 4K video a challenge.
Yeah, I should have said relatively low bandwidth. A 16x PCIe bus could do 1GB/sec with tons of bandwidth to spare. So it's not unreasonable to expect it could be done over Thunderbolt.
Games can be more intensive depending on their textures. A single 4K frame may pale in comparison to the number of pixels for all the textures required to render a scene. But at least games have the advantage of pre-loading things in VRAM.
The worse as faster presumes that more memory won't get stuffed onto the games oriented graphics cards and game shift to larger caching strategy.
Sure for more GPGPU work it will get worse, but frankly the only sizable market for eGPU is primarily focused on gaming performance 1-2 years from now.
TB v1 or v2 is irrelevant to the Red Rocket card's requirements. That's what was driving its x16 physical subthread.
2GB/sec is about 60FPS which is plenty for film. (non slow motion).
Frankly though, at anything like 50-60 FPS the new Mac Pro design is a better fit. There is 3,600 seconds in an hour. At 2GB/s that's 7.2TB/hour of data. two cameras 14.4TB/hour of data. Shoot 2-3 hours how do you put that all inside of current Mac Pro? You can't.
Super high storage data bandwidths drive storage out of a singular CPU focused box. You end up with multiple box systems because storage needs are out of control.
It doesn't. TB v2 doesn't do much for PCI-e data traffic other than offer better x4 PCI-e v2 transport (i.e., better QoS : quality of service). That's it. The shuffling of the deck chairs in TB v2 was to enable 4K video transport. Thunderbolt data packets are not the same thing as PCI-e or DisplayPort data packets. Thunderbolt can move at 10 , 20 , or 30Gbps but that isn't going to make that remote USB 3.0 controller any faster. Same thing is true for the underlying PCI-e data packet transport.
TB v2 isn't going x8 PCI-e v2 because there aren't four spare, largely unused lanes lying around in most Intel PC designs. There aren't any spare v3.0 lanes lying around either ( mainstream/laptop designs just have x16 of those and those are typically totally consumed by the GPU(s)). There aren't even one spare lane (either speed) lying around in this 2013 Mac Pro.
In "onramp" / "offramp" bandwidth increase is coming largely from the other set of ramps to/from Thunderbolt; DispayPort. That bandwidth has increased in newer Intel PC reference designs. TB v2 adapts to that.
TB v3 may move the PCI-e on/off ramp to PCI-e v3.0, but that will take a while. And that probably depends upon cheaper fiber arriving too.
It won't if just avoid Thunderbolt all together. Plug a DisplayPort v1.2 cable into the PC host socket and it won't clog up Thunderbolt backbone traffic at all. That limits the flexibility of the TB topology but the 2013 Mac Pro has six sockets. If blow one or two on 4K video still have plenty left.
And there is always the HDMI socket (unless doing high gamut color).
Some folks will "have to do it". ( e.g., a laptop and only have two, or less, TB ) sockets. But this Mac Pro design should allow anyone to avoid that problem if need max PCI-e bandwidth with high QoS.
Right, I'm talking about the people who want to throw a GeForce GTX TITAN card into an external enclosure and use that as their main GPU. For a multi-stream 4K video workload, the limited bandwidth will probably make this unusable compared with the internal FirePro GPUs. There are plenty of other scenarios where it will work just fine, however.
Thunderbolt was never designed for that. That's why gobs of bandwidth are dedicated to what comes out of the GPU; not what goes in. One of principle inputs to a Thunderbolt controller is GPU out signals.
That is also why the new Mac Pro isn't trying to export them either.
The eGPU factor , if there every is a viably large enough market , is far more focused on supplementing iGPUs so that those systems have either a more servicable lifetime or so better characteristics though another down stream video out when stationary and plugged in. A Titan on x4 TB is going to be faster than a HD4600 or HD5000 solution inside of some Mac that only has one GPU.
As a supplemental GPU should work. You can use it to peel off a subset of work from the FirePro GPUs but to be $/performance effective would need to be leveraging them for at least some work. You can't pump all the work of two GPUs over a x4 PCI-e v2.0 link.
For anyone that owns an ATTO R644, here is some information you may find helpful.
1. If you are using both ports (internal & external), placing it in an external TB enclosure will obviously make you lose one port.
2. So if you are planning to place your HD's/SSD's that were in your MP into an external SAS box, you can only have one 4 drive box connected. Your other 4 lane port is now held prisoner in the TB enclosure.
3. You now have half the card you had before!
http://www.magma.com/expressbox-3t
Here there is a testimonial that mentions using the R644 in a Magma box. "And though we demonstrated the limits of the transfer speeds over Thunderbolt, those speeds are fully adequate for many video and audio applications".
The key words in that quote are "adequate" and "many".
They did not list the speeds before and after the enclosure! How limited are the "limits of the transfer speeds?"
Lastly, they did not mention loosing the internal port of the R644 in the enclosure!
1. If you are using both ports (internal & external), placing it in an external TB enclosure will obviously make you lose one port.
2. So if you are planning to place your HD's/SSD's that were in your MP into an external SAS box, you can only have one 4 drive box connected. Your other 4 lane port is now held prisoner in the TB enclosure.
3. You now have half the card you had before!
http://www.magma.com/expressbox-3t
Here there is a testimonial that mentions using the R644 in a Magma box. "And though we demonstrated the limits of the transfer speeds over Thunderbolt, those speeds are fully adequate for many video and audio applications".
The key words in that quote are "adequate" and "many".
They did not list the speeds before and after the enclosure! How limited are the "limits of the transfer speeds?"
Lastly, they did not mention loosing the internal port of the R644 in the enclosure!
Attachments
Wouldn't all three of your bullet points there be solved by drilling a hole in the case and cabling it out to a second 4-bay box or whatever?
No.
http://www.netstor.com.tw/_03/03_02.php?MTE2#
Since the above is no, this one evaporates too.
Buzzz, wrong answer..
A relevant point you could have made is that this is a x8 PCI-e v2.0 card and Thunderbolt isn't going to get you anything more than x4. So the card is a bit of a mismatch. It will work, but the bandwidth will go down. If it is just average HDDs hanging off though that may not be a problem ( 4 assigned to work space and another 4 assigned to backups ).
I think the
is missing from your post. But I did contact all makers of TB external enclosures and they said drilling a hole in their $1000.00 boxes would not void any warranty!Deconstruct60, After reading the specs on the box you linked, I saw where they connect to the internal port of the R644 with an internal cable. This is the same type cable I have connected to the R644 to currently bypass the logic board. There are 8HD's total in my raid. 4HD's in the MP via the internal port of the R644 and 4HD's externally in an OWC SAS raid box via the external port of the R644. Hence R6(44)
In the box you speak of, what does that cable do that would be connected to the "interior" 4Lane port of the R644? How can you attach 2 4HD SAS boxes to the one 4Lane port? Does that interior cable attached to the interior 4Lane port of the R644 turn the exterior 4Lane port into an 8Lane port?
Explain how using this box can get 8Lanes out of this card if 4Lanes are inside?
Tesselator's answer would work but obviously in jest!
Last edited:
For Mac Pro
That's for a Mac Pro, which pretty much defeats the whole purpose of an external graphics card. The whole point of an external graphics card is to allow laptops like my 2011 13 inch MacBook Pro to use a desktop graphics card without having to buy an older Mac Pro or a PC desktop, or even having to spend $2500 on a new Mac Pro.
That's for a Mac Pro, which pretty much defeats the whole purpose of an external graphics card. The whole point of an external graphics card is to allow laptops like my 2011 13 inch MacBook Pro to use a desktop graphics card without having to buy an older Mac Pro or a PC desktop, or even having to spend $2500 on a new Mac Pro.