Hi.... I'm looking for an explanation for a issue I've had to deal with recently.
This issue was something I thought I had resolved several years ago at the office.
This subject is of prime importance for me as I employ Thunderbolt Bridges between MP6,1s at the office. One of the MP6,1 acts as the File Server (not a true File Server mind you) for all other Macs in the office in addition for doing Production work. The MP6,1s are beasties and pack a lot of horse power. The MP6,1 File Server has a ton of RAID devices as well as devices for backing up the RAID devices every day in the early hours of the morning. The MP6,1 File Server's RAID devices can deliver well over 700 MBytes/sec, the MP6,1 File Server's internal 1TB SSD can deliver over a 1000 MBytes/sec and the MP6,1's internal RAM can deliver over 5000 MBytes/sec.
The office MP6,1 are both 12core, 64GB RAM, Dual D700s and have 1TB Apple PCIe SSD inside. They are running El Capitan's latest build.
The MP6,1 File Server can deliver extremely high-speed i/o to its Client MP6,1s so long as the Thunderbolt Bridge and the underlying transfer protocols are working correctly. The MP6,1 systems employ Thunderbolt 2 so 20 Gbps is possible. That is a whopping 2,500 MBytes/sec. This is more than enough to allow the MP6,1 File Server to deliver its i/o from its RAID devices, internal SSD and 1/2 of the RAM bandwidth.
I had an ongoing issue for using the Thunderbolt Bridge with apple a few years ago. At first they gave me decent attention to the issues I was having. However, after some time their help dwindled to nothing. They simply stopped communicating even though they had said their Engineers were looking into things for me. Since then I've been relying on my own testing to arrive at the best configuration for maximizing the i/o transfers over the Thunderbolt Bridge.
This past weekend I had to resolve some disk failures for the office MP6,1 File Server and after resolving that I wanted to ensure the Thunderbolt Bridge was performing as expected.
My previous testing a few years back showed that the SMB file transfer protocol was superior to Apple's AFP. With that in mind I made sure SMB was the selected choice for File Sharing. In Sierra, SMB is selected by default.
My testing after the issues with the MP6,1 File Servers RAID disk failures showed that the best i/o rates across the Thunderbolt Bridge was around 150 MBytes/sec. This was determined using AJA, Blackmagic and the dd command in Terminal. All three were in reasonable agreement. This result was troubling me as I was expecting at least 400 to 500 MBytes/sec.
When monitoring the transfers (using Activity Monitor and XRG) I could see that during the i/o transfer of large files (10 GB in size) the real time transfer were in fact hovering around 400 to 700 MBytes/sec. However AJA, Blackmagic and dd were all showing the overall average i/o rate was no better than 150 MBytes/sec.
I asked my son to monitor performance of his production workload on one of the MP6,1 Client systems and to let me know if he experienced poor performance. It's quite obvious performance is bad if the i/o rates being delivered by the MP6,1 File Server are too low to sustain smooth video editing, footage playback and adding effects. My son has indicated the Client MP6,1's performance isn't great but is acceptable for him. This report is not what I consider to be optimal and tells me there's something wrong with the Thunderbolt Bridge configuration.
This issue caused me to scratch my head as I thought I had fixed this issue long ago (several years ago).
I therefore started to explore more options using my home Macs; iMac17,1 and MBP9,2, both running macOS Sierra 10.12. The iMac has two Thunderbolt 2 ports on a single bus and the MBP has single Thunderbolt 1 port.
I started off using the same File Sharing configuration I had in the office. That is, using SMB. The results for file transfer rates were pretty darn close to what I had seen at the office. Max overall average i/o rate was around 150 MBytes/sec.
As a further test I decided to switch from SMB to AFP in the File Sharing panel.
I ran all my i/o tests again and was mightily surprised.
The following were results using my MBP to write and read data to/from the iMac. Before each test run I executed 'sudo purge' on both the MBP and iMac to ensure data was being purged from the kernel buffer cache.
The results below were for data being written to and read from the iMac's internal SSD. The dd command when reading from the iMac was directed to /dev/null to avoid the i/o from being throttled by the slow MBP's internal disk.
AJA now was reporting: 728 MBytes/sec for writes and some 462 MBytes/sec for reads.
Blackmagic was showing similar results to AJA
The dd command was showing 410 MBytes/sec for writes and 432 MBytes/sec for reads
The results below was for data being written to and read from the iMac's Promise Pegasus2 R6 RAID-0 device that can deliver over 700 MBytes/sec.
AJA now was reporting: 729 MBytes/sec for writes and some 485 MBytes/sec for reads.
Blackmagic was showing similar results to AJA
The dd command was showing 410 MBytes/sec for writes and 406 MBytes/sec for reads
Using the MBP the following result was for data being read from the iMac's RAM that can deliver over 5000 MBytes/sec.
The dd command (outputting to /dev/null) was showing 3,605 MBytes/sec for reads.
Thus, use of AFP clearly was trumping the use of SMB. This was a very surprising result.
I suspect Apple has made SMB the default to benefit users sharing files with Microsoft systems... but dunno if this is being actually realized by these type users. From my testing the AFP protocol is vastly better for Mac to Mac Thunderbolt Bridge configurations.
After the above testing at home using the iMac and MBP I had my son at the office switch from using SMB to AFP and it has made a huge difference for the MP6,1 Client machines.
Data is now moving across the office Thunderbolt Bridge at over 500 MBytes/sec at times. Earlier today son had to run things at half resolution on the MP6,1 Client. With the change from SMB to AFP he can now run at full resolution on the MP6,1 Client. That's all the proof I need to know that makes AFP the better choice.
I'm curious to know if anyone else can confirm my findings here. Thanks in advance....
This issue was something I thought I had resolved several years ago at the office.
This subject is of prime importance for me as I employ Thunderbolt Bridges between MP6,1s at the office. One of the MP6,1 acts as the File Server (not a true File Server mind you) for all other Macs in the office in addition for doing Production work. The MP6,1s are beasties and pack a lot of horse power. The MP6,1 File Server has a ton of RAID devices as well as devices for backing up the RAID devices every day in the early hours of the morning. The MP6,1 File Server's RAID devices can deliver well over 700 MBytes/sec, the MP6,1 File Server's internal 1TB SSD can deliver over a 1000 MBytes/sec and the MP6,1's internal RAM can deliver over 5000 MBytes/sec.
The office MP6,1 are both 12core, 64GB RAM, Dual D700s and have 1TB Apple PCIe SSD inside. They are running El Capitan's latest build.
The MP6,1 File Server can deliver extremely high-speed i/o to its Client MP6,1s so long as the Thunderbolt Bridge and the underlying transfer protocols are working correctly. The MP6,1 systems employ Thunderbolt 2 so 20 Gbps is possible. That is a whopping 2,500 MBytes/sec. This is more than enough to allow the MP6,1 File Server to deliver its i/o from its RAID devices, internal SSD and 1/2 of the RAM bandwidth.
I had an ongoing issue for using the Thunderbolt Bridge with apple a few years ago. At first they gave me decent attention to the issues I was having. However, after some time their help dwindled to nothing. They simply stopped communicating even though they had said their Engineers were looking into things for me. Since then I've been relying on my own testing to arrive at the best configuration for maximizing the i/o transfers over the Thunderbolt Bridge.
This past weekend I had to resolve some disk failures for the office MP6,1 File Server and after resolving that I wanted to ensure the Thunderbolt Bridge was performing as expected.
My previous testing a few years back showed that the SMB file transfer protocol was superior to Apple's AFP. With that in mind I made sure SMB was the selected choice for File Sharing. In Sierra, SMB is selected by default.
My testing after the issues with the MP6,1 File Servers RAID disk failures showed that the best i/o rates across the Thunderbolt Bridge was around 150 MBytes/sec. This was determined using AJA, Blackmagic and the dd command in Terminal. All three were in reasonable agreement. This result was troubling me as I was expecting at least 400 to 500 MBytes/sec.
When monitoring the transfers (using Activity Monitor and XRG) I could see that during the i/o transfer of large files (10 GB in size) the real time transfer were in fact hovering around 400 to 700 MBytes/sec. However AJA, Blackmagic and dd were all showing the overall average i/o rate was no better than 150 MBytes/sec.
I asked my son to monitor performance of his production workload on one of the MP6,1 Client systems and to let me know if he experienced poor performance. It's quite obvious performance is bad if the i/o rates being delivered by the MP6,1 File Server are too low to sustain smooth video editing, footage playback and adding effects. My son has indicated the Client MP6,1's performance isn't great but is acceptable for him. This report is not what I consider to be optimal and tells me there's something wrong with the Thunderbolt Bridge configuration.
This issue caused me to scratch my head as I thought I had fixed this issue long ago (several years ago).
I therefore started to explore more options using my home Macs; iMac17,1 and MBP9,2, both running macOS Sierra 10.12. The iMac has two Thunderbolt 2 ports on a single bus and the MBP has single Thunderbolt 1 port.
I started off using the same File Sharing configuration I had in the office. That is, using SMB. The results for file transfer rates were pretty darn close to what I had seen at the office. Max overall average i/o rate was around 150 MBytes/sec.
As a further test I decided to switch from SMB to AFP in the File Sharing panel.
I ran all my i/o tests again and was mightily surprised.
The following were results using my MBP to write and read data to/from the iMac. Before each test run I executed 'sudo purge' on both the MBP and iMac to ensure data was being purged from the kernel buffer cache.
The results below were for data being written to and read from the iMac's internal SSD. The dd command when reading from the iMac was directed to /dev/null to avoid the i/o from being throttled by the slow MBP's internal disk.
AJA now was reporting: 728 MBytes/sec for writes and some 462 MBytes/sec for reads.
Blackmagic was showing similar results to AJA
The dd command was showing 410 MBytes/sec for writes and 432 MBytes/sec for reads
The results below was for data being written to and read from the iMac's Promise Pegasus2 R6 RAID-0 device that can deliver over 700 MBytes/sec.
AJA now was reporting: 729 MBytes/sec for writes and some 485 MBytes/sec for reads.
Blackmagic was showing similar results to AJA
The dd command was showing 410 MBytes/sec for writes and 406 MBytes/sec for reads
Using the MBP the following result was for data being read from the iMac's RAM that can deliver over 5000 MBytes/sec.
The dd command (outputting to /dev/null) was showing 3,605 MBytes/sec for reads.
Thus, use of AFP clearly was trumping the use of SMB. This was a very surprising result.
I suspect Apple has made SMB the default to benefit users sharing files with Microsoft systems... but dunno if this is being actually realized by these type users. From my testing the AFP protocol is vastly better for Mac to Mac Thunderbolt Bridge configurations.
After the above testing at home using the iMac and MBP I had my son at the office switch from using SMB to AFP and it has made a huge difference for the MP6,1 Client machines.
Data is now moving across the office Thunderbolt Bridge at over 500 MBytes/sec at times. Earlier today son had to run things at half resolution on the MP6,1 Client. With the change from SMB to AFP he can now run at full resolution on the MP6,1 Client. That's all the proof I need to know that makes AFP the better choice.
I'm curious to know if anyone else can confirm my findings here. Thanks in advance....
