This is perhaps one of the strangest networking problems I've encountered.
On either of my home Macs (M1 Max 14" on 12.6.3) and my Mac Studio (M1 Max 13.2) I am experiencing a bandwidth clamp in only one direction and only when accessing the Internet. Local traffic is fully capable of saturating the 10gbe link between the switch in my network closet and my home office.
Switches are all Mikrotik with 10gbe SFP+ cages, flow-control is enabled on all ports.
Here is an example - iperf3 local traffic using the built in 10gbe adapter in the Mac Studio.
As you can see, this is a full speed high quality link. I've performed the test over a multi our period and had no dropped packets or issues at the switch level. Fwiw, Windows works as expected in any direction to or from any host on my PC using a Intel Killer 2.5gb adapter. Same applies for Linux using a 10gbe Aquantia card.
Let's perform the same test using the built in Mac Studio 10gbe, but this time target the NYC Fios iperf3 server.
This test (downstream) works as expected and is very fast. Let's do the same test on downstream data.
Here's where it gets very strange, same test on the Mac Studio using a Plugable USB-C 2.5gbe adapter.
Somewhat confoundingly, it's reversed. This adapter works just fine on the MacBook Pro, providing expected speeds in both directions. The behavior is the same on the Mac Studio no matter if the adapter is plugged into a thunderbolt port or a usb-c port. Strangely, if I use a Sonnet Solo10g on either computer, the problem is the same.
I thought it was maybe a negotiation issue with the SFP+ cage, or the cable, or any of the physical link layer -- I tried them all using Windows and Linux and performance was as expected.
Things I've tried otherwise:
Edit: of course as soon as I post I have a minor breakthrough. If I force the Mac Studio 10gbe card or the Solo10g to negotiate at 2500base-T the problem goes away. Obviously this is less than ideal.
On either of my home Macs (M1 Max 14" on 12.6.3) and my Mac Studio (M1 Max 13.2) I am experiencing a bandwidth clamp in only one direction and only when accessing the Internet. Local traffic is fully capable of saturating the 10gbe link between the switch in my network closet and my home office.
Switches are all Mikrotik with 10gbe SFP+ cages, flow-control is enabled on all ports.
Here is an example - iperf3 local traffic using the built in 10gbe adapter in the Mac Studio.
As you can see, this is a full speed high quality link. I've performed the test over a multi our period and had no dropped packets or issues at the switch level. Fwiw, Windows works as expected in any direction to or from any host on my PC using a Intel Killer 2.5gb adapter. Same applies for Linux using a 10gbe Aquantia card.
Let's perform the same test using the built in Mac Studio 10gbe, but this time target the NYC Fios iperf3 server.
This test (downstream) works as expected and is very fast. Let's do the same test on downstream data.
Here's where it gets very strange, same test on the Mac Studio using a Plugable USB-C 2.5gbe adapter.
Somewhat confoundingly, it's reversed. This adapter works just fine on the MacBook Pro, providing expected speeds in both directions. The behavior is the same on the Mac Studio no matter if the adapter is plugged into a thunderbolt port or a usb-c port. Strangely, if I use a Sonnet Solo10g on either computer, the problem is the same.
I thought it was maybe a negotiation issue with the SFP+ cage, or the cable, or any of the physical link layer -- I tried them all using Windows and Linux and performance was as expected.
Things I've tried otherwise:
- Jumbo frames while pulling all other non jumbo frame capable devices off of the network. No change.
- Flow-control on/off.
- Changing TCP window size - I have Linux configured to 4mb and it works perfectly, no change in MacOS.
- Every possible combination of Autonegotiation on/off, flow control on/off, energy efficient ethernet on/off
- Every possible combination of speed testing utilities and websites, all with the same results.
Edit: of course as soon as I post I have a minor breakthrough. If I force the Mac Studio 10gbe card or the Solo10g to negotiate at 2500base-T the problem goes away. Obviously this is less than ideal.
Last edited:
