MP All Models Mellanox ConnectX-3 40 GbE using ATTO FastFrame macOS driver

[thefloyd]

hi churek:

A copy of the NVRam contents would be ideal. On my machine with the current drivers it looks like the command is (from a terminal window):

cd /Applications/ATTO/FastFrame2

sudo ./atnetnvr2 -s ~/Documents/attonvram.backup

(alternatively, sudo ./atnetnvr2 -s /full/path/to/attonvram.backup if you don't want it in your personal Documents folder)

You should end up with a file called attonvram.backup in your 'Documents' folder (modern OSX may prompt you to allow things to write to 'Documents' when you execute this, feel free to save anywhere you're comfortable, this just seemed easy). It doesn't appear from the command line help that you need to specify a device specifically:

thefloyd@Patricks-MacBook-Air-2 FastFrame2 % ./atnetnvr2

Usage:

atnetnvr2 <options> [command]

Options:

-a {length} Set the TX ring buffer length.
-b {length} Set the RX ring buffer length.
{length} is the number of ring buffer entries.
Valid values are 64-8192. 0 indicates driver default.
-c {channel} Selects a specific controller channel for the operation,
starts at 1. For dual port adapters, you must also
specify the port number with -c {channel}:{port},
eg. -c 1:1 for the first port and -c 1:2 for the second.
-d {feature} Disable an NVRAM feature.
-e {feature} Enable an NVRAM feature.
{feature} values:
adapcoal Adaptive interrupt coalescing.
checksum Hardware checksum offloads.
dcb Enable DCB.
rsc Receive Side Coalescing.
tso Enable Transmit Segmentation Offload.
-h Display extended help.
-i {usecs} Set the interrupt coalescing. {usecs} is a number of
microseconds from 0 to 1000.
-l List the controllers in the system.
-m {bytes} Threshold size for inline transmits.
Valid values are 0-104, with 0 disabling inlining.
-p Print the contents of the NVRAM when other operations
are complete.
-q {length} TX queue length.
Valid values are 8-8192. 0 indicates driver default.
-r {filename} Restore the NVRAM from a file.
-s {filename} Save the NVRAM to a file.
-t Restore the NVRAM to default settings.
-v Print tool version information.
-z {length} Set the Transmit Segmentation Offload length.
{length} is the number of TSO bytes.
Valid values are 1500-65536. 0 indicates driver default.


No options found.


I'm going to try and poke at this more at some point today myself, but without a 'genuine' NVRam we might not get far.

Minor edit, it appears the command needs to be invoked per channel (port) on the card as such:

sudo ./atnetnvr2 -s ~/Documents/atto.nvram.port1 -c 1:1
sudo ./atnetnvr2 -s ~/Documents/atto.nvram.port2 -c 1:2

FWIW I reset the NVRAM to factory and power-cycled the card. The ATTO tool lists the MAC address as 00'd out still. I brought the card back to a linux box and booted it up and it's fine.

Going to need a working NVRAM dump to compare against the NVRAM dump from my card to see if it's obvious if/where the MAC address is stored in the file, then it might be possible to edit the file and atnetnvr2 -r [filename] to bring the card to life.

 

[jan2ooo]

Are there any news on actual devices? Is it possible to flash 100gbe Mellanox cards to work with ATTO drivers?

 

[unphased]

The real question is when will Atto and Chelsio release driverkit drivers for Apple Silicon?

 

[thefloyd]

The real question is when will Atto and Chelsio release driverkit drivers for Apple Silicon?

atto drivers already exist for Apple Silicon. They have for some time.

 

[thefloyd]

Are there any news on actual devices? Is it possible to flash 100gbe Mellanox cards to work with ATTO drivers?

as per my last post nothing seemingly works without the contents of a 'genuine' atto nvram on MacOS. It appears the driver pulls the card's MAC address from the NVRAM and thus the card initializes with a mac of 00:00:00:00:00:00 without the proper NVRAM. Been waiting on someone with a genuine ATTO card to give me an NVRAM dump so I can see if it's something that can be dealt with.

 

[unphased]

@thefloyd have you gotten in touch with @churek ? They posted earlier in this topic saying they have an ATTO card.

 

[rdmitry911]

Hi Simonji,
I have the original atto card. May I help you with something ?

Hi Churek, can you post here nvram from the original atto card, as it seems strictly required to make melanox card to work with atto drivers.

Thank you, Dmitry

 

[hasselix]

So, friends, is there a successful way to use atto-driven now?

 

[unphased]

atto drivers already exist for Apple Silicon. They have for some time.

I am interpreting this as saying that *at this time* since we don't have a resolution to this very thread yet, we still need to shell out $750 or $1000 for these ATTO FastFrame NQ41/42 products to get above 10Gbit via one card on Apple Silicon.

As I understand it, there exist multiple potential Aquantia (now Marvell) 10G options available for apple silicon with a native driver at the $200 or so price point.

I'm still trying to find a sub-$200 10Gbit option for use with my M1 Max Macbook Pro.

Being able to use a MCX353A or 354A (or even 313A/314A) to get 40Gbit for $30 or ~$150 would still fit the bill here. But obviously a company like ATTO would prefer us not to figure out how to do that.

It's annoying that the multiple AQC chip bearing devices I have don't work with my eGPU enclosure (asus areion aqc107 card, innodisk egpl-t101 aqc113 m.2 2280 -- It seems like a aqc107s card or something might be my ticket but if i'm buying something, then i'd just get something with a more compact enclosure)

 

[hasselix]

as per my last post nothing seemingly works without the contents of a 'genuine' atto nvram on MacOS. It appears the driver pulls the card's MAC address from the NVRAM and thus the card initializes with a mac of 00:00:00:00:00:00 without the proper NVRAM. Been waiting on someone with a genuine ATTO card to give me an NVRAM dump so I can see if it's something that can be dealt with.

After flashing nvram, did you succeed?

 

[LeoSpikes]

Hello folks,
I bought a cx3pro with two QSFP+ ports and successfully flashed the firmware onto the card. But my big problem is:
Under Windows 10 it works just fine but it prevents my Mac Pro 5,1 from booting. No chime and no display output. Does it have something to do with the Mac’s NVRAM?
And as soon as I unplug the Mellanox card, the Mac boots just fine then. As if nothing has happend.

I tried the card on three different Mac Pro’s. It prevented all of them from booting.

 

[Florist9266]

It looks like this guy just got an ATTO NQ3401 (Mellanox connectx-3 pro), we might be able to ask him to dump the data.

 

[xjn819]

hi guys, i just registered to the forum for this topic, as i own this ATTO card with dual ports.
quite interesting and I hope I can help.
unfortunately I am not a tech guy, please guide me how to acquire the firmware so that i can share.

it is better to get the fw by using softwares, i dont want to disassemble it as it may lose warranty.

thanks

 

[Florist9266]

hi guys, i just registered to the forum for this topic, as i own this ATTO card with dual ports.
quite interesting and I hope I can help.
unfortunately I am not a tech guy, please guide me how to acquire the firmware so that i can share.

it is better to get the fw by using softwares, i dont want to disassemble it as it may lose warranty.

thanks

@xjn819 could you follow the steps here? https://forums.macrumors.com/thread...-fastframe-macos-driver.2225779/post-30902243

1. Open your terminal and run the following commands:

sudo ./atnetnvr2 -s ~/Documents/atto.nvram.port1 -c 1:1
sudo ./atnetnvr2 -s ~/Documents/atto.nvram.port2 -c 1:2


2. @unphased @thefloyd, are you able to help him?

I am looking forward to the progress.

 

[xjn819]

hi
the nq3402 works well for my M1pro,

tried to acquire the nvram files, it came out :

 /Applications/ATTO/FastFrame2/atnetnvr2 -s ~/Downloads/atto.nvram.port1 -c 1:1
WARNING: NVRAM not found
Falling back to default NVRAM for this port

/Applications/ATTO/FastFrame2/atnetnvr2 -s ~/Downloads/atto.nvram.port1 -c 1:2
WARNING: NVRAM not found
Falling back to default NVRAM for this port

although it still generated atto.nvram.port1, no port2 could be generated.

find the attached file pls

 

[Florist9266]

hi
the nq3402 works well for my M1pro,
View attachment 2138310


View attachment 2138311
tried to acquire the nvram files, it came out :

 /Applications/ATTO/FastFrame2/atnetnvr2 -s ~/Downloads/atto.nvram.port1 -c 1:1
WARNING: NVRAM not found
Falling back to default NVRAM for this port

/Applications/ATTO/FastFrame2/atnetnvr2 -s ~/Downloads/atto.nvram.port1 -c 1:2
WARNING: NVRAM not found
Falling back to default NVRAM for this port

although it still generated atto.nvram.port1, no port2 could be generated.

find the attached file pls

Thanks!

 

[xjn819]

I guess I should plug on both ports to acquire the two files?

 

[Florist9266]

I guess I should plug on both ports to acquire the two files?

Worth a try

 

[LeoSpikes]

The file atto.nvram.port1 contains a string (I think), it says: FFRM]h

I think this string can be used on both ports? If I just rename the file to atto.nvram.port2 and flash it onto the 2nd port.
Does this make sense? Cause I have no Idea.

Is this string maybe something like a firmware lock? So the firmware needs to read this specific string so that the network port can be initialised?

And how is it possible to flash the nvram to the card?

 

[LeoSpikes]

I think this string can be used on both ports? If I just rename the file to atto.nvram.port2 and flash it onto the 2nd port.
Does this make sense? Cause I have no Idea.

Ok just renaming the file does not make any sense. I meant is it worth trying to flash this one file onto both ports' nvram?

 

[willowthewisp]

The file atto.nvram.port1 contains a string (I think), it says: FFRM]h

I think this string can be used on both ports? If I just rename the file to atto.nvram.port2 and flash it onto the 2nd port.
Does this make sense? Cause I have no Idea.

Is this string maybe something like a firmware lock? So the firmware needs to read this specific string so that the network port can be initialised?

And how is it possible to flash the nvram to the card?

You should know something about the Mellanox / ATTO cards.
First - the TB based card firmware does NOT work for PCIe cards. The driver won’t load, but the fw can be modified.
Second - the NVRAM stores only port modifications, if needed at boot time (1/10 gbe mode…fixed). It is not containing the IP-Adresse needed by the driver. You can desolder the I2C chip from the PCIe and the cards still work without a hiccup!
Third : The first 2kb in the SPI flash chip contain the IP-Adresses used by the macOS driver. The area is write protected.
You are only able to flash this area in liverish mode or if you desolder the SPI flash. That means you have to remove the heat sink, desolder the 8 pin SPI bug, read the fw. A CH341 will do !
For more than one clone in your network you have to modify the IP-Adresses with a hex-editor (for both ports) and so for times before flashing - this IP is hard coded !
Forth: Even on the old 10gbe Intel x520/x540 cards this worked flawlessly!

 

[grad]

It would be even more interesting if someone made it work for Mellanox ConnectX-4 and the ATTO driver/firmware for the corresponding ATTO FastFrame 3 Ν311 and N312 cards. If I wanted to upgrade from 10 GbE I would rather go all the way to 100 GbE.

 

[nforce4]

It would be even more interesting if someone made it work for Mellanox ConnectX-4 and the ATTO driver/firmware for the corresponding ATTO FastFrame 3 Ν311 and N312 cards. If I wanted to upgrade from 10 GbE I would rather go all the way to 100 GbE.

I also have got Mellanox ConnectX-4 MCX4121A-ACAT 25GE card and thunderbolt to pciex16 dock （ Intel J7440 chipset）。Intel x520 works fine on it ，my laptop is M1 Macbook Air。if someone can make it ，i can upgrade 10Gbe to 25GBe。but for the thunderbolt interface limit，the maximum speed can reach only 26gbps

The Video Link I test x520 on thunderbolt to pciex16 dock

 

[rdmitry911]

You should know something about the Mellanox / ATTO cards.
First - the TB based card firmware does NOT work for PCIe cards. The driver won’t load, but the fw can be modified.
Second - the NVRAM stores only port modifications, if needed at boot time (1/10 gbe mode…fixed). It is not containing the IP-Adresse needed by the driver. You can desolder the I2C chip from the PCIe and the cards still work without a hiccup!
Third : The first 2kb in the SPI flash chip contain the IP-Adresses used by the macOS driver. The area is write protected.
You are only able to flash this area in liverish mode or if you desolder the SPI flash. That means you have to remove the heat sink, desolder the 8 pin SPI bug, read the fw. A CH341 will do !
For more than one clone in your network you have to modify the IP-Adresses with a hex-editor (for both ports) and so for times before flashing - this IP is hard coded !
Forth: Even on the old 10gbe Intel x520/x540 cards this worked flawlessly!

Does that mean, that closed FNP jumper and patched firmware is enough to spoof the device and get it to work with ATTO drivers?

 

[willowthewisp]

That means you have to desolder the SPI chip, flash it with the ATTO fw, solder it back and the ATTO driver will work. More than one clone means to modify the MAC with a hex editor before burning the SPI chip. The first 2kb of the fw a important !
I never used the lifefish/FNP method, always used a SPI programmer.
Most people tell you about a hot air soldering solution - use soldquick and a good soldering iron, heat up both sides of the 25xxx chip, use lots of flux. Always clean the stuff (pcb) with break cleaner (hexan) and a q-tip.
But remember that the PCIe 2.0 x8 bus will limit the port to appx. 3GB on a Mac Pro 5.1. If you want to go 40/50 Gbit a PCIe switch (2.0 x16 to 3.0 x8) is needed.