5120x1440 120Hz is 966MHz using CVT-RB timing.
That's like one 5120x2880 60Hz displays.
966MHz * 24bpp = 23Gbps.
But Thunderbolt is only 40 Gbps.
So you cannot connect two of those to the same Thunderbolt port unless you can reduce the bpp using DSC or 4:2:0. The displays probably don't support 4:2:0 (and it wouldn't look nice anyway) so DSC would be the way to go (only if your GPU supports DSC and the OS allows it).
Now, you could add a DisplayPort 1.4 MST Hub to each display. Mac OS doesn't support multiple displays using MST, but it can use a MST Hub to convert DisplayPort signals. A DisplayPort 1.4 MST Hub could take a HBR2 + DSC (12bpp) signal and convert it to HBR3 30bpp for the display.
However, macOS probably isn't smart enough to let you connect two DisplayPort 1.4 MST hubs at HBR2 link rate - it would probably connect one at HBR3 and the second at HBR.
HBR can do 966MHz only if DSC is set to 8bpp instead of 12bpp. I don't think macOS attempts DSC at any mode other than 12bpp.
But macOS might not choose to use DSC at all.
Two displays to a Thunderbolt port won't work if macOS sets one to use HBR3 and the other to use HBR.
On Intel Macs, I would use AGDCDiagnose to check the display link rates (HBR, HBR2, HBR3) and DSC status. I don't know how to do that on M1 Macs.
On Intel Macs, I would use WhateverGreen/Lilu to patch some GPU stuff (though currently it can't patch CoreDisplay framework in user space). I don't think anyone's looked at patching M1 Mac stuff.
An old Thunderbolt dock like the CalDigit TS3+ only supports HBR2 max link rate. I don't know if that output would support DisplayPort 1.4 MST Hub DSC input.