I'm curious: why aren't you connecting the 5k directly and the rest via DisplayLink? Have you ran out of ports? There are TB3 hubs out there to help with that, but I haven't tried any. Also see above: it may be a good idea to write to DisplayLink support. They've answered to my queries pretty fast and with a useful response, not copy paste.
The only reason is because I already have a Pro Display XDR connected directly. My old setup was an iMac Pro with the Blackmagic eGPU to get 6k out of the Pro Display XDR (couldn't without it), then I had the 5k iMac Pro screen, and I used a third 32-inch Asus PQ321Q 4k display. It was a lot, but those three amazing screens helped with my workflow.
After I decided to go M1 MacBook Pro, I sold the iMac Pro (decided to sell while the getting was good).
Using DisplayLink with the Asus PQ321Q 4k display wasn't good -- it would not scale and keep HiDPI. This is because it is an MST display, and MST doesn't seem to work, and in SST mode its only 30 Hz, which I could not tolerate.
So, I've got my Pro Display XDR connected directly to my M1 MacBook Pro. That leaves me with only DisplayLink solutions for the other displays.
I got a Dell UltraSharp U2720Q 27 Inch 4K to use as my second display, but while it is 60 hz, it won't scale to look like 2560x1440. That resolution shows as not HiDPI if I try to change it using SwitchResX or EasyRes. If there is a way, I haven't found it yet. The best I can do is 1920x1080, which is a bit large. I am likely to return that display.
That display is a step down from the 5k iMac Pro display I was using as secondary. I was considering the LG 5K display, so that it wouldn't look so bad next to the Pro Display XDR. I'd also settle for a 4k display that can definitely do 2560x1440 in HiDPI.
I subsequently read about how 32-inch 4k displays scale better than 27-inch ones, so I started focusing on that size again. I would appreciate any tips!