Yes, you can mix display sizes with the W5700X (or any other GPU).
The Mac Pro 2019 can have 6 Thunderbolt buses (two per GPU, one for the top Thunderbolt ports, and one for the rear I/O card). Two Thunderbolt ports per Thunderbolt bus.
With the W5700X by itself, you have 4 Thunderbolt buses. Two on the GPU, plus two more for the the top and rear of the Mac Pro.
The W5700X has 6 DisplayPort connections. Two DisplayPort connections per Thunderbolt bus. 4 DisplayPort connections go to the two Thunderbolt buses on the card. 2 DisplayPort connections go to either of the top and/or rear Thunderbolt buses. You can have one DisplayPort connection per Thunderbolt port, or two DisplayPort connections to the same Thunderbolt port.
Connecting displays to the W5700X:
- 4K displays always take one DisplayPort connection so you can connect 6 of these.
The original, old 4K displays that use MST may count as two DisplayPort connections - they use two streams over one DisplayPort connection, one stream for the left side and the second stream for the right side of the display.
- The LG UltraFine 5K display (5K 60 Hz 10bpc) takes two DisplayPort connections over a single Thunderbolt connection (one DisplayPort connection for the left half of the display, and the second DisplayPort connection for the right side of the display). You can connect one LG UltraFine 5K display per bus, three buses max.
Dual cable 5K displays like the Dell UP2715K or HP Z27q are like the LG UltraFine 5K but they don't use Thunderbolt. Connect the two DisplayPort inputs of the display to any two DisplayPort outputs from the same GPU. Three pairs of DisplayPort connections max.
There exist 5K displays that use a single DisplayPort 1.4 connection (5K 60Hz 8bpc). You might be able to connect 6 of these (like the 4K).
- If the 6K display connects using DSC (6K 60Hz 12bpc) (the W5700X and XDR display support DSC - it's up to the graphics driver to use it), then it takes one DisplayPort connection and you should be able to physically and electrically connect 6 of these but Apple says only 3 are supported. Maybe 4 XDR displays is two many pixels? I don't think anyone has tried more than 3.
If the 6K display connects using two DisplayPort connections (this is a mode supported by the XDR for Macs with GPUs that don't support DSC to achieve 6K 60Hz 10bpc), then the count will be limited to 3 displays max, one per Thunderbolt bus, like the LG UltraFine 5K.
- 8K: not much info about this - 8K 30Hz has been shown to work on some Macs.
So, given the above, you can connect any combination of three displays of any size. Some combinations will allow up to 6 displays.