If the camera is important to you, get the XS.
If the OLED screen is important to you and you love its clarity, etc get the XS (personally I think I’m affected by PWM so my phone choice this year was pretty easy!).
If having a phone that is more or less one handed and can easily fit in your pocket get the XS.
If having a big screen is important to you, with the camera being of secondary importance and you’re ok with an lcd screen, get the XR.
I’m assuming that since you bought the Max in the first place that price isn’t a consideration!
Sounds like the XS will probably be best for you though.