I just tried this and it works as expected. To enable it, here are the instructions from Apple's website:
- On a Mac with Apple silicon: Press and hold the power button until you see “Loading startup options.”
You will then see the available USB drives to boot from. Interestingly, they need to be plugged directly into the Mac and you cannot use a hub.