If it is removed from the list, then it will log out automatically. It seems like Apple decides to make it all or nothing.
The concern is, if someone gets hand on the logged in device (logged in device is always trusted), and happens to figure the apple id password out, he/she can then get the authentication code on the very same device. I would think there should be an option to allow some devices just logged in but not fully "trusted".... but it does not seem to work.