This is not what that setting is for. What you are looking for is the Resume feature, which is located in the same settings panel. Make sure that “Close windows when quitting an application” is not selected. Also make sure that you do not uncheck “Reopen windows when logging back in” when you attempt to log out.