Siri can essentially recognize any voice out of the box (hence why you don't have to spend half an hour programing it to your voice like older, more archaic voice rec tech). Siri contacts Apple for each query, so its not actually remembering your voice but more your preferences, which are stored remotely so, no, it does not forget you when you turn it off.