I take it from your username you have experience this firsthand. ? :: eek ::
Yes
I have several flashlights that have aluminum bodies, including some penlights one can carry in one's pocket. But when I am going to be outdoors for a long time on a cold day or night, I may keep one or two of the pocket flashlights in one of my pockets, and grab the powerful flashlights that have rubber grips or the ones with composite plastic bodies. Also, when being outdoors and away from electricity, I take non-rechargeable flashlights and headlamps. The headlamps don't have metallic bodies or cases, which is a good thing since they are mounted in close proximity to one's forehead.
By the way, cold aluminum in one's barehand is nothing compared to a cold copper pipe. Grabbing a -20-degree F copper pipe with one's barehand can be painful most times.