I have lived in a few places that in hunting seasons it could get to like you (-20) or it could be +75. its the dramatic shifts that break down powder.. if it's frozen, warming it up slowly is a good thing. the same thing goes for your primers. More so for the primers. after only a few dramatic swings I have made new primers inert and will not fire. However they will burn..
the best thing you can do is insulate the powder from dramatic changes in temps. if you can limit the temp swing then that is even better. Powder and primers like a stable, slow to cool, slow to heat environment. I have some 1918 ammo (30-06) that still fires but is starting to show sparklies out the barrel and hang fires. I am going to stop using it for the fact it's now becoming a bit unstable and the brass is doing some really major splitting of the casing all over the place.
Just as a side note.. Somehow I can see this as a very interesting thread.. Do not, I repeat do not ever store ammo on our dashboard in the sun on hunting trips: EVER! I had a boss guilty of this. he would leave his ammo on his dashboard over the summer, over the winter, he bounced the ammo around, degraded the coating on the outside turning what was once H-4350 into something the equivalent to Bullseye. to say the least, that rifle catastrophically failed on an Elk hunt. Ron was/is okay. he was not truly harmed except a few small scratches and some splinters. the bolt was never found, the receiver was in multiple pieces and the scope kept him from loosing his life (we suspect). I also talked to a few of my fellow gunsmiths, this aslo goes for ranchers. they do the same thing with coyote gun ammo. I do not know this for a fact but more than one fellow gunsmith has told me this.