This is a timely post, and something I have thought about. I'm in North Carolina, and it just doesn't make sense for me here to get winter tires, so I run summer tires all year round. The few times it snows I just stay off the road, and when it gets below the mid-forties I don't push them, I drive gently.
I have wondered about damaging the tires from driving them in cold weather, but so far it hasn't happened. I ran the OEM Bridgestone Potenza RE050As (140TW) for a couple of winters, and did the same with the Firestone Firehawk Indy 500s (340TW) which I have now. This included driving them very infrequently in temperatures down into the 30s, and there were some nights when they sat in the unheated garage when temperatures went down into the 20s or even lower on a very few rare occasions. I know that theoretically you're not supposed to do this, but as I said I have not had any issues so far.