Perhaps others can chime in on this topic, but I had always heard that while rabbits can handle a good amount of cold, what is NOT good is for them to be switched frequently from warm to cold and back again.
I had read that if the inside and outside temps are more than a 10-15F difference, that it is not advised to continually change them from inside to outside. In other words, if it is cold outside (30s or 40s) and the rabbit is going to be staying outside at night, then it should stay out 24/7 so it can acclimate. Rabbits have a difficult time regulating their body temperature otherwise.
So my understanding is that in the temps the OP is describing that the rabbit should either remain indoors or remain outdoors, but not be switched back and forth.