You are correct that rabbits handle cold better than heat. However, do you know if this rabbit was living outdoors before you got him? If not, he may not have had reason to grow a winter coat. If this is the case, it would be better to house him indoors.
If he was housed outdoors, it is still an option to house indoors. Many (most?) here on RO house their rabbits indoors. Don't worry. If you are new to this, rabbits can certainly by housed indoors without an odor problem. They actually do quite well indoors. The greatest advantage is being able to build a bond with him. That just isn't likely to happen when he's outdoors. Single rabbits don't fare so well outdoors either since they will spend far too much time alone.
I'm sure with a 3 year-old, you are busy enough. You aren't likely going to have time to stand out in the chill for a couple hours per day trying to bond with the rabbit. But even 2 hours per day is very little time to expect bunny to get to know you. And that would still leave him alone for 22 hours every day.
Housed indoors, bunny is nearby 24/7. Bunny being in proximity like that makes the human family familiar and part of bunny's life.
It sounds like you've got the basics down with hay and pellets (those pellets should be limited, btw, at his age). Indoors, once he's settled, you could also begin to slowly introduce greens to his diet. There is a right way to do this so as not to upset bunny's tummy. You can see that
here.
Once you've browsed around this forum and looked through the following links to some of our indoor housing, you may want to consider bringing him indoors.
http://www.rabbitsonline.net/showthread.php?t=87073
Of course your 3 year old will need to be taught that she should not try to pick up bunny. Most rabbit people will tell you that a 3 year old is too young for a rabbit. Generally that is true. But with your guidance, she can learn to interact appropriately. [Just yesterday I happened upon an old photo of my youngest sitting on the patio when we took our indoor rabbit out for a bit. You can see that he is sitting quietly and gently petting without attempting to pick up.]