Grunting, boxing & nipping

Hello!
I have a 8 month old Dutch Rabbit. If I reach into his cage and pet him, he will close his eyes and act like he's really getting into it. If I stop rubbing him, he will even nudge my hand with his nose to try and get me to continue.
However, when he is out of his cage, he doesn't like to be held at all and if we try to pet him, he will grunt at us, use both of us front paws (with nails) to box at us and will even bite.
Any suggestions on how to better bond with him? I have tried giving him food/treats while he's out running around in the house, but he still doesn't want to be petted or held.
Any suggestions would be GREAT!!
Thank you!!

Normally it would be the other way around. Buns tend to be cage aggressive and don't want anyone putting their mits in their space. However I think he might be enjoying his pets and attention in a place he considers safe. If you're picking up your bun for snuggles, he probably doesn't like that. Rabbits are prey animals and most don't like being picked up at all. When their feet are off the floor, they feel very unsafe. Try to resist the urge to pick him up. Sit quietly in the floor with a treat and let him come to you. Then give rubs like you do when he's in the cage. Is he neutered?

Yep! He is neutered. Sometimes he grunts and boxes even when he isn't being held. Today for example he was on a step and I went to pet him and he grunted and boxed at me. He also thumps sometimes if we try to walk up to him. Sometimes he can be really sweet while being held...he will even teeth pur.