Think of rabbits as OCD little animals =), back when i first got my rabbits we kept them in their cage and they grew extremely attached to their toys, everytime we touched it Latte (the dominant female) would growl or swipe at you. If you do manage to move one of her toys, a couple mins later she would run over and move it back to how it was.
Shortly after they were both spayed and litter trained, we let them out 24/7 and of course they soon forgot about their toys, now they don't even play with them anymore =(