I started out with does as pets. I think it depends a lot on how you interact with your rabbits. My girls all went through the "don't touch me" stage. One was even mildly cage aggressive when she reached about 6 months old. But I was young and too in love with bunnies to let them sit. So despite their wishes, I'd reach in and pet them, carry them around, cuddle them, take them on walks, etc. Once they reached about a year old, the hormonal behavior started fading away again.
If you're adamant about handling your rabbit, even if they don't "want" to, any gender will be just fine.
Ever since I've gotten into breeding, I can say that bucks are definitely more naturally attention-seeking. Some to the point that it's almost annoying, like a clingy dog. LOL So for pet owners seeking a quicker bond with their bunny, bucks are definitely going to be more fitting.
My does have never been like that, naturally...I do have those that will seek attention, but only because of how I raised them.
I guess what I'm trying to say is that it depends a lot on your own personality and how you care for and handle your animals. Both genders can make great pets, even unaltered, but it depends on what your expectations are. In general, does take a lot of work to bring them to the point of interaction that seems to come naturally for bucks.