Geoff, the vet that comments on here sometimes, seems to prefer a hay and veggie diet for rabbits. I think, because of all the problems he's encountered treating rabbits that are fed pellets. I've read of other rabbit vets suggesting it as well. I like to feed pellets myself, just for the convenience of ensuring my rabbits get the needed nutrition, but I've encountered enough problems feeding pellets that I feed a very limited amount and am even considering a no pellet diet. I do have one rabbit that can't have pellets, and he's done well on hay and veggies. I do supplement him with a little alfalfa, to help keep his weight up. With your bunny being a young Flemish, it would be good to continue with feeding the alfalfa hay, if you decide to stop pellets, so he's getting enough protein and calcium for growing.
If you are reluctant to stop pellets, you could make a few diet changes to see if it helps get his poop looking better. Your bun might be getting too many sugars in his diet, and that could be causing too much of a digestive slowdown, and the irregular looking poop. Try cutting out the banana and instead of free feeding pellets, feed limited so that he will have to eat more hay and will be getting more fiber, which should help with the poop. When my bunnies were babies, I found that feeding a limited amount of pellets worked better than unlimited, and helped keep them from getting mushy poop or small poop. I fed twice a day, and enough that would last them most of the day but they would run out about 3 hours before the next feeding time, then they would eat hay until it was time to feed them again. You could try that and see how it works.
This is a good article about a digestive slowdown if you want to take a look.
http://rabbit.org/sluggish-motility-in-the-gastrointestinal-tract-2/
ETA: Here's this on a pellet free diet. Though it's not terribly informative.
http://rabbit.org/pellet-free-diet/