First off a couple of questions need to be answered and this will help you determine if it's a good idea.
Why do you want a rabbitry?
Will your rabbits be for show, pet, or another use? There has to be an objective to breeding rabbits. It is irresponsible to just do so because you feel like it or because you want to try and make a little money
How many rabbits are you able to keep?
How many breeds do you want to raise (this kind of ties in with how many rabbits can you keep. Most great rabbitries only focus on one or two breeds. If you have more than that then the quality of rabbits will not be the same and as competitive between all breeds)
Can you handle death? (this may seem like an abrupt and scary question but you must be prepared for any one of your bunnies or babies to pass because it happens more than anyone would like)
What would be the short and long term goals of your rabbitry?