Rexes are supposed to be better for allergies. I think that since their fur is shorter, the dander has less to hang on to.
While angora's shed less, there is more fur for the dander. You can blow out the coat, but that spreads the dander around that area you are doing it. So the actual rabbit might be better, the grooming is not. To prevent matting, they need to be groomed several times a week.
I find that I am better with the longer haired breeds and the short fur gets into my nose and clothes while the longer fur sits on top of my clothes. The dander doesn't really bother me though.
Unless you have specifically been tested for an allergy to rabbit dander/fur/saliva, you could be allergic to something else. People can be allergic to the hay (especially dusty hay), bedding (wood shavings), food, or even the cleaning products. There are ways to minimize these and still have rabbits. A good air filter, using gloves, keeping the hay contained, using different bedding can help.