I don't think it will affect the babies or is urgent. If it were abscesses that had burst, the pus is usually thick, white/cream colored, and cottage-cheese or toothpaste-consistency. The inside of the abscess will have very bright red skin.
Many bunnies have a bit of matted fur on the chin from their under chin scent glands. Did the fur fall out or did you shave her chin? I would clip the fur off her chin with clippers and clean the skin there to get a good look at it. An abscess under the skin that had not yet burst would feel different from a normal chin, and if it is a burst abscess, clipping could let you see if there is a hole that the infection came out of.