Speaking from experience, always put safety, mental and physical health first.
You had the position before and found it stressful. Stress will only make you dislike what you do. Before you know it, you'll be forcing yourself out of bed to get to work, not enjoying it at all, and doing it...