If you haven't already, I'd severely reduce the entrance to help the colony defend themselves. One centimetre wide is a good starting point. It sounds like they're being attacked by another colony / other colonies and wasps. When did you last inspect them & how many frames were they on?