Late in the season bees are likely to object to interference, so give them another chance on warm spring days next year. However, if they continue to be defensive, requeen before they produce drones that will pollute the locality and amplify the problem for others.
Did they swarm or supersede...