I missed much of this as I think I'm a tad younger than one or two of you. However, another vote from the 'it's not just physical violence' camp from me, although I'm sure I've been through much less than many.
In primary school we had practice papers for SATs. There was a small group that the teacher was coaching to practice the harder level six papers at lunchtimes, unlike the rest of us who were on the level five ones in normal lesson times. I heard and asked if I could try the level six papers and was told I wouldn't be able to join the group because the papers were too hard. So I asked if I could take one to try at home just to practice, at which I was told it wasn't allowed (she'd let the others take one home IIRC). I asked why I couldn't and was told (this wording) 'because you're not clever enough'.
That stuck with me. I'm not sure how much it has affected me but I know it affected me less than it might- when the SATs results came out, none of the special group had achieved any sixes, only three people had got straight fives. Two were girls from the special group- and the other was me. Which showed me the teacher was neither nice nor correct- those in authority are not to be automatically believed. The injustice stuck with me but also shaped me into someone who, I hope, will never be like that teacher.
I think the same teacher also separately told me off for (correctly) commenting on one of the practice level five exams that the question was factually inaccurate...
All fairly mild I'm sure, but mattered to me.