"Fuck them" is not helpful a response (to the self and others). It's clear that the so-called victims of racism often hold similar tribal/racist tendencies. It's more important to examine why.
The argument is not against collective punishments/rewards but how those punishments and rewards should be distributed to right past injustices. (I am against that view myself).
It makes sense when you view it through group conflict and see the world as being comprised of the power dynamics of groups exclusively. Tribalism is (I think) part of human nature.
I think this is the ideology of nation-states, and before that of kings and queens. It's helpful to think of what a farmer in Provence thought was "my group" in say, the 14th century.
I would think extended family and maybe even local community/village/tribe, that is the vast majority of human existence and evolutionary history as a whole. Nations are pretty new.