Its quite normal for people from a previous immigration wave to be hostile or even openly racist towards those in the next. Patel and Braverman are classic examples.
I think accusations of racism might be an exaggeration. Lots of people are selfish arseholes, some of them happen to have brown skin.
A white Eton educated Tory politician might well strongly condemn white working-class trade unionists who are engaged in a dispute with an employer of South Asian heritage, but no one would be surprised, or accuse him of being racist. Except for people from far-right organisations who would more than likely express outrage and accuse him of racism.
So yeah, some people with brown skin have embraced Tory values of selfishness, looking after yourself, and **** everyone else.