But of course it does. He would never tell Pelosi to "go back to where she came" from, nor any white Rep or Senator.
"Go back to X" is about the oldest one in the book, and has been applied to virtually anyone who isn't a white protestant of English ancestry. It's been used against immigrants from other western European countries, Irish Catholics, eastern European immigrants, people of Asian descent (especially the Japanese), black people, Muslims, etc.
It just happens to be that POC and Muslims are the conservative scapegoat du jour.