Apple20 has got a very good answer.
I may add that much of other African countries in Southern and East Africa consist of high altitude plains, and isn't as hot all year round as you would imagine. And while South Africa may indeed be cooler than the African countries north of it, it is in fact sunnier here. South Africa is a colder but sunnier country than Nigeria, for example.
It is indeed true that most current South Africans (black and white) are of comparatively recent origin. The Khoisan was the original people of Southern Africa. But these people's original homeland straddled the equator in East Africa, so the warm=black generalization fails there..Also, it needs to be mentioned that black South Africans have comparatively light skins. This is not because they now live in the "cold", but probably because of some Khoisan ancestry.
The deterrent, if any, that may have caused the recent arrival of "blacks" was not likely to be cold, but less reliable rainfall than countries further up in Africa. This would have suited the nomadic lifestyle of the original Khoisan population better.