I saw Song of the South in theaters in 2 separate theatrical runs before it was swept under the rug. I remember it fondly from my youth, and as such I have followed this debate rather closely. I've come up with several conclusions that support the notion that the movie is not, in fact, racist. But there's many that believe it is, and with a number of factors leading to their "conclusion". For example, there are a lot of people out there who believe the term "*****" is offensive in any form. These people would suggest that any use of the term is offensive regardless of how or why the term is used, despite the fact that once upon a time the word was common, widespread, and accepted enough that even today the term is being brought back. The 2010 census even included the distinction right alongside black and African American, as they found there are still quite a few older people who prefer to identify with that particular term. Regardless, however, there are still quite a few who take offense to the term, and that remains one factor that leads people to think the movie is "racist".
Another factor is the erroneous notion that the movie took place during slavery. Just watching Uncle Remus calmly leave his home for Atlanta refutes that notion, but there's still a popular opinion that the movie is an attempt to "whitewash" slavery as a kind and gentle practice.
The movie does perpetuate several racially based stereotypes, but they're ideals that were never really considered to be racist in the past. For example, the ***** spirituals used in the film were pointed to as typecasting of blacks of the time. However, to suggest that this was stereotypical would also suggest that the notion of ***** spirituals is somehow racist. This notion has never even been suggested, as ***** spirituals have been accepted as a fond part of America's past, and the backbone of current American practices (such as gospel music).