Well, the confederate flag is the symbol of defeated separatists who believed they should have their own nation to enslave people.
Is that racist? I don't think it's a stretch to interpret it that way, but I prefer to understand why someone is flying such a flag.
In my experience, the confederate flag is largely flown by redneck d-bags who think it's cool. If they're under 20 or so, there's a chance they'll grow out of it. If it's being flown by someone in their 30's or older, you can pretty much assume you've got an undereducated, racist hick on your hands.
Sometimes, of course, it comes up in the teaching of history or a discussion like this one. There, it's probably OK. It is, after all, a reality and it should be discussed. So in the context of a history class, I'd think it's reasonable.
So it's not as bad as flying a red/white/black swastika, but it's on the same ladder.