It depends how you use it. Me and my family live in North Carolina, and we all have relatives who fought in the Confederate Army. We represent the rebel flag all the time, because we're proud southerners, we're proud of our ancestors who fought, etc. Also we disagree with the majority of what our wonderful government does now a days, so we like to consider ourselves rebels from them. Its also to represent our southern heritage.
We have plenty of black friends as well, and we all think Slavery was cruel and unnatural. We never use the Confederate Flag as anything racist. And I hate that people look at it like that.