I have heard that one all my life. Even when I was in elementary school I had other children come up to me and declare "You talk white," (because apparently speaking proper English is something that only white Americans do) and "Are you really black?" (In case you can't tell from my avatar, my skin is dark enough that there is no mistaking the fact that I'm black.)
In high school it was "Oh, you're an Oreo, you're a snowflake, you think you're white," and my personal favorite "why do you listen to that 'white music'?" Just because my music of choice happened to run toward Nirvana and Nine Inch Nails and not rap. (For the record, I never got "You act white" from white people. From them I'd get "Oh, you're not like other black girls." Or, "You know, I don't even think of you as black.") As far as whether or not it ever bothered me, I can't say that it did. It just meant that I learned at a very early age that the most important thing was to be myself and that anyone who didn't accept me for being genuine wasn't worth my time.
EDIT: I agree that the worst part of it is that people seem to say it when one displays good manners, a decent upbringing and speaks in an educated manner. Sadly, (and ignorantly, I might add) the implication is always that such an individual is rare if he or she happens to be black. What does this say about them? And what does this say about "black culture" as it's perceived in the US? Every time I think about it, it saddens me.