No, it's based on what you look like.
I mean, when a Black and White person mix, the way the child will look is based on chance. It could come out a variety of ways, from dark to medium to light. There's a high likelihood that it will come out darker, because dark is dominant over light when it comes to skin color.
I've known people who were half-Japanese, half-White and looked White.
So it's not about what you are, it's what you look like. If people can't really tell just by looking at you, then you wouldn't know, now would you? I mean how can you be sure you don't have an Asian great-great grandfather either?
But if you know that you have a certain ethnicity, then it's good to mention it. I've met many Blonde girls who know they have some Native American blood in them, even though they look completely White.