I had a neighbor like that once. Let me tell you about it. I listened to my music, but not loud. They complained all the time. Even called the cops. To demonstrate how loud my music was they went into my neighbors bedroom, which was against my living room. The cops heard how loud the music was. While one stayed there the other went to my door. Upon inviting the cop in to see how loud the music was, he had to comment that it was not loud at all. The problem was with the construction of the apartments, the walls offered virtually no soundproofing at all. Once we all understood the nature of the problem we made concessions and got along much better.
It may be that you are not unreasonably loud, but circumstances can make matters worse. Instead of fighting with your neighbor, try to understand how bad the noise is in their apartment. This may help you understand the nature of the problem.
Something I've noticed is that when some people walk, they hit hard with their heel and make quite a boom and don't even realize it. To people down below the sound is amplified. My first wife had a habbit of speaking quite loudly and not even realize it while I've always spoke with just enough volume to be heard by a small group to whom I was speaking with.
There are many things we do and don't even realize it. I doubt your neighbor is complaining just to complain. There must be something to it. Try to work it out with them. If all else fails, concider moving. Maybe even to move to the first floor.
ADDITIONAL: Wow ! ! ! There are a lot of hostile answers below. A few good answers, but I would advise taking a kinder approach. Starting a war only means casualties. If you make bitter enemies of them then they may "Put a subwoofer" against the celing and drive you crazy.
It never works to try to out do the other person. You may win the battle but at a very high cost. It could end up in court and someone could be fined. Since you are the accused of being too noisy and they've filed other complaints YOU WILL LOOSE. Don't fight with them, it's a no win situation.