In many cases, they are interchangable but the terms are used rather loosely at times.
In order to charge a battery, you need to have a charging voltage that is at least slightly higher than the voltage of a fully charged battery. Stand alone chargers that are intended for charging NiMH cells may have timers, peak voltage detectors, or temperature sensing circuits that will shut down the charger once the battery is full.
An adapter that plugs into the device that uses a rechargeable battery will not have the charge monitoring circuitry. Those functions will be handled by the device itself, i.e. a cell phone, laptop computer, MP3 player, or GPS. Those adapters are sometimes also called battery chargers eventhough they often provide power to the device and charge the battery at the same time.
I have a rechargeable flashlight that uses an adapter that is significantly higher than the battery voltage. It will not provide enough current to power the bulb. The charge controlling circuitry is in the flashlight so the charger is technically an adapter. There is switching arrangement that ensures that the bulb is never connected to the adapter so technically the adapter is a charger.
Clear as mud?