Looking at this as a pessimist, you should always get the other driver's details.
Firstly, as above, there is the possibility of serious damage that you can't see from the outside. Unless you're familiar with car bodywork, you may not notice the extent of any problems.
Secondly, and this happens far too often, there's always the possibility that the other driver made a note of your registration number & will claim against you as a hit & run - you'll have no defence as you don't have his details to report the incident first.
Sorry, but you should always exchange details.