If there were substantial evidence to support the physical abuse that took place in your home, the courts would have at least required supervised visitation. The problem in most cases is that people don't document the physical abuse. They either don't report it or they don't file charges when it occurs. With little to nothing on record, the abuser suffers no loss of privilege. Sadly, the only thing the courts can go on is documented abuse. There must be proof or it didn't happen as far as they're concerned. Just make sure that you keep an eye on your kids for signs of abuse. It's really all that you can do at this point. If you do recognize any signs, be certain that they're documented. Take pictures, keep physician's reports, file police reports, etc. If he does abuse the kids you will be able to have the visitations stopped, or at least supervied. Good luck to you.