Pretty much what Arimatthewdavies said. Try a soft cloth or tissue moistened with water first, or mild soap and water. If that doesn't dissolve the gunk, isopropyl or ethyl alcohol (rubbing alcohols from the pharmacy). CDs are the most prone to damage, especially if anything scratches the label side where there is little protection for the data layer, and can suffer from "bit rot" no matter how carefully they've been treated. DVDs in comparison are incredibly rugged; it takes gunk on the surface, deep scratches or actual cracks to cause playback (or ripping) errors.
Hundreds of ex-rental and public library DVDs, and handfuls of loose DVDs from the recycling yard.