"Drive safely" is the formally correct phrase.
Saying "drive safe" sounds casual and informal; however, many people do it. This is because, in general, people sometimes use the adjective form as an adverb (usually this means not adding -ly) in casual speech. It is not recommended in any formal situations.
"Do good" is a different kind of issue, because the form depends on the meaning you want to convey.
If "do good" means "do the right/good thing", then "do good" is the formally correct phrase.
If "do good" means "perform correctly/at a high level", then "do well" is the formally correct phrase. (But, as mentioned above, casually you could also say "do good" here.)