What happens is that the oil is high enough that the crankshaft whips the oil so that it gets air in it. Not much good for lubricating your bearings properly. Also, it often gets on the cylinder walls, making it hard for your rings to seal. (that's where the smoke is coming from)
An oil change with the proper amount added would fix it, but you should be able to just drain off enough to bring the level back down to where it should be. The air in the oil will work itself out in short order. Just let it idle for the first few minutes afterwards. You can leave the old filter on whether you change the oil, or just drain some.
Hopefully, if you didn't drive it too hard or for too long like this, the other stuff I mentioned will be OK. Even then, you may have fouled your spark plugs, and if it doesn't start running better soon, you may have to have them replaced or cleaned.
ASE Certified Tech