At the very least you took the clear coat off your paint, most likely you did that and scratched the paint, possibly down to the primer (but unlikely). This is fixable to an extent but a little complicated, something like a polishing compound can meld the paint around the light scratches back into them and thereby fill them, but it is an abrasive compound so you'll have to make sure to use it only in the exact area you caused the marks, using it in other areas can remove more clear coat and make things worse. Start by polishing it into the swirls fairly lightly, let it dry and remove it and check your results; you can apply more pressure as needed but keep in mind what it is you're doing (applying an abrasive compound).
If you're feeling nervous about the polishing compound you can go a step down and try "step 1" of mother's brand waxes, it's a cleaner and Very slightly abrasive (much less than the polishing compound); I'd buff this Hard into and around the areas of the swirls as per the instructions and then, after removing, finishing it off with their final step carnuba wax ("step 3" of their system). You can of course use their step 2, but I don't think it will bring much to the party (I own and use all three steps on my cars).
Either way you'll never get the clear coat back unless you try spraying your own back on but I would never recommend that to anyone who hasnt experience spraying (even the can version).
Another option would be to take it to a car detailer and ask them what they can do, if they describe a process like the above (melding the surrounding paint into the marks via a slightly abrasive polish) you're probably in a good place.
Goes without saying to never use anything as rough as the backside of a kitchen scrubber on your paint again, a good purpose built soft bristle brush and elbow grease should handle all washing.
edit- this link has a good synopsis of compounds and polishes worth the read, I can not endorse any of the products they link to though, as I have never heard of them (and I've heard of plenty!)
Brands I'd trust for waxes and polishes would be Mothers (first and foremost) turtle wax for compounds and meguirs (large brand but I am not a fan)
also I'd do all of this by hand with a cotton cloth or applicator pad, so that you can have the most control on the pressure and where you're applying this stuff. A power buffer in the hands of someone not used to one can cause a lot of trouble. Good luck