Why do you think there is a spam filter in the first place? Email addresses are tremendously easy to obtain. It doesn't take a lot of effort to guess email addresses, plus there have been various online companies over the years who have either had their database stolen, or they've deliberately sold their database to spammers. This has been going on so long in emails that people have spent resources trying to automatically sort out spam from legitimate email.
So, it's no big deal if they have your email address, it's easy to get. What would be harder to do is to obtain your actual name from your email address. Sometimes it's easy to figure it out from the email, like for example, if your email address is email@example.com, then they'd know your name must be John Doe. However, if your email is firstname.lastname@example.org, then that's much harder to guess your name from it. And it's nearly impossible for automated routines to guess your name. If the phisher doesn't use your actual name, rather than just your email address, then that means that they don't actually know who you are.
So just leave it in the spam folder, because your email provider was clever enough to figure out that it was spam, so you didn't have to.