If they are going to mess up your game, then they won't help enough to offset their burden. People do have unconscious bias, and part of that is them going to judge you more harshly than someone who might be taller or appear older.
Hopefully, though, these people are professionals, and will judge you more on your qualifications or other attributes. There are a lot of unknowns you can't control for. Who else are you in competition with, and what do they bring to the table is the biggest. I once got a job, and was told later that the other cadidate was so close, that the only reason I was the one who got it over him, was because I dressed up more than he did for my interview.
Another factor you can't control for are the people doing the interviewing. They are going to get a vibe from candidates, they are going to carry individual personal biases, and they may already have a preference before the interview begins.
I'm not saying this to freak you out, just the opposite. If so many things are beyond your control, the best advice is to just relax as much a possible, so that you can present you in the best light you can.
If you can do heels without trouble, do them. If they are just going to distract you the whole time, lose them.