The children weren't the problem, the parents were; parents who are over-concerned busybodies don't belong in upscale restaurants. My parents took me out to dinner when I was little, but I knew how to behave so I didn't scream or cry, and I am sure they would NEVER have told other diners what to discuss at their table!
I'm glad my parents took me to upscale restaurants; it's not like we went every week, more like once or twice a year, but I remember that I loved going, even at a very young age I liked seafood, French cuisine, all kinds of gourmet food much better than hamburgers, fries or pancakes (okay so I was a geeky kid, so what ;)). Don't take that pleasure away from a child because the parents don't know how to behave.
And if I ever meet this father, I would like to tell him that in my experience, it was the kids whose parents were *most* concerned with what they heard who were all excited about the 'forbidden fruits', who whispered the words in the corner of the playground and shared 'dirty' pictures. My parents weren't too concerned about overhearing a conversation like this, of course they didn't talk about everything right in front of me but when I did overhear a word they just said something like 'that's adult stuff' and didn't make a big deal out of it. I am convinced it's because of this that when the 'forbidden fruits' were going around, me & the classmates whose parents weren't so uptight just shrugged and weren't interested at all, or at least not until many years later when our hormones caught up and we finally noticed the boys.