I grew up there. The town was beautiful then, and even better now.
I preferred the Longshore area (as a kid) and spent a good deal of time around there. Just past the beach is a winding road to the top of the peak, tucked away, which overlooks the sound (about 20 houses up there - can't recall the name). This is where I would live, but if you are seeking more privacy and you/daughter aren't interested in the beach stuff, then look elsewhere.
Saugatuck shores is nice, but has had flooding issues. It is a nice tight nit community over there
There are massive estates on the gold coast if you continue north along the coast towards Fairfield/Southport. The old Freddie Couples and Phil Donahue neighborhood.
Also, there are private lots if you want to feel like a farmer like Paul Newman and various CEO's.
A good realtor will point you in the right direction. My mother was one for 25 years in Westport. There are specific areas that are clicky and exclusive to Wasps, Jews, etc. It may be a question you want answered.
The public schools are as good as it gets, right up there with the sub-elite private schools of the Northeast, IMO. I went to both, and enjoyed a fuller experience (more distractions) at Staples.
It really is a nice place to live. Hope to get back there in 6 or 7 more years. Good Luck!!!!!