It's a fashion that swings back and forth. I used to love the idea but I don't like it anymore for all the reasons you've stated. At any given time people will have different preferences for their own reasons. Personally, the first thing to give in my transition away from open plan living was the kitchen. It's blissful to either shut oneself in there or to shut the door and not worry about the dishes yet to be done. I've taken down many walls for people, but secretly I like putting walls back, maybe not in the exact same place, each age has its own needs of a building. There are ways to find good compromises between compartmentalisation and flexibility from sliding doors, to partition curtains, to glass walls that allow light to pass through.