I completely agree with you! All that psychological stuff is encoded in our DNA somewhere. I've thought about this as well.
The way I see it - Physical traits as well as traits of the mind are controlled by natural selection.
All psychological things are the result of two different factors. These two factors, genetics and environment, are what create the consciousness. Genetics are the blue prints of evolution. The environment is in the driver seat.
For example - If you were born in an Amazon jungle with the same genetics you carry today, you would only recognize yourself physically.
Here's the curious thing...
As the population grows, the weak (lesser evolved or dumb) gain protection. This allows bad traits to sneak by natural selection. Just like how herd immunity protects the non-vaccinated population from harmful infections.
This would explain why we see "dumb" people in our world today. The reality is that "dumb" is an opinion. Opinions are created by the consciousness. Consciousness is influenced by the environment and it evolves.
The fear is that the higher evolved will notice a need to decrease breeding. A type of environmental stress on evolution that somehow misses the lesser evolved. Instead, they continue pumping out kids like there's a farm to run...Eventually they over populate and evolution runs backwards.
Similar to how technology has allowed us to live the lives we have today. Without it, We wouldn't find time to be fat and lazy ;)
Interesting to think about at any rate!