Visible light covers less than two octaves of the EM spectrum, so we certainly don't see the world as it is, purely on the basis of failing to experience most of it at all. What we can say is that what we see has a strong correlation to certain aspects of an independent external world.
What red "is like" for us is rooted in the physical mechanism by which we detect red and the mental interpretation and experience fabricated in our brains. A certain shade of red can be independently defined by a spectral response but our experience is essentially personal. Obviously there is strong agreement about what red is like among humans because we all share a very similar mechanism.
The idea of colour blindness more or less assumes normal human vision is correct, and it certainly has greater resolution of frequency. But in principle we can imagine creatures that see across five octaves of visible light using twenty types of colour receptors in their eyes. To them, we would all be colour blind as a species, much like we regard the vision of dogs. And it is unlikely their experience or red (quale) would be like ours.
But what the world "is really like" is probably a meaningless or oxymoronic idea because "being like" tacitly assumes a protagonist who can experience what things are like, while the phrase "is really like" is searching for an external independence of reality.