Assuming they have never learned sign language, they would have no experience of externally-established syntax. They might spontaneously establish their own, unique syntax (which seems very likely) and so their stream of consciousness would proceed in a way that adhered to that syntax - but it would not be sensed as words. It most likely would be purely visual.
In fact, we who can hear do most of our thinking in visual symbols. Many of us, after months and years of speaking, run a parallel process that provides verbal narration of what we are seeing in our minds as we think. The words are there just in case we want to share the visions with others - they are not the ideas we are thinking. Most of the flickering visions that make up our streams of consciousness are never translated into words, yet we "think" them nonetheless. Take away the standby word-track, and you probably have something very like what a person who is deaf from birth experiences, mentally.