To my understanding, basically, the small sword was a status symbol of rank for Samurai. It showed that they were servants of the emperor or in later cases the Shogun and his representatives. Technically, a samurai's life was at the mercy of his daimyo or warlord. The daimyo can demand the samurai take his own life at anytime by committing seppuku or ritual suicide to show his devotion to his daimyo. In exchange for this, a Samurai held certain privileges in Japanese civilian society and pretty much got away with anything short of murder because their special status allows it. The two swords was a badge of first class citizenship. Hence only the Samurai were allowed to carry two swords as a symbol of their way of life, the longer one for fighting their enemies and the other shorter one for fighting indoors and also for taking their own life. After all, it is quite difficult to try to commit hara kiri with a katana, the shorter wakizashi is more convenient. They are also forbidden from entering a Daimyo's house or be in the presence of the Shogun or the Emperor while carrying a katana, the wakizashi however, is allowed due to their status as Samurai, so this can also serve as an indoor weapon. Think of it this way, the difference between a commissioned officer and an enlisted infantry soldier or a private in the modern infantry army today is that the officer is usually issued a pistol and a rifle while the ordinary grunts are usually just issued a rifle. When an officer enters an office building, he usually doesn't go around the office carrying a rifle on his shoulders, but he can carry a sidearm strapped to a holster in his hip anytime indoors or outdoors.