I do volunteer outreach and advocacy for the homeless in my area. They go to the library to actually use it, yes. Sometimes that's also the only place they can go that is warm and dry (or cool on very hot days), has a toilet they can use and has a way for them to stay in contact with their families via email. Some cities with larger homeless populations, like mine, have other places to go for the same things, but they generally aren't open as many hours or days as our central library. When the homeless population in your city is, quite literally, well over 4,000 men, women and children, (and that was the conservative estimate based on our yearly street count back in January) there is no one place you can provide services for them all. So they spread out.
There are less visible problems that need attention. I know of two subsidized housing buildings that are brand new and over half empty. They are both high rise buildings and they are nearly empty because the city says they "can't afford" to subsidize the rent on the rest of the units. They seem to be ignoring the fact that getting anywhere from $40 a month (easily panhandled) all the way up to $300 a month (what people on Social Security would pay) for an otherwise empty unit is between $40 and $300 a month more than NOTHING. This needs to be rectified, as does the method of assigning housing in such developments.
I have one homeless woman who has been on the waiting list for six years now. They use a lottery system. You get on the list for subsidized housing by a lottery system, then when one becomes available they choose who gets it by a lottery system. So even if you're on the list, you may not get housing - ever. This is obviously a major problem. We've been trying to get that changed for years now and so far, no luck.
It's also very difficult for many of these people to find work if they are able to work. Most have no phone number or mailing address. Even if they were to get a job, they would miss the assigned shower times at some of the shelters and most of them don't know how or don't blend well enough to sneak onto the one of the college campuses and use the showers in the gym. Some of them have mental illness issues and it's fairly obvious and others have dependency issues. Most of the ones with dependency issues aren't currently using because they are aware it's dangerous to use drugs or alcohol while you're homeless. I'd say active drug users or alcoholics who still use while out on the streets is less than 10%. They're just very noticeable. Which gives a lot of people very wrong ideas. (other homeless folks don't like them for that very reason)
It's getting worse, too, because of the horrific recession we're in right now in the US. A lot of the people I've helped with survival gear and food lately have been people who were solidly middle class just 18-12 months ago. They have *no* survival skills whatsoever, so they have to be taught. I try to team them up with older, more experienced homeless folks, but I won't lie - I've actually done the "urban camping" thing a couple times and walked them through a few days. I cannot in good conscience just throw anyone to the wolves. Especially if they ended up in the wilds of a large, urban environment through no major fault of their own. After a few days their confidence improves, they learn how to move around the city easily and where to go for food, where to panhandle laundry money (might be why your homeless people stink - no one does a homeless person's laundry, they have to panhandle the money for the laudromat and some soap), which cops are cool and will tell them about occasional resources like Christmas to the Streets (summer even) or Stand Down (homeless veterans resource event) rather than "fixed" resources such as the shelters - which are full anyway.
I know it's disturbing to a lot of people to see homeless folks for a variety of reasons. Do keep one thing in mind, though. Almost none of these people were born homeless. The most important part of that entire sentence is the word "people". They are all human and deserving of our compassion and whatever help we can give that will get them back into a productive role in our society.
(and they really dig on it when you hand them bags of things like razors, deodorant, little shampoo bottles, snacks, etc. - so that's always fun and a nice thing to do)
I have odd "hobbies" and a deep sense of commitment and duty to my community.