For most songbirds, there is a good rule of thumb: the rule of the thumb! By that I mean, can the bird sit up on your thumb (or finger) on its own? If it can, then it is a FLEDGLING and should be left alone! Nestling songbirds cannot balance on their own or grip a perch until right at fledging. If the bird can balance ok, then it is SUPPOSED to be out of the nest. It may not look like it, but it is. It will have some feathers over much (but not all) of its body, and the wing and sometimes the head feathers will be sticking out of little tubes (the feather sheathes). It might still have tufts of down on its head or on other parts of the body.
So, if it seems ok, not injured, just unable to fly, and it can perch on its own, you should PUT IT BACK WHERE YOU FOUND IT. Even if the parents weren't right there yelling at you, chances are that they know where the baby was and were doing all they could to take care of it. Don't worry about them abandoning it because you touched it; birds don't do that. (They might, however, abandon nests if you get too close, especially if they have eggs and not nestlings.) Just get it back to where they can find it and where it will be safe. Don't put it in an enclosed area that the parents will be afraid to go into. Get it somewhere it can eventually move off on its own