There is no reliable way to tell if one species is the ancestor or descendant of another species. Nevertheless, often times two species are so similar that they can be considered close relatives. Stratigraphy (the relative age of the fossils) can be an important clue, although disappointingly the cladistic fundamentalists often totally ignore stratigraphy when they try to figure out evolutionary relationships. For example, the cladistic fundamentalists claim that birds evolved from a bird-like dinosaur. However the most bird-like theropod dinosaurs are found in rock layers that are 20-80 million years AFTER Archaeopteryx, until now considered the oldest known bird. As we go back in time, these theropod dinosaurs are not bird-like at all. Therefore people who are not religious like the cladistic fundamentalists think that these bird-like dinosaurs, such as Deinonychus, probably evolved their similarity to birds independently of birds. Scientists call such independent evolution convergence or convergent evolution. Indeed some people even suggest that these bird-like dinosaurs are not dinosaurs at all, but flightless birds. Personally I would need more evidence (such as feathers) to accept the claim that they are indeed flightless birds.
So, one important line of evidence in theorizing about evolutionary relationships is stratigraphy, as supposed ancestors simply cannot live later than supposed descendants. Using stratigraphy, we therefore conclude that modern humans (H. sapiens) cannot be ancestral to H. erectus, an archaic species of humans that evolved 1.8 million years ago, whereas modern humans do not appear in the fossil record until 150,000-200,000 years ago. H. erectus is so similar to us that many scientists think that we evolved from it. Indeed it is highly unlikely that we and H. erectus are only convergently similar since all the similarities are unique among hominids. H. erectus differs from modern humans in having a smaller brain, heavier brow ridges, and a protruding jaw line. Modern humans OTOH have a flatter face. Nevertheless, it remains a possibility that we did not actually evolve from H. erectus and we may have evolved from an as yet unknown species of archaic human. Since DNA evidence is rarely found in fossils, especially those that are more than a few million years old, it cannot be used to establish ancestor-descendant relationships. We must rely instead on stratigraphy, uniquely shared similarities and also on overall similarity. Overall similarity can sometimes be unreliable. Take for example, the thylacine or marsupial wolf. It is so similar to the placental wolf that if we rely on overall similarity then we may mistake it as yet another placental mammal, but since it is a marsupial, it simply cannot be a close relative of the gray wolf.
Indeed, many scientists have criticized the cladistic fundamentalists for their reliance on overall similarity or phenetics when establishing evolutionary relationships. If we rely on unique similarities, we can look for species that have feathers to figure out if they can be the ancestor of birds. In AD2000, a group of scientists discovered that a fossil found in the 1970s and housed in a museum in Russia, actually has feathers that share unique similarities with bird feathers. Because it lived 75 million years before Archaepteryx, stratigraphy makes it a more likely ancestor of birds than those bird-like dinosaurs that lived 20-80 million years after Archaeopteryx. To me, the most likely ancestor of birds is Longisquama insignis, the only species of reptiles that has been found with feathers. that share unique anatomical details with bird feathers. Nevertheless, Longisquama may also be an evolutionary deadend, meaning it became extinct without leaving any descendants. It is possible that Longisquama may have only been a close relative of the actual ancestor of birds. What we need is more fossils, and the more we have the better. We can then study them all and try to decide which one is the most likely ancestor of birds. Since there is no other reptile that has been found with feathers, Longisquama is the most likely ancestor of birds according to the available evidence.