It's not just a panic response, it's an instinctual response. When deer and many other potential prey animals detect something coming at them, they take off running in a random direction, expecting for the "something" to be a predator chasing it. Most of these potential prey species have eyes on opposing sides of the head, making depth perception very hard, but it widens their view so they can spot that potential predator better. This means that deer can't tell how fast a car is going, just that it is approaching. However, cars stay on a certain path (hopefully they stay on the road), and deer can't understand this. Therefor, when a deer sees a car moving fast, they take off in a random direction thinking it may be something trying to run them down and eat them, and sometimes that random direction happens to be in the path of the car.