I would argue that, from the standpoint of being run down, it is safer to walk the railroad right of way.
As a rule, even though private property, being picked up for trespassing is unlikely, except for rail yards. These are always posted "no trespassing" and in most cases fenced, either in whole or in part. Around these yards is where you are most likely to run into difficulty with the unsavory folks found around and about. The yards are also the most dangerous places as far as moving equipment is concerned. Go around them.
You must keep your wits about you, however. Fijibabi speaks the truth about getting nailed on account of not being able to hear equipment approaching from the rear, especially when considering that locomotives don't always pull cars. They shove them also, and the cars sneaking up on you make very little noise.
The biggest danger is where there is impaired side clearance, such as where the road is cut into the side of a mountain or along river banks and levees, and always on a bridge or trestle. If you get caught on one of these bridges or trestles by someone, since they are also almost always posted "no trespassing," you are subject to arrest, but you maintain the right to remain silent.. If caught by a train, you'll not be arrested, but you will have the right to remain dead.
Wear boots with good ankle support to avoid a sprain on all the uneven ground and use a little common sense. Works every time, just like a de-rail.