The simple answer is: "To Zero: To adjust a firearm's sights so that the point of aim is the point of impact."
Trouble is, most firearms don't hit in EXACTLY the same place...they hit close, but not exactly. The variation of points of impact from the same barrel from exactly the same position (such as benchrested in a vise) is expressed in minutes of arc (MOA).
The more accurate a particular firearm is, the smaller the MOA number will be...and generally, the more expensive the firearm is...rising almost logarithmically the smaller the fraction of MOA goes.
So a "zero" is technically the closest "point of aim" that coincides with a "point of impact".
But wait, there's more.
The military has two distinct zero determinations: Mechanical Zero and Battlesight Zero.
Mechanical Zero is the sights, mechanically set to their exact centers. Mechanical Zero is RARELY, if EVER, the same as the Battlesight Zero.
The Battlesight Zero is where those sights are adjusted FROM Mechanical Zero in order to coincide the point of aim to the point of impact. Battlesight Zero is the same as "Zeroing" the sights, and is abbreviated BSZ.
But wait, there's still more. Because the actual flight of the projectile from a firearm is an ARC, the point of impact CHANGES at different distances. So in the process of zeroing your sights, your point of impact coinciding with your point of aim may only occur at one, or at most two ranges.
That's where you get the cryptic phrase, "Zeroed at 100 yards."
It means that at 100 yards, you put the sights on the bullseye, the bullet hits the bullseye. At 250 yards, you put the sights on the bullseye, the bullet may hit at the bottom of the black on the target.
This is why "flatter" shooting rifles are easier to zero: Because, with higher velocities, gravity has less time to act upon the projectile in flight, and thus, the arc is less pronounced. This translates to less of a variation in vertical points of impact at different distances from point of aim. Less of a variation, but they do still exist.
Finally, the last variation for zero is called parallax, which is actually the error of the nature of optical viewing caused by different distances in perspective.
Parallax occurs when something in between what you're looking at, and where you are, appears to move in relation to that what you're looking at in the distance based upon your own position.
Let me try that again: In the case of a rifle scope, it means that when you look through your scope, and you move your eye just a little bit to the left or right, your crosshairs seem to move off-target just a little bit in the opposite direction. THIS is parallax. It is caused by your crosshairs not quite being on the same sight plane as the target as seen through your objective.
What this means to your zeroing is that if your parallax is out of adjustment, you can zero at a range, and then wonder why you're still not hitting to point of aim. This could be a problem with scopes that "refuse to zero", although it is not the most common.
The most common "refuse to zero" issue I've seen with scopes is the mount. Everything from something that doesn't align properly to something that isn't the right size/isn't the right model for the rifle or the scope, to something that just shoots loose from the recoil after a couple hundred rounds.
A telescopic sight should be ROCK SOLID to the rest of the gun, preferably attached directly to the receiver. Many zeroing problems can be solved by simply going through good scope mounting techniques, long before the covers come off the adjustment screws.