Why is egg considered non-veg food?

You are not killing any living organism.If egg can be considered non-veg then why not milk also as it comes from animal body.What is the logic for it to be considered non-veg.Correct me if I am wrong.
Please don't be rude.
Update: most vegetarian people drink milk.
Update 2: the embryo is never developed in case of table eggs.
38 answers 38