If we go by the definition of non-vegetarian then it states those who eat flesh are non-vegetarians, the egg is indeed vegetarian as it has no flesh or life.
But Eggs come from chicken and they are not obtained by killing a chicken, like caviar or fish eggs are obtained. Also not each & every product that comes out of an animal is non-veg. the best example being milk.
The egg white is a suspension of the protein albumin in water. It contains no animal cell. Hence, egg whites are vegetarian.
Most of the yolk is a suspension of cholesterol, fat, & protein in water, but since the gamete cells cannot completely be demarcated from the yellow portion, the yolk is non-veg