Good points made by Btiz. Also, we fly barometric altitudes. You may
have a non-standard lapse rate and the pressure up high in the flight
levels is something different than what the altimeter expects. After
all, the local altimeter setting you are using is on the ground, you
are 6 miles ABOVE that. You may not have all the air (or may have more)
above you than the altimeter expects.