After but a few generations in the Fields Men Know, the offspring of elves are born smaller, short-lived, and eventually twisted.
|... and after.|
Elves don't choose to linger in mortal lands by preference of course, but those few who do appear in the mundane world are exiles -- driven from the undying kingdom after a failed coup -- or on quests for their ageless lords.
Some elves attempt to stave off the decline of their bloodlines by switching their infants for those of mortals -- and thus a "half-elf" or "halfling" child might be raised among humans.
This idea doesn't fit in any of my three current campaigns ... but I bet I'll get some mileage out of it at some point.