My take on it is that it is the magic in the world that fades, and with that the long livety of the elves. That means that they either leave for the undying lands, finds a focal point where there still is magic, dies, or slowly fades away into spirits.
Just that in the early 4th age chronicle I am sketching on, it isn't natural, but engineered.