I would think that "the Elves that remained in Middle-Earth" refers to those who did not depart to Valinor after the destruction of Beleriand. This would include Galadriel but not Glorfindel.
As for the "fairest" part, go ask Gimli and Eomer who is more beautiful.
