The Dwarves

[Dain]

Dain didn't want to "get rid of them" when he let Balin go to Moria, he just knew he couldn't stop them. All dwarves dreamed of re-opening Khazad-Dum, and Dain couldn't hold them back.

As for Dwarven settlements at the time of LotR, I believe there were Dwarves in the Blue Mountains (not Nogrod or Belegost), Erebor and the Iron Hills.
The Blue Mountains, where Thorin stay while in "exile," wouldn't have been abandoned when he reclaimed the Lonely Mountain--there were dwarves there before his people fled there, and they would remain after he left. Same with the Iron Hills. Dwarves would have come to Erebor from other places, and I think the excitement of regaining their old kingdom could have spurred a little population growth, too. Dwarves are always looking for new places to mine, and never forget about old places they have lost. After their success with Erebor, I think they would have caught the exploring bug again...

In the fourth age, we have dwarves spreading again, to Aglarond and to Moria. They eventually disappeared, but I think Dwarves had a bit of a recovery in the 4th age.