[The Might]

I guess they were able to live well under the Reunited Kingdom. Just as the Shire, the Druadan Forest was part of the kingdom, but still was off-limits for any outsiders.
Now that they had been left alone and that no more Orcs would trouble them I see no particular reason why they wouldn't be able to thrive.

I however agree with Lalwende's idea. Definitely a loss of their natural habitat would have a severe blow to the Druedain. We know they would have been able to live in a more developed society such as the Gondorian one, since some of their ancestors lived in Numenor for a while, however it was in the forest that they felt most at home and that's where they belong.

They clearly had special powers considering the essay in the Unfinished Tales and Ghan-buri-ghan's affirmations in LOTR. I guess that as long as no exterior factors would come to destroy this balance, nothing could really go wrong.

Lastly, there is one other idea I have. Tolkien hinted on several occasions that the appearence of a "new shadow" over Gondor would eventually take place. If matters would get worse, I am sure the Wild Men would leave the forest, just as they left Numenor when they started feeling that a disaster is no longer avoidable. If they were to settle in a new area I would think they would head westward searching for a safer place to live and perhaps meeting their brethren living in the Druwaith Iaur eventually.