Basically all the Men scattered around Middle-earth who were not already completely under the yoke of Sauron (or Saruman, as is the case with the Dunlendings). And there must have been quite a lot of Men, not only in Dale but in other settlements, for example there were Men way up north around the ice bay of Forochel, and there must have been settlements to the south of the Shire with all that talk of 'squint-eyed Southerners'. Of course there were Rangers and remnants of Arnor.
But all these Men were not united in any kind of common cause against Sauron, they were fighting their own battles. The enemy must have known well about the concept of 'divide and conquer' - for example, send a few Orc bands up to harry those Men in the villages of Arnor and they'll be far too busy fighting for their own livelihoods to worry about Gondor facing an army 1,000 times as large.
Only Rohan really still had any formal bonds with Gondor, which was probably why Saruman was keen to destabilise what remained (which wasn't that much) of the old alliance.
Anyway, old Butterbur would have been a great asset in war. After all, everyone needs beer.
