The only mention at all of the five wizards in 
The Lord of the Rings is when Saruman screams out:
Yes,  when you also have the Keys of Barad-dûr itself, I suppose; and the  crowns of the seven kings, and the rods of the Five Wizards, and have  purchased yourself a pair of boots many times larger than those you wear  now.
All material in 
The Silmarillion, Unfinished Tales, and  other later books edited by Christopher Tolkien was not published by  Tolkien in his life time.
Personally, I don’t 
believe any of it. It is all just a story. I was simply pointing out what Tolkien himself wrote in a proposed index to 
The Lord of the Rings  written to be included in the third volume (then not yet published)  “until it became clear that size and cost were ruinous.” I believe that  Tolkien meant what he wrote when he wrote it. Nothing more.
But let everyone believe in this matter as they think best, or not.