My first instinct was to say that the rings were with Sauron, what made me change my mind was Gandalf's words at The Council, some wise people are present and none contradict the statement that Gandalf makes
, however I will concede that in other parts of LotRs this is contradicted, if this is so it is Tolkien to blame not Gandalf.