Well, as a smoker, 'the halflings' leaf' does on occasion 'slow the mind', especially when in need of sleep or when stressed; a nice smoke works wonders! It also invigorates though, as the first smoke of the day proves. So I can see what Saruman might mean when he says it slows the mind. And then there is the addiction factor - `I can only go so long without tobacco and I start to get not only exceedingly grumpy, but I can't think straight. A very good reason not to start on the habit, by the way.
But if Gandalf was having a nicotine craving then he would indeed find that his mind was slowed as he would probably be thinking of nothing else apart from his next smoke! This happens in Moria when Gandalf is feeling mental confusion and has to stop and have a smoke!
H-I's quotes are good. The second one appears to set Saruman up as an anti-smoking wizard, while the first shows how he had a secret addiction to the substance; this made me laugh, thinking of him trying to hide the fact that he smoked, keeping a can of air freshener by the back door to Orthanc, with which he would spray himself and hope to hide the smell of smoke!
I wonder if he also carried lots of packets of polo mints?