A Probably too Sympathetic View of Gríma

[Encaitare]

I got the impression that Saruman was saying that sarcastically:

'Wormtongue!' called Frodo. 'You need not follow him. I know of no evil you have done to me. You can have rest and food here for a while, until you are stronger and can go your own ways.'

Wormtongue halted and looked back at him, half prepared to stay. Saruman turned. 'No evil?' he cackled. 'Oh no! Even when he sneaks out at night it is only to look at the stars.'

It seemed to me that Saruman was mocking what Frodo said, suggesting that when Grima sneaked out at night it was not at all to look at the stars, but to do eviller things. He's probably lying and just making a bogus claim to belittle Grima even further, like he does when he suggests that he ate Lotho... I hope that he was lying then, at the very least!