One of my players once decided to play a double agent himself and i would handle any NPC the same way, meaning, whenever you absolutely have to make a dice roll, just also roll for the NPC (or let the Player roll) so that your (other) players don't get suspicious of him or her.
Apart from that I'd strongly suggest to limit dice rolls to a minimum in the actual situation. "walks through walls" has already given you some handy advice.
Generally speaking, though, you should create a way of gaming in which dice rolls do not automatically indicate that "something is happening" or players just failed to notice that "something that is happening just now but you don't have any clue about". Personally, i tend to reward well placed perception or insight tests as i usually drop some hints that players can catch. Should they get all paranoid however and try to overdo that, I feed them misleading "clues" in their growing paranoia. That combination has played out quite well so far.
Hope i could help.