Toudou's "miracle" was directly due to information provided by Lelouch when he was young. Further, Toudou knew Lelouch when he was in Japanese custody, and would know that Lelouch hates Britannia more than anything else in the world He would know that he wouldn't be the type to use them for a bid for ascension to the throne, but rather to make real his dream of destroying his homeland. Toudou has also shown that he KNOWS that something isn't right with Zero, and that his motives are impure. But Toudou doesn't care, because as long as Japan is freed in the process he is happy, no matter what really goes on. He is pragmatic enough to realize that the recording of the conversation that will be leaked to the Knights in episode 19 effectively changes nothing, and it could be argued that Toudou would understand the true nature of the conversation between Lelouch and Suzaku because he knew both of them personally.
Toudou is the only person among the ranks of the Black Knights whom I could see not turning on Lelouch. China might also support him considering the fact that Xingke thought he had a good estimation of Zero after Zero decided against having the Tanzi married off.
Everyone else though, Ogi especially, would be more than happy to ditch him.