I think it happend during Reaper Man
Death is not invisible. Most people just refuse to acknowledge him for who he is, unless he insists. Under normal circumstances, only those of a magical disposition (e.g. witches and wizards), children and cats can see him, or allow themselves to see him. In Wyrd Sisters Death briefly took the place of an actor playing him in a play and was shocked to discover when he walked onto the stage that all of the audience could see him as they were expecting Death to appear; it was stated that he was quite nervous at this as he is usually seen only by one person at a time and was not comfortable with so many people watching him. Death can of course ignore things like walls or magic spells that stand between him and his object: this is because he's much "realer" than they are. A castle might stand for centuries, but Death has existed for billions of years: to him, the walls of the castle are less substantial than a cobweb. However, he can only go where people can die, as shown in Hogfather, and can only see people who can die, as shown in Thief of Time.