I hear soundless voices too, unless I've heard a memorable voice actor voice the part in an anime.
For example, when I read the AMS manga, I very distinctly 'hear' Kikuko Inoue as Belldandy and Aya Hisakawa as Skuld. Yumi Toma's voice doesn't leave the same impression on me as the other two, so I don't read in her voice for Urd.
One of the few characters I've ever assigned a voice to without having seen in an anime is Sakurako Tenmaku from 'Aikora'. I always read her as Yui Horie for some reason.