I always liked his short stories compared to his longer stuff. The Rats In The Walls and The Shunned House.
To be honest though, Lovecraft was kind of a mediocre writer. He has some definite inspired moments, but really you just read him because what he talks about (ancient evil societies, indescribable horrors of twists flesh, etc.) are cool as hell.