He's not extra vulnerable to damage, he just can't transform. In other words, he has no way to escape a hit, unlike the majority of Logia types that just transform into their element in order to prevent damage to their physical bodies.
In all honesty, though, if he can draw ANYTHING in with his gravity powers, even logia-types in elemental form, and then neutralize their powers when they touch his hand, then he is undeniably hax.
Also, to the guy saying Enel's weakness was gold, consider this: He made the entire Arc out of gold because he can move freely through it and get to any part of the ship instantaneously. His ability to mitigate damage by transforming was completely shut down by Luffy's rubber, hence how Luffy could hit him with the giant gold ball without him just phasing through it.