Meh, probably. It's subconcious though, I've always done it.
I should mention that I'm not grabbing across my chest, I'm grabbing on the same side. So it's not quite the same as the "MY boobies, not yours!" thing that I think the earlier posters were mentioning.