Asians makes everything serious business. People identify other people they don't know as bad stereotypes. Everyone thought Tomoya was a delinquent because he skipped classes, were always late, no club activities, didn't pay attention in class as well as hanging out with Sunohara who everyone knew was a troublemaker. Since Tomoya rarely made any effort to make new friends, more people started to avoid him and didn't get to know the real him. As for the otaku case: the general otaku stereotype is a gross guy with bad hygiene that watches anime and collects figurines/posters and masturbates to doujinshi. If a girl like Haruka were to be identified as a otaku like that, people would start getting the wrong idea about her.