While a woman not to trifle with, she IS a lot less likely to shoot you than most of the other women in the show.
If you DONT go to her office willingly she'll be sure to send a few of her henchmen to FORCE you to go, THEN she might be in a bad mood. Or she might ask Revy to go after you. In which case youre totally fucked.
While no absolute guarantee of survival, I'd say your chances of living are significantly greater if you accept her invitation.