I think it was more that he wanted to stay close to the Sibyllae, otherwise he would've had to leave the whole way of life behind. Wasn't a great deal of his characterization that he never really abandoned the mindset of a Sibylla? Not that he wanted to stay close to a(n old?) lover.
..And he's not that out of place as a man, come on. Nice, motherly guys are guys too.