Well he was the one who broke up with him five years ago.

He shakes his head as Raleigh follows him, but he opens the passenger door for him.
"I want to help you," he says, irritated. "I want to be here for you."

And he can't do that if Raleigh's going to resist any help.

....and he misses him. But he just made it pretty clear that he's not going anywhere, so he's not even going there.