Their relationship is odd to say the least. Complicated rivals isn't exactly correct, but might just be the best way to explain it. So similar, yet so different. The two of them had seemingly gotten off on the wrong foot, which kick started their little petty rivalry. Not that Yagami doesn't get a kick out of it. He'd be lying if he said it wasn't kind of fun.
That said, even though they have their excessive fair share of bickering, Yagami somewhat enjoys talking to him. Though playing it off as teasing, or not entirely genuine, he's oddly fascinated with him.