I just noticed that in Marineford, Mihawk commented that there is "never a dull moment" with Luffy:
"Never a dull moment with you... Straw Hat" rather implies that he may have been keeping an eye on Luffy as well.
And apparently he visited Shanks specifically to discuss Luffy:
So do you think Mihawk is interested in Luffy because of the encounter on Baratie, because he is Zoro's captain, because of the stories Shanks has told him, or because of Luffy's Plot-Plot No Mi? Or possibly a combination of some or all of these factors? And is he keeping an eye on Luffy beyond just normal interest in somebody causing the kind of ruckus Luffy is causing?

"Never a dull moment with you... Straw Hat" rather implies that he may have been keeping an eye on Luffy as well.
And apparently he visited Shanks specifically to discuss Luffy:

So do you think Mihawk is interested in Luffy because of the encounter on Baratie, because he is Zoro's captain, because of the stories Shanks has told him, or because of Luffy's Plot-Plot No Mi? Or possibly a combination of some or all of these factors? And is he keeping an eye on Luffy beyond just normal interest in somebody causing the kind of ruckus Luffy is causing?