It was obvious, at least during Roger's era, he didn't fight and didn't need to use his strength for any of his road poneglyphs while Luffy has done so for 2/3 and his equal pirate rival had no intention on becoming PK while Shikki lost because of the weather (although he admits inferiority to Roger).