Oden was getting little brothered by both of them, just accept it.
Roger swatted him aside to have such a clash with Whitebeard that, after witnessing it, Oden reconsidered the meaning of the word "strong".
Whitebeard literally called him little brother and traded him like a playing card to Roger.
Now if we're talking baby Oden vs Baby Roger or Whitebeard then fair play. He's got the feats.
That's all well and good, bro, but the version of Oden who witnessed Roger and Whitebeard clash wasn't even remotely as strong as his Wano version. And even that "weaker" version was already eating Prime Roger's attacks like they were nothing.
It's explicitly stated that he returned several times stronger, invincible, and triumphant, to the point that Kaido himself stated that Oden was on the same level as Roger and Whitebeard, both in terms of strength and character.
And as for Oden being passed around like a playing card, that doesn't change anything. Oden was originally recruited by Whitebeard, earned a place in his crew, and became a division commander there. Roger literally had to beg Whitebeard to lend him Oden, and the arrangement only lasted until Roger completed his final voyage.
How exactly is Roger supposed to make a temporary member, who wasn't even HIS, his right-hand man or anything like that in his crew? Does that really make any sense?
Let me see if I understand the logic:
Oden gets lent out by Whitebeard, joins Roger's crew, and is supposedly supposed to become the "Silver," Roger's right-hand man. Then, once the voyage is over, Roger just goes and returns his right-hand man to Whitebeard.
Wow.
The definition of "right-hand man" in this manga needs to be thrown out the window.