Where is this idea Yamato is Kaido x Toki son.
But for me, its because Kaido might had a huge honor toward Oden. When we see toki ran away from the castle and said that prophecy, she might actually pregnant. And then later on, she gave Yamato birth and she might died not long after that. Knowing this, Kaido adopted Yamato as a sign of his respect toward Oden. And kaido might trained Yamato to be strong as Oden was.
So, Yamato might never know who is his real father and his real mother. Thats why he said to luffy in the text spoilers "Im Yamato, it is
Kaido's son". but, he still inherit Oden's will, trying to open up land of Wano.