Kaido would just one shot him immediately so it'd be 4v2 with only Luffy consistently being able to deal damage to Kaido.
When his haki runs out, he'd be fucked since he wouldn't have a reliable crewmate there to save his ass.
Whenever the duo decide to use Hakai, the SN would just get evisercated and so the Yonko win.