I actually don't think that Vergo and King Baum are too far off in terms of tiers. King Baum tends to be forgotten, but he's clearly one of the strongest homies and both feats and implications put him somewhere in veteran level.
King Baum's major weakness from what we've seen, is his lack of attack feats. In the manga, I don't think he technically has anything to show as a fighting style. The anime though, fills some hold by having him seemingly go for grabs with his relatively small arms when fighting opponents, however, the true best attack feat King Baum has, would be when he essentially body-slammed a cracker clone, knocked it down, and sent it skidding. This was a filler scene in an episode of the anime and mind you, it was a surprise attack, so the clone didn't have it's guard up, but Cracker clones are capable of blocking and tanking Elephant guns without getting knocked down, so this is still pretty impressive. Not a veteran level stat, but clearly impressive.
More impressive are his other stats. Speed was showcased as particularly good as the straw hats considered it worth it to ride on him on the retreat even when Big Mom yourself was hot on their trails, IE King Baum has travel speed equal or greater than the Strawhats. Dodging is obviously a bit worse, after all, King Baum is absolutely huge, but I don't think King Baum will have all that much trouble tagging Vergo.
Durability is a bit of an unknown. In the anime, Cracker could slice most of his leaves off in a swing, but lets not forget that Cracker is a swordsman on a level capable of cutting into Gear 4th and that Anime Cracker's main body was capable of keeping up with Gear 4th in a melee (and it is anime Cracker that this anti-feat is up against). He has another anti-feat for durability against Armande would slices him into two, but she did this over time in both the anime and Manga, while King Baum was not resisting. Not to mention that Armande herself has some incredible cutting feats slicing clean through Perospero's Candy wall without relying on heat: the same thing which only the three germa brothers in their suits working together can otherwise accomplish, so this anti-feat doesn't really say much either. And of course, Big Mom's Attacks are capable of harming King Baum, elbaf spear cutting through him like paper and Promethius's expanded form drowning him in fire. But I'd argue that it's actually impressive the attacks didn't cause more damage. Especially since they're the two strongest attacks Big Mom seems to have and one of them is a direct elemental counter.
But that leads to endurance: put simply, his Homie nature means that his endurance would persist for as long as his face remains mostly intact and as long as he isn't scared to death. IE, he'd survive and live even if he ends up in a worse condition than Whitebeard does at Marineford. His endurance is exceptional. He even survives getting chopped clean in half just from his sides getting sown back up.
Finally, looking at portrayal, King Baum and Brulee together were the leaders of the usual "seducing woods" team, with King Baum clearly in charge of the fighting. This is relevant since the seducing woods is very clearly the most significant locational trap Big Mom uses against ambitious pirates; IE, a sheild for Whole Cake Island rather than something to be protected. And being able to consistently defend something against the monstrously strong new world pirates that doubtlessly attack Wholecake on a semi-consistent basis, would usually qualify as you as Veteran level. In fact, when the straw hats were getting captured, Brulee was actually complaining that her job was too easy, and most of that would've obviously been due to her access to King Baum.
Thus, I don't feel bad putting King Baum on the same tier as Vergo at all.
In terms of actual matchup, Vergo doesn't exactly have any fire themed attacks, though his bamboo stick doubling as a blow pipe and equipped with explosive rounds might semi-qualify. As said, Vergo would probably have some trouble outpacing King Baum, but since Vergo knows a variation of skywalk he can just retreat into the sky. This option for him is relevant since Vergo is the one with limited time power boosts here, in the form of Full-body armament which I'm 100% certain King Baum cant do damage against. Except that since we know Haki runs out, if King Baum can last against Vergo for long enough, he might no longer have access to his unbeatable defense. This is where King Baum's durability, size and endurance comes in. Vergo is clearly very strong, but his actual attacks aren't Law or Base Doflamingo tier.
I think there's a slim chance that King Baum outlasts Vergo's Haki use by running and endurance and ultimately what remains of King Baum finishes Vergo off afterwards with a series of body slams, assuming both sides are bloodlusted.
