Here's the thing. Yonko crews are usually extremely hesitant to fight each other.
Yet Shanks has 0 fear of fighting Blackbeard and goes to wano to do so despite Kuzan being an issue.
The logical conclusion is that Shanks simply doesn't see Kuzan as such a threat that he can't over come him and Blackbeard.
As in RHP vs Blackbeard + Kuzan and a few other commanders.
Invading Wano is simply seen as far far far more difficult than fighting Blackbeard.