Yamato, Catviper and Dogstorm stayed behind because they believed Momonsuke couldn't protect it alone. Ryokugyu then stated that Wano was only safe due to Kaido. Oda wouldn't have all those characters tell us of this risk if no one would attack Wano. Yamato staying behind would look like it was a waste. Someone will attack: Shiryu. He will take two mythical Zoans: Kaido's and Orochi's. The first for his sword and the latter for Blackbeard.