That was before Luffy lost. Situation has changed- he needs to rush to Kaido who is after Momo, because his captain is out of fight and Kaido is running loose.
Zoro also said that he's after Kaido before.
Luffy also said he'll defeat Kaido, only to lose right after.
Scabbards also said they got it. They lost.
Luffy was also allowing Scabbards to take on Kaido. He ended up joining.
Let's look at the actual build-up, portrayal and path of story instead of "He said he'll leave it to Luffy during a completely different situation, so that's what's going to happen."
That is not how story writing works. Oda is creating a great fiction, not a child's book lol