Crocodile was far from a scrub against Alabasta Luffy, Lucci also was far from a scrub back in Ennies Lobby. Soo you might be the one that need to re read the entire thing if you think they where ever treated as "just scrubs" in their own arc. Pica was mostly also not a gag fight asside from Pica's stupid laugh, even tough it was barely a fight at all since most of it was Pica running away from Zoro. Sanji is a whole diferent thing, a gag character gets gag fights, no surprise.
Thing is, when it was time for the final showdow against Luffy, the fights where serious, Luffy fighting Crocodile bellow alabasta was badass, most of Luffy's fight against Lucci was also serious even the small Luffy shit after using G3 was somewhat serious as Lucci was having none of that BS.
Right now with Kaido? Not soo much. Luffy already lsot to him 3 to 4 times, it wasnt supposed to be a completely goofy fight at this point.