It's always some unknown power source which gives them huge advantage kind of thing that beats luffy.
Against Crocodile - Logia powers
Against Lucci - he was weak and slow (Gear 2 fixed it)
Against Moria - Again Paramacia Hax (Haki problem)
Against Kizaru - Logia powers
Against Caeser - Carelessness and Logia powers
Against Katakuri - Future Sight
Against Kaidou - Advanced Conqueror.
Against loki (If he turns out to be a villain) , I'm still skeptical about it and won't make any comment unless stated explicitly.
But only power up I can see popping up now is , Disabling DF , like Shanks did with Aramaki and Joyboy Haki did to Gorosei.
Against Loki - Disabling Nika fruit and It really ain't that difficult to one shot base luffy with Loki level AdCoC + Physical Strength of giant.