The issue with tricksters is that you can't judge their accurate power level based on their performance against selected individuals. Doffy isn't a threat to the likes of Yonko while Hancock could potentially kill them, but we know for a fact that Doffy is the stronger fighter. It's the same with Zori and Shiryu. Zoro scales above Shiryu generally speaking as in he can operate at a much higher level than him as a fighter. Pit Shiryu against Kaido, he isn't even giving him paper cuts even if Kaido can't detect him. Do the same with Zori, Kaido is covered in blood before Zoro leaves the fight.
Shiryu is insanely strong, but he's not up to the mark overall(when you're comparing him to the top tiers) if he's gonna need 10000 IQ moves just to land a hit on an 80-year-old Garp. The Rain is only a serious threat to Zoro because he lacks the proper tools to fight him, not because he's generally stronger than him.