If it's Prime Rayleigh alone ... he could match Kaido , while Oden could fight King , Ashura vs Queen , Denjiro vs Jack , Kawamatsu vs one of Flying 5 ( without Drake of course )
The other 4 of Flying 5 , 4 Yakuza Bosses could fight them , maybe they'll lose though
Kiku , Inu , Neko , Raizo , Kanjuro , Kinemon , other Daimyo of Wano ( like Shimotsuki Ushimaru , Prime Hyogoro ) could fight Orochi , Kyoshiro , 2 other Orochi's main men , Mimawarigumi , Orochi Oniwabanshu
In the end i think the good guys as a whole could be weaker , but they would give hard fights
Especially Apoo and Hawkins weren't in Beasts Pirates too yet
__________________
If it's the entire Roger Pirates , then Roger + Rayleigh can steamroll Onigashima . the others can just watch while eating popcorns