this is just impression you guys got of numbers that they are all supposed to be Huge Giantly creatures... Apoo as mentioned was already part of Beast pirates before making alliance with Kaido, so what is he then ? just a headliner ? he shouldnt be, if Drake can become TobiRoppo then why not Apoo, I certainly think Apoo was of stronger Supernovas....but anyways it is said in official translation he was holding barrel in his hand not human and also as I said they were talking as equals on one table..... what if numbers are special ability users not just giant creatures, Apoo certainly has special ability unlike zoans
But you are missing the whole picture here by focusing on "ifs" and small, isolated details.
When we're introduced to the Numbers we have six clearly visible individuals. Four of them are giants, another is Apoo and the last is one of Apoo's subordinates. Let's ignore the latter because, well, he's simply a subordinate. So that makes five people. Out of these:
-Apoo is the only one to be small.
-Apoo is the only one who doesn't look like he'd eat a kid.
-Apoo is the only one who wouldn't be labelled as a thing or referred as a what.
-Apoo is the only one without horns.
Also, the only monster clearly visible in last chapter also sums up all these traits: it is big, horned, fanged and also a problematic drunk (which Queen was worried about). And he's attacking one of the small guys.
If we not only count the visible individuals but the shadows we have that Apoo is the only small guy out of arguably seven giants. On top of that we see three giant kanabos (the weapon of choice of an oni), those Numbers with their mouth open are all fanged (like an oni) and the Numbers as a whole are referred as drunkards (like an oni). On top of that, the kids stated them to be man-eating creatures (like an oni) from Oni Island.
I don't think this is just a matter of our impression. The Numbers have a huge amount of common traits, which doesn't happen with any other group within the Beast Pirates. They don't belong to a thing, they are a thing. You may think they are users of especial abilities, but it would be really coincidental that such abilities turn all of them but Apoo into arguable onis, not only in terms of their size but their weapons and behaviour.
That's the point here, that they aren't just "giants who may be able to srink because they ate a fruit". It is the whole package, and Apoo is completely disconnected from that idiosyncrasy.