Honestly, you’re not far wrong.
It was said in-series once in Thriller Bark, and I think the three were called monsters in Alabasta, if not quite the Monster Trio.
One (or two) references in twenty five years, the last time about fifteen years ago. It’s basically a throwaway line that has stuck.