They're all part of the Rocks pirates and were at GV.
All the Yonko being tied to a single event is pure nonsense.
40+ years of piracy and not a single individual that isn't connected to Roger,Garp or Rocks has made it to the Yonko position.
"All the Yonko being tied to a single event is pure nonsense. " How is that pure nonsense? you're just yapping now.
It's usually earth shaking events like this that determine the future for years to come, Rocks Pirates are supposed to be an assemblage of monsters who could all be leaders in their own right, but that's the point of showing Xebec's influence.
We also literally know the D are tied to fate, Buggy is definitely Xebec's son as well most likely. The narrative has already told us multiple times that coincidences don't exist, and what-ifs are pointless. Buggy's entire char is falling upwards, shanks is a child of fate etc.
E.G what if someone not involved with these figures became a yonko? that is irrelevant, these big players in the present were part of and witnessed big events in the past. That undoubtedly shaped their outlook and piracy, as well as their fate in terms of where they will end up.
If you don't like the idea of fate you should probably stop reading.
You then say this "I thought this was a story about dreams and that piracy was all about guys who grew up with nothing rising to the top through their willpower and ambition?"
Man it's like you're not even reading, which is probably the case. Even with this external factor of fate, the case is simply no one else was strong or lucky enough, If you grew up having nothing what are the odds you make it out? if someone defies those odds and we use your kind of logic that should also be BS, and it is BS, it's just as BS as the yonko. Because you want someone to roll a 0.1% success rate without being carried in some way. Which would intrinsically be tied to the concept of fate anyway.
Lets look at these kinds of pirates, chinjao for example was a big name in rogers era, yet he got crushed by garp and retired from piracy, moria had his crew destroyed by kaido. These are the literal "guys who grew up with nothing" except they don't rise to the top, their dreams great crushed.
So NO, the story is not about that, if it was about that everybody would be the main character overcoming all odds, the fact is not everyone achieves what they want.
Willpower and ambition are not enough, for example ace was also naturally gifted but he wasn't fated like roger, ace acted exactly like how roger would've acted when someone insulted his crew, we know roger was always on the front lines and destroyed armies because they talked badly about his crew. Ace was echoing this when he attacked akainu, but ace doesn't have the same fate as roger so he dies instead.
It's also like you forgot how rare devil fruits are that your "chadmirals" rely on?, most people in the one piece world aren't going to come across them, but our viewpoint in the narrative is focused on combatants likely to have them. Those "chadmirals" were also said to be extremely talented and monsters in their own right, all of them. then got fed bullshit power ups like magma and light.
If there's anyone you should be glazing it's garp and koby, and not mythical zoan and logia abusers.