Come on, it's obvious that Jinbe didn't join on FMI because he was stronger than Sanji at the time. Apparently Luffy, Zoro and Sanji always have to be the three strongest. It's the same with Yamato. The difference is that Oda didn't even bother with the excuse anymore. Even if Yamato still joins in the final arc, it won't be the same as with Jinbe, who will now adventure with the Straw Hats.