doubt it
1. Oda never showed Fuji in Oden's flashback, not even a hint
2. Fuji just allows Kaido and Orochi do whatever they want in his home?
3. He has grudge and prioritizes shichibukai, pretty random since there's Orochi and Kaido.
4. Appearance barely matters, Jinbe and Sasaki have similar outfits as fuji
5. Didin't Kaku use same swords?