The line is, "あの実は我々にとっても伝説だ", meaning, "Even to us, that fruit is a legend." That means that Luffy, Kaido, Momo, Brook, Robin, Law, Chopper, etc. can't be the user they're talking about because the Gorosei already knew of them. The fruit has to be one that has only recently come into their...