No, at this point that scene with Shanks and the gorosei is almost fully explained (the pirate he wanted to talk about is still unknown but or it is BB or it is Luffy, most likely). The gorosei had no problems being alone in a room with Shanks because they knew that at least together or even with Im if necessary they could at least put up a fight (but most likely not win without heavy losses, now the original question is reversed as well, why the gorosei and Im didn't gangbang Shanks? They know he is a big aid for neo Nika) and Shanks at the same time most likely knew (unlike Sabo) what kind of power the gorosei and Im have (otherwise he would have tried to erase them just like Sabo)(or at least we can suppose so, afterall Oda had Shanks act like a moron in Marineford letting an injured BB simply go away with his jew crewmates and even more importantly WB devil fruit).
The important question is, if actually it is the case, how did Shanks know about their power?