As far as I could understand, Roger was surprised that Oden, Neko, and Inu didn't immediately tell him that there was a red poneglyph in their country (Wano). Roger was recapping the poneglyphs he had collected, with one of them being the one that was stolen from Linlin. He was also still gathering for more so he would've completed all 4 like we could see later on in the chapter.