Because the Roger Pirates never faced anyone hellbent on destroying anything. We've seen their skirmish with the WB pirates during the Wano flashback, where they're all having fun, punching and haki slicing away. Even during the "terrifying" GV, IMU only wanted to take care of Xebec, and afterward DR Xebec was targeting people one by one, he wasn't aiming for mindless destruction.
Harald is an Ancient Giant. If the dude starts sending haki slashes around islands populated by braindead normies, then it is something the Roger Pirates have never encountered before. It is a form of evil Gaban was sheltered from in their happy-go-lucky adventures to Laugh Tale and back--for some reason. Like, I don't understand Oda not having them face the rot of the WG, aside from IMU's short appearance on GV, but that's that.