He was controlled by Imu doesn‘t mean he is delusional in what he was capable of before.
Look here there was Harald for a brief second free of Imu,himself stating noone could beat him with those Godly Powers.
My point is that he never makes a specific claim about what he could do before, he's just confirming that he could for sure tame Ragnir at this point. We also have an entire flashback of him, and Oda never depicted Ragnir and Harald interacting. To some extent, we have to acknowledge that empty hype does kind of exist. I doubt it's a coincidence that once he starts thinking of himself as a God, he wants Ragnir, when he wasn't interested previously.
But again, if Ragnir actually shows that he's stronger than Harald, I don't have a problem acknowledging it, but we haven't; actually seen anything from Ragnir proving he's on that level. At the very least, it would make Ragnir the third strongest on the island currently, only after Imu and Loki.