Kizaru was responsible for the worst defeat in the history of the mugi, and Oda showed us that they remember very well what happened.
Kizaru was present back in MF, had a connection with Vegapunk and was even part of the FT/Otohime FB.
Now he has returned and is the big villain of this important Arc.
Kizaru is a big enemy under any definition, just like and even more so than Katakuri, Doflamingo or Kaido.
He is not a final villain like Akainu, BB or Imu, but no one said that.