Plot twist:
Oden has been inside of Enma this whole time and the Haki drain was just his way of training his allies Ryou like he said he’d do in the flashback. Zoro and Oden will talk within the sword and Zoro will use Oden Nitoryu to do it.
With the amount of Oden wank Oda does seems likely.