Ushimaru was accompanied with a fox, the Spanish word for Fox is Zorro, there are some hints.
In my opinion, sometimes the easiest answer is the best. No need to made things more complicated.
Hyugoro specifically said he's the spitting image of Zoro, if Zoro had another father Hyugoro might had know him as well.
In my opinion if Ushimaru isn't Zoros father, there's only one possibility, he's Zoros uncle the identical twin of Zoros father.
If Ushimaru had an identical twin
, Then this twin of the family of the twin could have leaved wano 50 years ago with the other Samurai who founded Shimotsuki village. In my opinion that is unnecessary complicated.
The more likely scenario is that Zoros mother or some one Ushimaru trusted fled from Wano 20 years ago, Zoro was 1 year old. That's pretty possible. If people left wano 50 years ago, why not 20 years ago ?