Why are they ruined? Zoro is a nobody from East Blue who somehow has resemblance to Ryuuma and since Ushimaru is a descendant of Ryuuma, also to Ushimaru. Perfectly fine.
Also, Zoro doesn’t want to visit Ryuuma's grave yet, because he feels he still needs to prove himself as a master swordsman.
My headcanon rules.