I agree it's kinda shitty writing, but if we're to rationalize it, Rayleigh didn't want to get involved in Ace's adventure. Hancock has a direct connection with Rayleigh—He raised the siblings for a while, and she has no one to rescue her while Ace had WB and others. Rayleigh knew WB would come to his rescue. I don't think Rayleigh expected WB to fail as he wasn't aware of WB's health condition, and he had no plans to play an active role in this era until he saw Ace and WB die.