The spoilers legit says Rayleigh gave all parties involved an ultimatum. Rayleigh is not strong enough to beat BB, but he's strong enough to make BB back off because it's risky.
Except he didn't make Blackbeard back off due to his prowess threat. Lol.
Boa Hancock gets caught off guard by Blackbeard while she's kicking at someone.
Boa Hancock then: Yo I'll undo everyone here in stone, if ya'll leave.
Blackbeard: ".... I dont' trust you!! You'll turn every one of us here into stone...."
The man here is sacrificing two of his commanders, that he became a warlord to get, because he's that fearful of Boa's power and believe she'll turn everyone of em mfs into stone if he lets go of her (and he only got her while she was fighting someone else).
Rayleigh comes in and tells Boa to undo the stone on everyone. And essentially guarantees she won't turn them into stone after that.
It has 0 to do with Blackbeard not wanting to fight Rayleigh, hence that's never even presented as a thought for Blackbeard let alone verbal expression. Rayleigh allowed Blackbeard to leave with two of his core commanders + others, which is more beneficial for Blackbeard than just killing Hancock and killing Rayleigh. But he also ran the risk of Rayleigh freeing Hancock from him... and that would've led to the thing he feared in the first place.. Hancock owning all of em.