There's people too stoic (or stupid) to be affected: luffy (stupid) , zoro (stoic), law (you all know why), Hawkeye (stoic), shanks (stoic) and many others.
Boa's Devil Fruit was the most hyped thing in this chapter
When Blackbeard saw the chance of not needing to kill his subbordinates (since Boa obeyed Rayleigh), which was something he was willing to do cuz he thought Boa could turn literally everyone into stone once he had let her go, he simply accepted the terms cuz he probably thought it was safer that way
Adding Rayleigh there just made things harder for Blackbeard to decide, since he placed Boa's Devil Fruit in a high pedestal considering it was too risky to be dealt with even for BB himself, to the point he was certain his subbordinates would die in a way or another, and Rayleigh's presence would make things harder to everyone.