Who decide what is wrong to say? Example, IDF says that criticising the actions of the IDF is antisemitic. Are you saying that you are antisemitic?
Hmmm, a sociologist might answer you more precisly to the question but basically no one and everyone. What is acceptable to say in a society with freedom of speech is basicaly what people in a society deem acceptable to say or not.
In a society with a mainly "normal" freedom of speech, there is not someone who will say what is right or what is wrong to say. It will be a mix of what the people think, their influences (media/power in place) etc. It will be like a needle evolving on a spectrum for different subject.
If IDF says that criticizing the action of Israel is not acceptable then ok, but we need to add that vision to the vision of the entierity of the world on the subject.
When we do that we notice that only some states, some medias/personnalities and Israel is saying that, the rest of the world disagree that it is not acceptable to criticize Israel. Therefore criticizing Israel is perfectly acceptable in our society.
(I'm not making a judment of value here, I'm just stating hat is acceptable for the society in time T and what is not)
Now...
We, the left, knowing that this process can lead toward problematic and hurtfull ideologies becoming acceptable in society, we are trying to add one more parameter to the balance and give it more weight than the rest :
Science
We believe that by giving more weight to the conclusions of science (social or hard sciences), we will be able to better prevent problematic and hurtfull ideologies to reign over the field of free speech. Because when we look at scientific researches, we understand that language can become one of tools of oppression when uncontrolled.
This is why in definitive, we believe in free speech, but not absolute free speech.
For example, we known that giving a tribune for far right ideas (and therefore missinformations) in the medias will straighten said ideas. This will displace the overton windows radically to the right and make those ideas more acceptable in the mind of the public. Those ideas will be then more debated and in consequences even more normalized. This is one of the reason why the left explains that we must avoid debatting with the far right in huge public spaces and why we must absolutely avoid inviting far right personnalities in the medias.