Not all stuff that Anime adds can be swept under the rug. There are clearly scenes which have no root in the Manga.
For example, Brook fighting Oven and Sanji fighting Oven. That literally never happened in the Manga.
You can consider it not canon as well, however, it wont change that what Anime has shown is most likely ending what actually happened in Manga too but if you want to go another road is up to you and keep in mind that you might set yourself up for another failure due to the same thing. I totally expect Moe and Erkan to do this and since you usually follow their lead...
For example, Brook fighting Oven and Sanji fighting Oven. That literally never happened in the Manga.
It is cannon thus Idc.