So let me ask you. How does it make any sense for there to be a tree, that for no explicable reason, creates a fruit, that allows a person to make clothes out of nowhere?
How is that scenario, in your mind, better than say for example, a person who lived in an impoverished, cold nation, where people were dying all around them due to the extreme cold, having a strong desire to want to save the people of thier Country, by providing adequate protection against the elements, manifesting the ability within themselves?
And scenarios like that can practically be thought of for every conceivable devil fruit.
The concept of Manifestation power through Will is already established with haki. This just further expands upon it. You're not a fan of that.
Yet, a magical fruit tree, with nothing that could possibly give a substantial reason as to why it created this power, would be a better scenario for you.
Help me understand.