Yes, but in my opinion there are 2 crucial factors that should be considered:
1) WB was an incredible legend at the top of all the pirates. That was especially true for normal citizens that didn't know how bad was his condition at that time (and actually no one aside from his nakamas knew; maybe only Shanks).
The incredible consideration shown to WB (higher than even the other Emperors) came also from Garp, who did not know the severity of his disease:
2) The average person believed that WB could destroy the world with his powers, and actually not only the average person. That concept was repeated various times:
A type of fear that no one aside from him could cause, at that time.
So, at least in my opinion, that kind of reaction we saw in 581 was justified.