True, but if there is a will from the community to remove it, then it should be removed. Statues aren't for life, if their time is done then they should be removed.
And I'll say it again, this should never be done by mob rule.
But there wasn't a dominant will for it being removed from the community based on reading I've done about it from the media of the time. There was a group that wanted it to go and there was a group that wanted it to stay and a compromise was agreed to put up plaques explaining the history. The Local Authorities leading it didn't do their job decisively enough over the wording of the plaques after disagreement over it with both groups* and then the mob happened. My memory of reading on this may be wrong though.
*As said above, the LA should have gone warts and all about Colston's history on the plaques regardless of the complaints of either group. Probably doing the job right if they irritate both sides in thsi case.