I don't have much of an issue with any individual memo client allowing this functionality, but once these blocks/ignores are on-chain they become indicators that can be abused.
You can't censor onchain, without majority of miners cooperating to censor.
True, but memo is the conjunction of two networks: blockchain and humanity. "Muting/Blocking" on *EITHER* side (via miners or popular opinion) endangers this intersection's capacity.