I really wouldn't mind even if it were stored on chain, but an app should be able to override with local custom settings not stored on chain (for privacy).
There might be a way of storing it on chain by encrypting it first with your public key. This way it would work in every client you used.
In fact, you could decide that if 1 or more of those you follow blocks someone, they're blocked for you too. If downvoting happens, it could work same way. Not censored, but filtered.