You're making a case for ASICS(and Bitcoin in general). Right now you have to spend a butt ton to get even a small percentage of the network.
But if you are a malicious actor, and have a butt ton of money to spend without skipping a meal or losing your yacht, you can buy the power to vote everyone out. That's unfair.
You said yourself, a rich person could just buy up a server farm and attack. This actually happened to Bitcoin Gold which forked to an ASIC resistant PoW algo.