A patent that has not been turned into an actual product or is not currently under use (with proof that you are using it for example with investment into R&D) should be nulled. Eventually I believe we will find a much better way to implement patents than what we have now. Also having proof that you have been building something (using blockchain timestamping for example) before another one filed a patent, should be enough to give you the right to keep going with 0 fees.