Yeah, that view is only consistent if you don't understand rights or deliberately twist the meaning. Rights can only be taken away by moral agents, that is men, not by nature or reality itself.