Even "harm" is a difficult issue in reality. When a doctor prescribes a drug with disease-curing effects as well as side-effects (all of them have side-effects), does he harm the patient?
Harm specifies an action someone does -to- someone else _without_their_agreement_or_consent_. Consent or agreement are not default values in social interaction. Absence = No.
I find the suggestion that "harm" requires lack of consent highly problematic. But still, what if the patient is in a coma or only accepts it because the doctor lies about the effects?