If the referenced media is YT, the payment info could be in the YT video description which the memo client finds. Could then reasonably know that the address refers to the uploader.
I like that idea. If there was a reference on the chain to an address that wasn't in the description, or meta data in the case of a magnet you would just surface a warning.
