I just tried to show logic: If you can open a door (cause you have a key for) - it does not prove you own the house. - So it does not help to scream for 'show the key'!
No it is not absolute proof, but the probability that you own the house or are the person you say goes up a lot. What % of times is a door opened with a key by the owner vs not?