Thanks for the suggestion, I shall think about it. My immediate concern is an USB stick is far more than a passive device (it is quite hackable), and it connects to an insecure machine
Freshly set up offline computers that you trust do not grow on trees. Using an offline computer for payments...even using cryptosteel is probably easier than all this.
Encrypting wallet files does not really solve the fundamental problem. You have to decrypt at some point with keys. The security of this process is what the hd wallet is for
Then I would consider the computer to be part of the system to replace a HD wallet. And it would have a far bigger attack surface than the HD wallet has. Probably also less convenient.