Fee is paid on the size of a tx in bytes. Consolidation uses many inputs to the one output. Many inputs take more space. I had about 40 of them, tx was 5kb in size. (20x more)