hjelm_learning_2019: Learning deep representations by mutual information estimation and maximization
Learning deep representations by mutual information estimation and maximization
Local MI Maximization
Maximizing mutual information between the input and output may not be useful for the task. Trivial pixel-level noise is useless for image classification. By maximizing the average mutual information between the high-level representation and local patches of the image, one can encourage the encoding aspects of data that is shared across patches.