Jethro's Braindump

And the Bit Goes Down: Revisiting the Quantization of Neural Networks

Model Compression
(Stock et al., 2019)

This method minimizes the loss reconstruction error for in-domain inputs, and does not require any labelled data.

This method exploits the high correlation in the convolutions in ResNet-like architectures by the use of product quantization (PQ). The approach here focuses on reconstructing the activations, and not the weights. This results in better in-domain reconstruction, and does not require any supervision.

Vector Quantization (VQ) and Product Quantization (PQ) decompose the high-dimensional space into a cartesian product of subspaces that are quantized separately. These are typically studied under the context of nearest neighbour search.


Stock, P., Joulin, A., Gribonval, R'emi, Graham, B., & J'egou, Herv'e, And the bit goes down: revisiting the quantization of neural networks, CoRR, (), (2019).

Icon by Laymik from The Noun Project. Website built with ♥ with Org-mode, Hugo, and Netlify.