Collect ressearch papers

[DeepCowProductions]

• Quantization of Transformers models (BERT + GPT2)
• Details about internal architecture (activation functions, LayerNorm, Softmax)
• Details about quantization methods and the effect htey have on the model
• Success and failure cases of Quantization methods

[iksnagreb]

I can recommend this blog post as a starting point, if you have not seen it already: https://lilianweng.github.io/posts/2023-01-10-inference-optimization/

Regarding one of the challenges discussed there, the "emergence of outlier features", I have done some more reading a few months ago. If this gets relevant, I could share my summary, once I find some time to restructure it properly...