Skip to content

Latest commit

 

History

History
25 lines (20 loc) · 814 Bytes

README.md

File metadata and controls

25 lines (20 loc) · 814 Bytes
Raw

Is-Attention-all-I-need

Purpose ?

  • For a long time, I have heard of this paper and always wondered to know it properly.
  • This repo is one way for me for that redemption.

Method

  • Read the paper, understand the maths.
  • Correlate the knowledge with LSTMs and RNNs.
  • Implement the vanilla architecture using pytorch.
  • Train the model on the WMT2014 English to German Translation Task.
  • Rinse and repeat on some more models that came after that.

Time Duration ?

  • Start Date : 02/January/2022
  • Will Complete by : 02/February/2022

Resources ?

  1. Paper : https://arxiv.org/pdf/1706.03762.pdf
  2. Data : https://huggingface.co/datasets/wmt14
  3. How to write Pytorch Code : https://nn.labml.ai/transformers/mha.html

Video Resources ?

  1. https://www.youtube.com/watch?v=iDulhoQ2pro