Tiny Transformers

This repository contains an autoregressive decoder transformer trained from scratch on TinyStories for story generation.

Introduction

Our model is hosted on Huggingface here. If you would like to run the Streamlit demo locally, execute streamlit run app.py.

For points of contact, please directly contact the authors of this repository.

Tinystories (License: CDLA-Sharing-1.0) https://arxiv.org/abs/2305.07759

Name		Name	Last commit message	Last commit date
Latest commit History 23 Commits
figures		figures
model		model
train_logs		train_logs
.gitattributes		.gitattributes
.gitignore		.gitignore
README.md		README.md
app.py		app.py
app_utils.py		app_utils.py
constants.py		constants.py
generate.py		generate.py
main.py		main.py
model.py		model.py
plot_attention.ipynb		plot_attention.ipynb
requirements.txt		requirements.txt
tokenization.py		tokenization.py
train.py		train.py
upload_tokenized_data_to_hub.ipynb		upload_tokenized_data_to_hub.ipynb
utils.py		utils.py