Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Hugging Face Transformer Deployment Tutorial #49

Merged
merged 21 commits into from
Oct 24, 2023
Merged

Hugging Face Transformer Deployment Tutorial #49

merged 21 commits into from
Oct 24, 2023

Conversation

fpetrini15
Copy link
Collaborator

Tutorials to show how hugging face transformers can be quickly deployed in Triton.

@fpetrini15
Copy link
Collaborator Author

All generation scripts were removed and replaced with static files. This new tutorial covers deploying falcon7b, persimmon-8b, and mistral 7b. Down the road, these models may get there own READMEs in a "Popular Models Guide" folder cc @jbkyang-nvi.

rmccorm4
rmccorm4 reviewed Oct 3, 2023
View reviewed changes
Copy link
Collaborator

@rmccorm4 rmccorm4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great tutorial overall! Only minor comments 🚀

@fpetrini15
Copy link
Collaborator Author

@nnshah1. I preemptively removed Mistral from the tutorial. I can always revert if necessary.

@fpetrini15
Copy link
Collaborator Author

Incorporated some feedback from Dora incorporating how to gather performance metrics, load cached models, and adding comments.

@fpetrini15
Copy link
Collaborator Author

CC @nv-braf @matthewkotila in case there is any feedback regarding the PA/MA section.

@matthewkotila
Copy link
Contributor

PA stuff LGTM 👍

@tanmayv25 tanmayv25 merged commit de7da4a into main Oct 24, 2023
3 checks passed
@fpetrini15 fpetrini15 deleted the fpetrini-hf-transformer-tutorials branch October 24, 2023 00:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yeahdongcn yeahdongcn yeahdongcn left review comments

@Tabrizian Tabrizian Tabrizian left review comments

@nnshah1 nnshah1 nnshah1 left review comments

@rmccorm4 rmccorm4 rmccorm4 left review comments

@dyastremsky dyastremsky dyastremsky left review comments

@jbkyang-nvi jbkyang-nvi jbkyang-nvi approved these changes

@tanmayv25 tanmayv25 Awaiting requested review from tanmayv25

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
9 participants
@fpetrini15 @matthewkotila @yeahdongcn @Tabrizian @nnshah1 @rmccorm4 @dyastremsky @jbkyang-nvi @tanmayv25