Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add AutoModel support for SparseAutoModel #2003

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

mgoin
Copy link
Member

@mgoin mgoin commented Jan 24, 2024

Add support for loading Transformers models without specifying task attributes. This is especially useful for exporting models for embedding extraction. This current accessed by "model" or "base" task - I'm very much open to a better name here

example:

> huggingface-cli download sentence-transformers/all-MiniLM-L6-v2 --local-dir all-MiniLM-L6-v2
> sparseml.export --task model all-MiniLM-L6-v2
> ll all-MiniLM-L6-v2/deployment 
total 88M
-rw-rw-r-- 1 mgoin mgoin  612 Jan 24 19:50 config.json
-rw-rw-r-- 1 mgoin mgoin  87M Jan 24 19:50 model.onnx
-rw-rw-r-- 1 mgoin mgoin  350 Jan 24 19:50 tokenizer_config.json
-rw-rw-r-- 1 mgoin mgoin 456K Jan 24 19:50 tokenizer.json

dbogunowicz
dbogunowicz previously approved these changes Jan 25, 2024
bfineran
bfineran previously approved these changes Jan 29, 2024
Copy link
Member

@bfineran bfineran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe base_model but lgtm

@mgoin mgoin mentioned this pull request Apr 3, 2024
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants