Ipex llm upstream #33
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch (oobabooga#5257)
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
Merge dev branch
* Add bigdl-llm loader * Add BigDL-LLM if-else fork for encode * Remove 'cpu-embedding' parameter * Migrate source code to ipex-llm * Add command-line flags for ipex-llm in readme Edit description of options in GUI
* update style * minor * fix * revert
|
@jason-dai @shane-huang Removed some arguments, this is a cleaner version for upstream. |
shane-huang
approved these changes
Apr 9, 2024
| # bitsandbytes 4-bit | ||
| group = parser.add_argument_group('bitsandbytes 4-bit') | ||
| group.add_argument('--load-in-4bit', action='store_true', help='Load the model with 4-bit precision (using bitsandbytes).') | ||
| group.add_argument('--load-in-4bit', action='store_true', help='Load the model with 4-bit precision (using bitsandbytes or ipex-llm).') |
There was a problem hiding this comment.
"using bitsandbytes or ipex-llm" is a bit confusing. Should we tell user in which case to use bitesandbytes and in which case ipex-llm?
There was a problem hiding this comment.
No which case? If one is using bitsandbytes or ipex-llm, this argument will both take effect.
|
shall we mention how to use IPEX-LLM in webui somewhere? Need to modify some documents in webui? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
@jason-dai @shane-huang @sgwhat Please take a look to see if this is appropriate for upstream.
https://github.com/oobabooga/text-generation-webui/wiki/Contributing-guidelines Seems no need to add tests. The style checks have been verified.