add dedicated endpoints recipe #122
Conversation
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
MoritzLaurer
changed the title
| @@ -0,0 +1,924 @@ | |||
| { | |||
There was a problem hiding this comment.
"Have you ever wanted to create your own machine learning API?"
"...thousands of models on the HF Hub, create your own API on a deployment platform you control, and on hardware you choose."
Reply via ReviewNB
| @@ -0,0 +1,924 @@ | |||
| { | |||
There was a problem hiding this comment.
| @@ -0,0 +1,924 @@ | |||
| { | |||
There was a problem hiding this comment.
| @@ -0,0 +1,924 @@ | |||
| { | |||
There was a problem hiding this comment.
"...the TGI container running on the endpoint applies..."
"Here is the slightly modified code..."
Reply via ReviewNB
| @@ -0,0 +1,924 @@ | |||
| { | |||
There was a problem hiding this comment.
I think it makes more sense to nest this under Creating your first endpoint since they're related topics.
Reply via ReviewNB
| @@ -0,0 +1,924 @@ | |||
| { | |||
There was a problem hiding this comment.
This can probably be a second level section ## Additional information since it is pretty general and not specific to creating endpoints for a variety of models.
Reply via ReviewNB
| @@ -0,0 +1,924 @@ | |||
| { | |||
There was a problem hiding this comment.
Readability rewrite:
You can use InferenceClient to easily send requests to your endpoint. It's a convenient utility available in the huggingface_hub Python library that allows you to easily make calls to both Dedicated Inference Endpoints and the Serverless Inference API.
Reply via ReviewNB
| @@ -0,0 +1,924 @@ | |||
| { | |||
There was a problem hiding this comment.
Now let's create an endpoint for a vision language model (VLM). VLMs are very similar to LLMs, with the difference of taking text and image inputs simultaneously. These models autoregressively generate text, just like a standard LLM. VLMs can tackle many tasks varying from visual question answering to document understanding. For this example, we will use Idefics2, a powerful 8B parameter VLM.
I feel like document reasoning is a very cool industry use case for multimodal so I thought I'd add it, IDEFICS2 has good doc capabilities, other than that mostly did rewriting for clarity
Reply via ReviewNB
What does this PR do?
Adds the enterprise hub cookbook for dedicated endpoints
Who can review?
@merveenoyan and @stevhliu.