Adding automatic_embedding_tei_inference_endpoints.ipynb #5

datavistics · 2023-12-15T05:56:17Z

What does this PR do?

Adds a automatic_embedding_tei_inference_endpoints notebook.

Fixes #4

Who can review?

Feel free to tag members/contributors who may be interested in your PR.
@MKhalusova

review-notebook-app · 2023-12-18T14:32:26Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

notebooks/automatic_embedding_tei_inference_endpoints.ipynb

MKhalusova · 2023-12-21T18:43:01Z

notebooks/automatic_embedding_tei_inference_endpoints.ipynb

@@ -0,0 +1,962 @@
+{


Hub 0.20.1 is already available, so we probably don't need this block of code anymore. Let's remove it then.

Reply via ReviewNB

MKhalusova · 2023-12-21T18:43:01Z

notebooks/automatic_embedding_tei_inference_endpoints.ipynb

@@ -0,0 +1,962 @@
+{


You can briefly explain what the repos are. E.g. DATASET_IN is where the text data is, DATASET_OUT is where we'll store the embeddings.
Normally, it's good to define the global variables all at the beginning, but I think it may make the notebook a bit easier to follow if the variables related to Inference Endpoint creation are declared in that section.

Reply via ReviewNB

MKhalusova · 2023-12-21T18:43:01Z

notebooks/automatic_embedding_tei_inference_endpoints.ipynb

@@ -0,0 +1,962 @@
+{


Why do we convert to pandas and to dict?

Reply via ReviewNB

This is unanswered.

MKhalusova · 2023-12-21T18:43:01Z

notebooks/automatic_embedding_tei_inference_endpoints.ipynb

@@ -0,0 +1,962 @@
+{


Line #11. revision="7302ac470bed880590f9344bfeee32ff8722d0e5",
What is this revision of?

Reply via ReviewNB

MKhalusova · 2023-12-21T18:43:02Z

notebooks/automatic_embedding_tei_inference_endpoints.ipynb

@@ -0,0 +1,962 @@
+{


"Its possible to have inputs that exceed the context. In those scenarios its up to you have to handle them. In my case Id like to truncate rather than have an error. Let's test that it works." =>
"You may have inputs that exceed the context. In such scenarios, it's up to you to handle them. In my case, I'd like to truncate rather than have an error. Let's test that it works."

Reply via ReviewNB

MKhalusova · 2023-12-21T18:43:02Z

notebooks/automatic_embedding_tei_inference_endpoints.ipynb

@@ -0,0 +1,962 @@
+{


"lets pause" -> "let's pause"

Reply via ReviewNB

MKhalusova · 2023-12-21T18:43:02Z

notebooks/automatic_embedding_tei_inference_endpoints.ipynb

@@ -0,0 +1,962 @@
+{


"I find its easiest" -> "I find it easiest"

Reply via ReviewNB

MKhalusova · 2023-12-21T18:43:02Z

notebooks/automatic_embedding_tei_inference_endpoints.ipynb

@@ -0,0 +1,962 @@
+{


Is the user you in this case? Maybe, clarify a little. E.g. "I'm uploading the dataset to the account of the user I logged in with, but....."

Reply via ReviewNB

MKhalusova · 2023-12-21T18:43:02Z

notebooks/automatic_embedding_tei_inference_endpoints.ipynb

@@ -0,0 +1,962 @@
+{


Maybe say a few words, why you would want to delete the endpoint?

Reply via ReviewNB

MKhalusova

This is a really cool and super useful example! Thanks for working on this.
I think we can bring it to the next level by polishing the text a bit. I found some typos, and it would be cool to add some additional explanations in a couple of places.
Please add the image to the https://huggingface.co/datasets/huggingface/cookbook-images, so that we don't store it in this repo.

datavistics · 2024-01-25T06:28:20Z

Hey @MKhalusova , I made some changes. Can you let me know what you think?

HuggingFaceDocBuilderDev · 2024-02-06T12:10:56Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

MKhalusova · 2024-02-19T14:15:02Z

notebooks/automatic_embedding_tei_inference_endpoints.ipynb

@@ -0,0 +1,845 @@
+{


Let's reserve H1 (#) header only for the notebook title, and make this an H2 (##)

Reply via ReviewNB

MKhalusova · 2024-02-19T14:15:02Z

notebooks/automatic_embedding_tei_inference_endpoints.ipynb

@@ -0,0 +1,845 @@
+{


Line #1. # !pip install -q -r ../requirements.txt
I think we can remove this comment

Reply via ReviewNB

MKhalusova · 2024-02-19T14:15:02Z

notebooks/automatic_embedding_tei_inference_endpoints.ipynb

@@ -0,0 +1,845 @@
+{


Line #5. MAX_WORKERS = 5 # This is for how many async workers you want. Choose based on the model and HW
What is HW?

Reply via ReviewNB

MKhalusova · 2024-02-19T14:15:02Z

notebooks/automatic_embedding_tei_inference_endpoints.ipynb

@@ -0,0 +1,845 @@
+{


Let's say "Hugging Face offers a number of GPUs that you can choose from" instead of "We have"

Reply via ReviewNB

MKhalusova · 2024-02-19T14:15:02Z

notebooks/automatic_embedding_tei_inference_endpoints.ipynb

@@ -0,0 +1,845 @@
+{


it's

Reply via ReviewNB

MKhalusova · 2024-02-19T14:18:00Z

There are a few important things that we need to address in this PR to be able to merge it:

Add the notebook to the _toctree.yml
Add the notebook to the list of the latest notebooks in the index.md. As this is the most recent addition, put it at the top of the list.
Move the notebook under notebooks/en
Make sure H1 (#) header is only used for the notebook title, and all the other headers are H2(##) or H3(###)
Right after the first header (notebook title), add yourself as an author, like this: _Authored by: [Your Name](https://huggingface.co/your_profile)_ Feel free to use either your Hugging Face profile, or GitHub profile, it's up to you which one to link.

I also left a few minor comments. Once these are addressed, we can merge.

…om_pretrained(..)

Co-authored-by: Maria Khalusova <[email protected]>

MKhalusova · 2024-02-19T14:55:44Z

notebooks/en/_toctree.yml

+  - local: automatic_embedding_tei_inference_endpoints
+    title: Automatic Embeddings with TEI through Inference Endpoints


index should be the top one here. Feel free to move it under index

Adding automatic_embedding_tei_inference_endpoints.ipynb

72ff7fd

MKhalusova self-requested a review December 18, 2023 14:20

MKhalusova reviewed Dec 21, 2023

View reviewed changes

datavistics added 2 commits January 25, 2024 10:22

Added changes from Maria's comments!

6836b6c

Moving image to dataset

0d92928

Fixing notebook to add maria's changes and update the image

b11b997

MKhalusova reviewed Feb 19, 2024

View reviewed changes

datavistics and others added 18 commits February 19, 2024 18:24

Adding latest changes

17605e5

Merge remote-tracking branch 'origin/main'

a5ccc30

Adding structure

0f01b42

a notebook for RAG with Zephyr and LangChain

9356f39

addressed feedback

31a490f

Add files via upload

a3aa032

image link updated

856bee6

code llm fine-tuning notebook

2e63b46

addressed feedback

33beb00

minor improvement: removed the base_model.cuda(), added .cuda() to fr…

6d911e2

…om_pretrained(..)

Created using Colaboratory

2c79913

Created using Colaboratory

040ea94

contribution guidelines

b0c2e18

removed emoji from the file name

208fce8

Add advanced RAG notebook

aee45be

Improve images

eca9259

Update graph

e9cb8da

Split notebook in two; finish advanced RAG notebook

1e0a7fb

aymeric-roucher and others added 27 commits February 19, 2024 18:51

Answer comments

df6683a

Update graphs

d86d8c7

Replace LLMs used in RAG evaluation

dfb48b0

Improve layout

66df885

removed unnecessary outputs (e.g. pip install)

b9b84c0

a bit more output cleanup

3396f78

Added GitHub workflows

38fb68b

moved notebook files under notebooks/en

14e0c19

Added _toctree.yml

86757eb

Two new notebooks added to _toctree.yml

3280090

rm unneeded .ipynb extensions

ff572db

shorter titles, added index.md

d6b0e04

placeholder

68e688b

better title for index

ae4a536

Add "convert_notebooks: true" to github workflows

a82a940

fix extra space " " in directory name

850a8e1

[wip] test doc preview

3561a25

Rm hub_base_path for now

e3627db

index.md content

c1d2e28

[Doc build] only run when necessary

5e3c095

Check notebooks are lowercase

d211029

user lowercase

d03f8bc

rm unneeded line

b5b52b1

Update .github/workflows/check_notebooks_lowercase.yml

ea63977

Co-authored-by: Maria Khalusova <[email protected]>

link fixed

98776d8

Adding structure

0b6fc60

Merge remote-tracking branch 'origin/main'

256740c

MKhalusova reviewed Feb 19, 2024

View reviewed changes

MKhalusova approved these changes Feb 19, 2024

View reviewed changes

MKhalusova merged commit fe78cf0 into huggingface:main Feb 19, 2024
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding automatic_embedding_tei_inference_endpoints.ipynb #5

Adding automatic_embedding_tei_inference_endpoints.ipynb #5

datavistics commented Dec 15, 2023

review-notebook-app bot commented Dec 18, 2023

MKhalusova Dec 21, 2023 •

edited

Loading

MKhalusova Dec 21, 2023 •

edited

Loading

MKhalusova Dec 21, 2023 •

edited

Loading

MKhalusova Feb 19, 2024

MKhalusova Dec 21, 2023 •

edited

Loading

MKhalusova Dec 21, 2023 •

edited

Loading

MKhalusova Dec 21, 2023 •

edited

Loading

MKhalusova Dec 21, 2023 •

edited

Loading

MKhalusova Dec 21, 2023 •

edited

Loading

MKhalusova Dec 21, 2023 •

edited

Loading

MKhalusova left a comment •

edited

Loading

datavistics commented Jan 25, 2024

HuggingFaceDocBuilderDev commented Feb 6, 2024

MKhalusova Feb 19, 2024 •

edited

Loading

MKhalusova Feb 19, 2024 •

edited

Loading

MKhalusova Feb 19, 2024 •

edited

Loading

MKhalusova Feb 19, 2024 •

edited

Loading

MKhalusova Feb 19, 2024 •

edited

Loading

MKhalusova commented Feb 19, 2024

MKhalusova Feb 19, 2024

		- local: automatic_embedding_tei_inference_endpoints
		title: Automatic Embeddings with TEI through Inference Endpoints

Adding automatic_embedding_tei_inference_endpoints.ipynb #5

Adding automatic_embedding_tei_inference_endpoints.ipynb #5

Conversation

datavistics commented Dec 15, 2023

What does this PR do?

Who can review?

review-notebook-app bot commented Dec 18, 2023

MKhalusova Dec 21, 2023 • edited Loading

Choose a reason for hiding this comment

MKhalusova Dec 21, 2023 • edited Loading

Choose a reason for hiding this comment

MKhalusova Dec 21, 2023 • edited Loading

Choose a reason for hiding this comment

MKhalusova Feb 19, 2024

Choose a reason for hiding this comment

MKhalusova Dec 21, 2023 • edited Loading

Choose a reason for hiding this comment

MKhalusova Dec 21, 2023 • edited Loading

Choose a reason for hiding this comment

MKhalusova Dec 21, 2023 • edited Loading

Choose a reason for hiding this comment

MKhalusova Dec 21, 2023 • edited Loading

Choose a reason for hiding this comment

MKhalusova Dec 21, 2023 • edited Loading

Choose a reason for hiding this comment

MKhalusova Dec 21, 2023 • edited Loading

Choose a reason for hiding this comment

MKhalusova left a comment • edited Loading

Choose a reason for hiding this comment

datavistics commented Jan 25, 2024

HuggingFaceDocBuilderDev commented Feb 6, 2024

MKhalusova Feb 19, 2024 • edited Loading

Choose a reason for hiding this comment

MKhalusova Feb 19, 2024 • edited Loading

Choose a reason for hiding this comment

MKhalusova Feb 19, 2024 • edited Loading

Choose a reason for hiding this comment

MKhalusova Feb 19, 2024 • edited Loading

Choose a reason for hiding this comment

MKhalusova Feb 19, 2024 • edited Loading

Choose a reason for hiding this comment

MKhalusova commented Feb 19, 2024

MKhalusova Feb 19, 2024

Choose a reason for hiding this comment

MKhalusova Dec 21, 2023 •

edited

Loading

MKhalusova Dec 21, 2023 •

edited

Loading

MKhalusova Dec 21, 2023 •

edited

Loading

MKhalusova Dec 21, 2023 •

edited

Loading

MKhalusova Dec 21, 2023 •

edited

Loading

MKhalusova Dec 21, 2023 •

edited

Loading

MKhalusova Dec 21, 2023 •

edited

Loading

MKhalusova Dec 21, 2023 •

edited

Loading

MKhalusova Dec 21, 2023 •

edited

Loading

MKhalusova left a comment •

edited

Loading

MKhalusova Feb 19, 2024 •

edited

Loading

MKhalusova Feb 19, 2024 •

edited

Loading

MKhalusova Feb 19, 2024 •

edited

Loading

MKhalusova Feb 19, 2024 •

edited

Loading

MKhalusova Feb 19, 2024 •

edited

Loading