diff --git a/00-course-setup/SETUP.md b/00-course-setup/SETUP.md index f733aa817..e7bee3be2 100644 --- a/00-course-setup/SETUP.md +++ b/00-course-setup/SETUP.md @@ -1,6 +1,6 @@ # Setup Your Dev Environment -We have instrumented this repository with a _dev container_ that comes with a Python 3 runtime. Simply open the repo in GitHub Codespaces or on your local Docker Desktop, to activate the runtime automatically. Then open th Jupyter notebook and select the Python 3.x kernel to prepare the Notebook for execution. +We have setup this repository and course with a _dev container_ that comes with a Python 3 runtime. Open the repo in GitHub Codespaces or on your local Docker Desktop, to activate the runtime automatically. Then open th Jupyter notebook and select the Python 3.x kernel to prepare the Notebook for execution. ## 1. Create `.env` file @@ -10,18 +10,20 @@ To configure this, we need to setup local environment variables for Azure as fol 1. Look in the root folder for a `.env.copy` file. It should contain a list of name-value pairs like this: - ```bash - AZURE_OPENAI_ENDPOINT='' - AZURE_OPENAI_DEPLOYMENT='' - AZURE_OPENAI_KEY='' - AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT='' - ``` + ```bash + AZURE_OPENAI_ENDPOINT='' + AZURE_OPENAI_DEPLOYMENT='' + AZURE_OPENAI_KEY='' + AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT='' + ``` 2. Make a copy of that file called `.env` using a command like this at the terminal: - ```bash - cp .env.copy .env - ``` - This should create an identical copy _except that this file is .gitignore-d and will never get checked into source control_. We can now populate **this .env file** with the environment variable values (secrets) without fear of them being checked in accidentally. You can now move to the next section to start populating these variables. + + ```bash + cp .env.copy .env + ``` + + This should create an identical copy _except that this file is .gitignore-d and will never get checked into source control_. We can now populate **this .env file** with the environment variable values (secrets) without fear of them being checked in accidentally. You can now move to the next section to start populating these variables. 3. (Option) If you use GitHub Codespaces, you have the option to save environment variables as _Codespaces secrets_ associated with this repository. In that case, you won't need to setup a local .env file. **However, note that this option works only if you use GitHub Codespaces.** You will still need to setup the .env file if you use Docker Desktop instead. @@ -31,19 +33,18 @@ The above steps should be executed also if you are using the non-Azure OpenAI en OPENAI_API_KEY='' ``` - ## 2. Populate `.env` file Let's take a quick look at the variable names to understand what they represent: -| Variable | Description | -|:---|:---| -|AZURE_OPENAI_ENDPOINT| This is the deployed endpoint for an Azure OpenAI resource| -|AZURE_OPENAI_KEY | This is the authorization key for using that service | -|OPENAI_API_KEY | This is the authorization key for using the service for non-Azure OpenAI endpoints | -|AZURE_OPENAI_DEPLOYMENT| This is the _text generation_ model deployment endpoint | -|AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT | This is the _text embeddings_ model deployment endpoint | -| | | +| Variable | Description | +| :--------------------------------- | :--------------------------------------------------------------------------------- | +| AZURE_OPENAI_ENDPOINT | This is the deployed endpoint for an Azure OpenAI resource | +| AZURE_OPENAI_KEY | This is the authorization key for using that service | +| OPENAI_API_KEY | This is the authorization key for using the service for non-Azure OpenAI endpoints | +| AZURE_OPENAI_DEPLOYMENT | This is the _text generation_ model deployment endpoint | +| AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT | This is the _text embeddings_ model deployment endpoint | +| | | For context, the last two variables refer to specific models that are used in chat completion (text generation model) and vector search (embeddings model) activities that are frequently used in generative AI applications. In the following sections, we'll locate the _values_ for these variables and set them in `.env` (replacing the content within the `' '`, but preserving the quotes). @@ -51,15 +52,16 @@ For context, the last two variables refer to specific models that are used in ch The Azure OpenAI endpoint and key values will be found in the [Azure Portal](https://portal.azure.com?WT.mc_id=academic-105485-koreyst) so let's start there. -1. Navigate to the [Azure Portal](https://portal.azure.com?WT.mc_id=academic-105485-koreyst) +1. Go to the [Azure Portal](https://portal.azure.com?WT.mc_id=academic-105485-koreyst) 1. Click the **Keys and Endpoint** option in the sidebar (menu at left). 1. Click **Show Keys** - you should see the following: KEY 1, KEY 2 and Endpoint. 1. Use the KEY 1 value for AZURE_OPENAI_KEY 1. Use the Endpoint value for AZURE_OPENAI_ENDPOINT Next, we need the endpoints for the specific models we've deployed. + 1. Click the **Model deployments** option in the sidebar (left menu) for Azure OpenAI resource. -1. In the destination page, click **Manage Deployments** +1. In the destination page, click **Manage Deployments** This will take you to the Azure OpenAI Studio website, where we'll find the other values as described below. @@ -74,11 +76,12 @@ This will take you to the Azure OpenAI Studio website, where we'll find the othe Now update the environment variables to reflect the _Deployment name_ used. This will typically be the same as the model name unless you changed it explcitly. So, as an example, you might have: ```bash -AZURE_OPENAI_DEPLOYMENT='gpt-35-turbo' +AZURE_OPENAI_DEPLOYMENT='gpt-35-turbo' AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT='text-embedding-ada-002' ``` **Don't forget to save the .env file when done**. You can now exit the file and return to the instructions for running the notebook. ### 2.3 Use OpenAI Public API -Your OpenAI API key can be found in your [OpenAI account](https://platform.openai.com/api-keys?WT.mc_id=academic-105485-koreyst). If you don't have one, you can sign up for an account and create an API key. Once you have the key, you can use it to populate the `OPENAI_API_KEY` variable in the `.env` file. \ No newline at end of file + +Your OpenAI API key can be found in your [OpenAI account](https://platform.openai.com/api-keys?WT.mc_id=academic-105485-koreyst). If you don't have one, you can sign up for an account and create an API key. Once you have the key, you can use it to populate the `OPENAI_API_KEY` variable in the `.env` file. diff --git a/04-prompt-engineering-fundamentals/1-introduction.ipynb b/04-prompt-engineering-fundamentals/aoai-assignment.ipynb similarity index 100% rename from 04-prompt-engineering-fundamentals/1-introduction.ipynb rename to 04-prompt-engineering-fundamentals/aoai-assignment.ipynb diff --git a/05-advanced-prompts/assignment.py b/05-advanced-prompts/python/aoai-assignment.py similarity index 100% rename from 05-advanced-prompts/assignment.py rename to 05-advanced-prompts/python/aoai-assignment.py diff --git a/05-advanced-prompts/solution.py b/05-advanced-prompts/python/aoai-solution.py similarity index 100% rename from 05-advanced-prompts/solution.py rename to 05-advanced-prompts/python/aoai-solution.py diff --git a/06-text-generation-apps/app-recipe.py b/06-text-generation-apps/python/aoai-app-recipe.py similarity index 100% rename from 06-text-generation-apps/app-recipe.py rename to 06-text-generation-apps/python/aoai-app-recipe.py diff --git a/06-text-generation-apps/app.py b/06-text-generation-apps/python/aoai-app.py similarity index 100% rename from 06-text-generation-apps/app.py rename to 06-text-generation-apps/python/aoai-app.py diff --git a/06-text-generation-apps/notebook-azure-openai.ipynb b/06-text-generation-apps/python/aoai-assignment.ipynb similarity index 100% rename from 06-text-generation-apps/notebook-azure-openai.ipynb rename to 06-text-generation-apps/python/aoai-assignment.ipynb diff --git a/06-text-generation-apps/history-bot.py b/06-text-generation-apps/python/aoai-history-bot.py similarity index 100% rename from 06-text-generation-apps/history-bot.py rename to 06-text-generation-apps/python/aoai-history-bot.py diff --git a/06-text-generation-apps/study-buddy.py b/06-text-generation-apps/python/aoai-study-buddy.py similarity index 100% rename from 06-text-generation-apps/study-buddy.py rename to 06-text-generation-apps/python/aoai-study-buddy.py diff --git a/06-text-generation-apps/openai/app-recipe.py b/06-text-generation-apps/python/oai-app-recipe.py similarity index 100% rename from 06-text-generation-apps/openai/app-recipe.py rename to 06-text-generation-apps/python/oai-app-recipe.py diff --git a/06-text-generation-apps/openai/app.py b/06-text-generation-apps/python/oai-app.py similarity index 100% rename from 06-text-generation-apps/openai/app.py rename to 06-text-generation-apps/python/oai-app.py diff --git a/06-text-generation-apps/openai/notebook-openai.ipynb b/06-text-generation-apps/python/oai-assigment.ipynb similarity index 100% rename from 06-text-generation-apps/openai/notebook-openai.ipynb rename to 06-text-generation-apps/python/oai-assigment.ipynb diff --git a/06-text-generation-apps/openai/history-bot.py b/06-text-generation-apps/python/oai-history-bot.py similarity index 100% rename from 06-text-generation-apps/openai/history-bot.py rename to 06-text-generation-apps/python/oai-history-bot.py diff --git a/06-text-generation-apps/openai/study-buddy.py b/06-text-generation-apps/python/oai-study-buddy.py similarity index 100% rename from 06-text-generation-apps/openai/study-buddy.py rename to 06-text-generation-apps/python/oai-study-buddy.py diff --git a/09-building-image-applications/app-variation.py b/09-building-image-applications/python/aoai-app-variation.py similarity index 100% rename from 09-building-image-applications/app-variation.py rename to 09-building-image-applications/python/aoai-app-variation.py diff --git a/09-building-image-applications/app.py b/09-building-image-applications/python/aoai-app.py similarity index 100% rename from 09-building-image-applications/app.py rename to 09-building-image-applications/python/aoai-app.py diff --git a/09-building-image-applications/notebook-azureopenai.ipynb b/09-building-image-applications/python/aoai-assignment.ipynb similarity index 100% rename from 09-building-image-applications/notebook-azureopenai.ipynb rename to 09-building-image-applications/python/aoai-assignment.ipynb diff --git a/09-building-image-applications/solution.py b/09-building-image-applications/python/aoai-solution.py similarity index 100% rename from 09-building-image-applications/solution.py rename to 09-building-image-applications/python/aoai-solution.py diff --git a/09-building-image-applications/generated-image.png b/09-building-image-applications/python/generated-image.png similarity index 100% rename from 09-building-image-applications/generated-image.png rename to 09-building-image-applications/python/generated-image.png diff --git a/09-building-image-applications/openai/app-variation.py b/09-building-image-applications/python/oai-app-variation.py similarity index 100% rename from 09-building-image-applications/openai/app-variation.py rename to 09-building-image-applications/python/oai-app-variation.py diff --git a/09-building-image-applications/openai/app.py b/09-building-image-applications/python/oai-app.py similarity index 100% rename from 09-building-image-applications/openai/app.py rename to 09-building-image-applications/python/oai-app.py diff --git a/09-building-image-applications/openai/notebook-openai.ipynb b/09-building-image-applications/python/oai-assignment.ipynb similarity index 100% rename from 09-building-image-applications/openai/notebook-openai.ipynb rename to 09-building-image-applications/python/oai-assignment.ipynb diff --git a/11-integrating-with-function-calling/Lesson11-FunctionCalling.ipynb b/11-integrating-with-function-calling/aoai-assignment.ipynb similarity index 100% rename from 11-integrating-with-function-calling/Lesson11-FunctionCalling.ipynb rename to 11-integrating-with-function-calling/aoai-assignment.ipynb diff --git a/13-continued-learning/README.md b/13-continued-learning/README.md deleted file mode 100644 index e75e95685..000000000 --- a/13-continued-learning/README.md +++ /dev/null @@ -1,135 +0,0 @@ -# Additional resources - -Here are links to other great resources to continue your learning and building with Generative AI. - -Are we missing a great resource? Let us know by submitting a PR! - -## 🧠 One Collection to Rule Them ALl - -After completing this course, check out our [Generative AI Learning collection](https://aka.ms/genai-collection?WT.mc_id=academic-105485-koreyst) to continue leveling up your Generative AI knowledge! - -## Lesson 1 - Introduction to Generative AI and LLMs - -🔗 [How GPT models work: accessible to everyone](https://bea.stollnitz.com/blog/how-gpt-works/?WT.mc_id=academic-105485-koreyst) - -🔗 [Fundamentals of Generative AI](https://learn.microsoft.com/training/modules/fundamentals-generative-ai?&WT.mc_id=academic-105485-koreyst) - -🔗 [How GPT models work: accessible to everyone](https://bea.stollnitz.com/blog/how-gpt-works?WT.mc_id=academic-105485-koreyst) - -🔗 [Generative AI: Implication and Applications for Education](https://arxiv.org/abs/2305.07605?wt.mc_id=github_S-1231_webpage_reactor) - -## Lesson 2 - Exploring and Comparing Different LLM types - -🔗 [How to use Open Source foundation models curated by Azure Machine Learning (preview) - Azure Machine Learning | Microsoft Learn](https://learn.microsoft.com/azure/machine-learning/how-to-use-foundation-models?WT.mc_id=academic-105485-koreyst) - -🔗 [The Large Language Model (LLM) Index | Sapling](https://sapling.ai/llm/index?WT.mc_id=academic-105485-koreyst) - -🔗 [[2304.04052] Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder (arxiv.org)](https://arxiv.org/abs/2304.04052?WT.mc_id=academic-105485-koreyst) - -🔗 [Retrieval Augmented Generation using Azure Machine Learning prompt flow](https://learn.microsoft.com/azure/machine-learning/concept-retrieval-augmented-generation?WT.mc_id=academic-105485-koreyst) - -🔗 [Grounding LLMs](https://techcommunity.microsoft.com/t5/fasttrack-for-azure/grounding-llms/ba-p/3843857?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [The Large Language Model (LLM) Index | Sapling](https://sapling.ai/llm/index?wt.mc_id=github_S-1231_webpage_reactor ) - -🔗 [[2304.04052] Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder (arxiv.org)](https://arxiv.org/abs/2304.04052?wt.mc_id=github_S-1231_webpage_reactor) - -## Lesson 3 - Using Generative AI Responsibly - -🔗 [Fundamentals of Responsible Generative AI](https://learn.microsoft.com/training/modules/responsible-generative-ai/?&WT.mc_id=academic-105485-koreyst) - -🔗 [Grounding LLMs](https://techcommunity.microsoft.com/t5/fasttrack-for-azure/grounding-llms/ba-p/3843857?WT.mc_id=academic-105485-koreyst) - -🔗 [Fundamentals of Responsible Generative AI](https://learn.microsoft.com/training/modules/responsible-generative-ai?WT.mc_id=academic-105485-koreyst) - -🔗 [Being Responsible with Generative AI](https://learn.microsoft.com/shows/ai-show/being-responsible-with-generative-ai?WT.mc_id=academic-105485-koreyst) - -🔗 [GPT-4 System Card](https://cdn.openai.com/papers/gpt-4-system-card.pdf?wt.mc_id=github_S-1231_webpage_reactor) - -## Lesson 4 - Understanding Prompt Engineering Fundamentals - -🔗 [Getting Started with Azure OpenAI Services](https://https://learn.microsoft.com/training/modules/get-started-openai/?&WT.mc_id=academic-105485-koreyst) - -[Apply Prompt Engineering with Azure OpenAI services](https://learn.microsoft.com/training/modules/apply-prompt-engineering-azure-openai/?&WT.mc_id=academic-105485-koreyst) - -[Introduction to Prompt Engineering](https://learn.microsoft.com/azure/ai-services/openai/concepts/prompt-engineering?&WT.mc_id=academic-105485-koreyst) - -🔗 [Prompt Engineering Overview](https://learn.microsoft.com/semantic-kernel/prompt-engineering/?WT.mc_id=academic-105485-koreyst) - -🔗 [Azure OpenAI for Education Prompts](https://techcommunity.microsoft.com/t5/education-blog/azure-openai-for-education-prompts-ai-and-a-guide-from-ethan-and/ba-p/3938259?wt.mc_id=github_S-1231_webpage_reactor ) - -🔗 [Introduction to Prompt Engineering](https://learn.microsoft.com/azure/ai-services/openai/concepts/prompt-engineering?WT.mc_id=academic-105485-koreyst) - -🔗 [Prompt Engineering Overview](https://learn.microsoft.com/semantic-kernel/prompt-engineering?WT.mc_id=academic-105485-koreyst) - -🔗 [Azure OpenAI for Education Prompts](https://techcommunity.microsoft.com/t5/e1.ucation-blog/azure-openai-for-education-prompts-ai-and-a-guide-from-ethan-and/ba-p/3938259?WT.mc_id=academic-105485-koreyst) - -## Lesson 5 - Creating Advanced Prompts - -🔗 [Prompt Engineering Techniques](https://learn.microsoft.com/azure/ai-services/openai/concepts/advanced-prompt-engineering?WT.mc_id=academic-105485-koreyst) - -## Lesson 6 - Building Text Generation Applications - -🔗 [Prompt Engineering Techniques](https://learn.microsoft.com/azure/ai-services/openai/concepts/advanced-prompt-engineering?pivots=programming-language-chat-completions&WT.mc_id=academic-105485-koreyst) - -## Lesson 7 - Building Chat Applications - -🔗 [System message framework and template recommendations for Large Language Models (LLMs)](https://learn.microsoft.com/azure/ai-services/openai/concepts/system-message?WT.mc_id=academic-105485-koreyst) - -🔗 [Learn how to work with the GPT-35-Turbo and GPT-4 models](https://learn.microsoft.com/azure/ai-services/openai/how-to/chatgpt?&WT.mc_id=academic-105485-koreyst) - -🔗 [Fine-Tuning language models from human preferences](https://arxiv.org/pdf/1909.08593.pdf?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [Build natural language solutions with Azure OpenAI Services](https://learn.microsoft.com/training/modules/build-language-solution-azure-openai/?WT.mc_id=academic-105485-koreyst) - - -🔗 [OpenAI Fine-Tuning](https://platform.openai.com/docs/guides/fine-tuning/when-to-use-fine-tuning?wt.mc_id=github_S-1231_webpage_reactor ) - -## Lesson 8 - Building Search Applications - -🔗 [Azure Cognitive Search](https://learn.microsoft.com/training/modules/improve-search-results-vector-search?WT.mc_id=academic-105485-koreyst) - -🔗 [OpenAI Embedding API](https://platform.openai.com/docs/api-reference/embeddings?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [Cosine Similarity](https://en.wikipedia.org/wiki/Cosine_similarity?wt.mc_id=github_S-1231_webpage_reactor) - -## Lesson 9 - Building Image Generation Applications - -🔗 [Generate Images with Azure OpenAI Service](https://learn.microsoft.com/training/modules/generate-images-azure-openai?WT.mc_id=academic-105485-koreyst) - -🔗 [OpenAI's DALL-E and CLIP 101: A Brief Introduction](https://towardsdatascience.com/openais-dall-e-and-clip-101-a-brief-introduction-3a4367280d4e?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [OpenAI's CLIP paper](https://arxiv.org/pdf/2103.00020.pdf?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [OpenAI's DALL-E and CLIP 101: A Brief Introduction](https://towardsdatascience.com/openais-dall-e-and-clip-101-a-brief-introduction-3a4367280d4e?WT.mc_id=academic-105485-koreyst) - -🔗 [OpenAI's CLIP paper](https://arxiv.org/pdf/2103.00020.pdf?WT.mc_id=academic-105485-koreyst) - -## Lesson 10 - Building Low Code AI Applications -🔗 [Create bots with Microsoft Copilot Studio](https://learn.microsoft.com/training/paths/work-power-virtual-agents/?&WT.mc_id=academic-105485-koreyst) - -🔗 [Add intelligence with AI Builder and GPT](https://learn.microsoft.com/training/modules/ai-builder-text-generation?&WT.mc_id=academic-105485-koreyst) - -🔗 [Get Started with AI Builder](https://learn.microsoft.com/training/modules/get-started-with-ai-builder?WT.mc_id=academic-105485-koreyst) - -🔗 [Detect Objects with AI Builder](https://learn.microsoft.com/training/modules/get-started-with-ai-builder-object-detection?WT.mc_id=academic-105485-koreyst) - -🔗 [Build a canvas app solution with Copilot in Power Apps](https://learn.microsoft.com/training/modules/build-canvas-app-real-estate-power-apps-copilot/?WT.mc_id=academic-105485-koreyst) - -🔗 [Power Platform Copilot Prompt Library](https://pnp.github.io/powerplatform-prompts/?wt.mc_id=github_S-1231_webpage_reactor&WT.mc_id=academic-109639-somelezediko) - -## Lesson 11- Integrating Applications with Function Calling - -🔗 [OpenAI Functions Documentation](https://learn.microsoft.com/azure/ai-services/openai/how-to/function-calling?WT.mc_id=academic-105485-koreyst) - -## Lesson 12 - Designing UX for AI Applications - -🔗 [Best practices for building collaborative UX with Human-AI partnership](https://learn.microsoft.com/community/content/best-practices-ai-ux?WT.mc_id=academic-105485-koreyst) - -🔗 [Designing Human-Centric AI Experiences: Applied UX Design for Artificial Intelligence by Akshay Kpre](https://www.linkedin.com/learning/ux-for-ai-design-practices-for-ai-developers?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [UX for AI: Design Practices for AI Developers](https://www.youtube.com/watch?wt.mc_id=github_S-1231_webpage_reactor&v=Lkbkd_hkDLY) - -🔗 [New skills in the age of AI by John Maeda](https://www.amazon.com/Designing-Human-Centric-Experiences-Artificial-Intelligence/dp/1484280873?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [Designing Human-Centric AI Experiences: Applied UX Design for Artificial Intelligence by Akshay Kpre](https://www.amazon.com/Designing-Human-Centric-Experiences-Artificial-Intelligence/dp/1484280873?WT.mc_id=academic-105485-koreyst) diff --git a/13-continued-learning/translations/cn/README.md b/13-continued-learning/translations/cn/README.md deleted file mode 100644 index 0504f6c24..000000000 --- a/13-continued-learning/translations/cn/README.md +++ /dev/null @@ -1,123 +0,0 @@ -# 拓展学习资源 - -以下是其他重要资源的链接，可帮助您继续学习和使用生成式 AI 进行构建。 - -我们是否错过了一些很棒的资源？ 通过提交 PR 让我们知道！ - -## 第一章 - 生成式人工智能和 LLMs 介绍 - -🔗 [How GPT models work: accessible to everyone](https://bea.stollnitz.com/blog/how-gpt-works/?WT.mc_id=academic-105485-koreyst) - -🔗 [Fundamentals of Generative AI](https://learn.microsoft.com/training/modules/fundamentals-generative-ai?&WT.mc_id=academic-105485-koreyst) - -🔗 [How GPT models work: accessible to everyone](https://bea.stollnitz.com/blog/how-gpt-works?WT.mc_id=academic-105485-koreyst) - -🔗 [Generative AI: Implication and Applications for Education](https://arxiv.org/abs/2305.07605?wt.mc_id=github_S-1231_webpage_reactor) - -## 第二章 - 探索和比较不同的 LLMs - -🔗 [How to use Open Source foundation models curated by Azure Machine Learning (preview) - Azure Machine Learning | Microsoft Learn](https://learn.microsoft.com/azure/machine-learning/how-to-use-foundation-models?WT.mc_id=academic-105485-koreyst) - -🔗 [The Large Language Model (LLM) Index | Sapling](https://sapling.ai/llm/index?WT.mc_id=academic-105485-koreyst) - -🔗 [[2304.04052] Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder (arxiv.org)](https://arxiv.org/abs/2304.04052?WT.mc_id=academic-105485-koreyst) - -🔗 [Retrieval Augmented Generation using Azure Machine Learning prompt flow](https://learn.microsoft.com/azure/machine-learning/concept-retrieval-augmented-generation?WT.mc_id=academic-105485-koreyst) - -🔗 [Grounding LLMs](https://techcommunity.microsoft.com/t5/fasttrack-for-azure/grounding-llms/ba-p/3843857?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [The Large Language Model (LLM) Index | Sapling](https://sapling.ai/llm/index?wt.mc_id=github_S-1231_webpage_reactor ) - -🔗 [[2304.04052] Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder (arxiv.org)](https://arxiv.org/abs/2304.04052?wt.mc_id=github_S-1231_webpage_reactor) - -## 第三章 - 负责任地使用生成式人工智能 - -🔗 [Fundamentals of Responsible Generative AI](https://learn.microsoft.com/training/modules/responsible-generative-ai/?&WT.mc_id=academic-105485-koreyst) - -🔗 [Grounding LLMs](https://techcommunity.microsoft.com/t5/fasttrack-for-azure/grounding-llms/ba-p/3843857?WT.mc_id=academic-105485-koreyst) - -🔗 [Fundamentals of Responsible Generative AI](https://learn.microsoft.com/training/modules/responsible-generative-ai?WT.mc_id=academic-105485-koreyst) - -🔗 [Being Responsible with Generative AI](https://learn.microsoft.com/shows/ai-show/being-responsible-with-generative-ai?WT.mc_id=academic-105485-koreyst) - -🔗 [GPT-4 System Card](https://cdn.openai.com/papers/gpt-4-system-card.pdf?wt.mc_id=github_S-1231_webpage_reactor) - -## 第四章 - 提示工程基础 - -🔗 [Introduction to Prompt Engineering](https://learn.microsoft.com/azure/ai-services/openai/concepts/prompt-engineering?&WT.mc_id=academic-105485-koreyst) - -🔗 [Prompt Engineering Overview](https://learn.microsoft.com/semantic-kernel/prompt-engineering/?WT.mc_id=academic-105485-koreyst) - -🔗 [Azure OpenAI for Education Prompts](https://techcommunity.microsoft.com/t5/education-blog/azure-openai-for-education-prompts-ai-and-a-guide-from-ethan-and/ba-p/3938259?wt.mc_id=github_S-1231_webpage_reactor ) - -🔗 [Introduction to Prompt Engineering](https://learn.microsoft.com/azure/ai-services/openai/concepts/prompt-engineering?WT.mc_id=academic-105485-koreyst) - -🔗 [Prompt Engineering Overview](https://learn.microsoft.com/semantic-kernel/prompt-engineering?WT.mc_id=academic-105485-koreyst) - -🔗 [Azure OpenAI for Education Prompts](https://techcommunity.microsoft.com/t5/e1.ucation-blog/azure-openai-for-education-prompts-ai-and-a-guide-from-ethan-and/ba-p/3938259?WT.mc_id=academic-105485-koreyst) - -## 第五章 - 创建高级的提示工程技巧 - -🔗 [Prompt Engineering Techniques](https://learn.microsoft.com/azure/ai-services/openai/concepts/advanced-prompt-engineering?WT.mc_id=academic-105485-koreyst) - -## 第六章 - 创建文本生成应用 - -🔗 [Prompt Engineering Techniques](https://learn.microsoft.com/azure/ai-services/openai/concepts/advanced-prompt-engineering?pivots=programming-language-chat-completions&WT.mc_id=academic-105485-koreyst) - -## 第七章 - 创建聊天应用 - -🔗 [System message framework and template recommendations for Large Language Models (LLMs)](https://learn.microsoft.com/azure/ai-services/openai/concepts/system-message?WT.mc_id=academic-105485-koreyst) - -🔗 [Learn how to work with the GPT-35-Turbo and GPT-4 models](https://learn.microsoft.com/azure/ai-services/openai/how-to/chatgpt?&WT.mc_id=academic-105485-koreyst) - -🔗 [Fine-Tuning language models from human preferences](https://arxiv.org/pdf/1909.08593.pdf?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [OpenAI Fine-Tuning](https://platform.openai.com/docs/guides/fine-tuning/when-to-use-fine-tuning?wt.mc_id=github_S-1231_webpage_reactor ) - -## 第八章 - 创建搜索应用 - -🔗 [Azure Cognitive Search](https://learn.microsoft.com/training/modules/improve-search-results-vector-search?WT.mc_id=academic-105485-koreyst) - -🔗 [OpenAI Embedding API](https://platform.openai.com/docs/api-reference/embeddings?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [Cosine Similarity](https://en.wikipedia.org/wiki/Cosine_similarity?wt.mc_id=github_S-1231_webpage_reactor) - -## 第九章- 创建图像生成应用 - -🔗 [Generate Images with Azure OpenAI Service](https://learn.microsoft.com/training/modules/generate-images-azure-openai?WT.mc_id=academic-105485-koreyst) - -🔗 [OpenAI's DALL-E and CLIP 101: A Brief Introduction](https://towardsdatascience.com/openais-dall-e-and-clip-101-a-brief-introduction-3a4367280d4e?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [OpenAI's CLIP paper](https://arxiv.org/pdf/2103.00020.pdf?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [OpenAI's DALL-E and CLIP 101: A Brief Introduction](https://towardsdatascience.com/openais-dall-e-and-clip-101-a-brief-introduction-3a4367280d4e?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [OpenAI's CLIP paper](https://arxiv.org/pdf/2103.00020.pdf?wt.mc_id=github_S-1231_webpage_reactor) - -## 第十章 - 创建低代码的人工智能应用 - -🔗 [Add intelligence with AI Builder and GPT](https://learn.microsoft.com/training/modules/ai-builder-text-generation?&WT.mc_id=academic-105485-koreyst) - -🔗 [Get Started with AI Builder](https://learn.microsoft.com/training/modules/get-started-with-ai-builder?WT.mc_id=academic-105485-koreyst) - -🔗 [Detect Objects with AI Builder](https://learn.microsoft.com/training/modules/get-started-with-ai-builder-object-detection?WT.mc_id=academic-105485-koreyst) - -🔗 [Build a canvas app solution with Copilot in Power Apps](https://learn.microsoft.com/training/modules/build-canvas-app-real-estate-power-apps-copilot/?WT.mc_id=academic-105485-koreyst) - -🔗 [Power Platform Copilot Prompt Library](https://pnp.github.io/powerplatform-prompts/?wt.mc_id=github_S-1231_webpage_reactor&WT.mc_id=academic-109639-somelezediko) - -## 第十一章 - 为生成式 AI 添加 function calling - -🔗 [OpenAI Functions Documentation](https://learn.microsoft.com/azure/ai-services/openai/how-to/function-calling?WT.mc_id=academic-105485-koreyst) - -## 第十二章 - 为人工智能应用程序添加用户体验 - -🔗 [Best practices for building collaborative UX with Human-AI partnership](https://learn.microsoft.com/community/content/best-practices-ai-ux?WT.mc_id=academic-105485-koreyst) - -🔗 [Designing Human-Centric AI Experiences: Applied UX Design for Artificial Intelligence by Akshay Kpre](https://www.linkedin.com/learning/ux-for-ai-design-practices-for-ai-developers?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [UX for AI: Design Practices for AI Developers](https://www.youtube.com/watch?wt.mc_id=github_S-1231_webpage_reactor&v=Lkbkd_hkDLY) - -🔗 [New skills in the age of AI by John Maeda](https://www.amazon.com/Designing-Human-Centric-Experiences-Artificial-Intelligence/dp/1484280873?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [Designing Human-Centric AI Experiences: Applied UX Design for Artificial Intelligence by Akshay Kpre](https://www.amazon.com/Designing-Human-Centric-Experiences-Artificial-Intelligence/dp/1484280873?WT.mc_id=academic-105485-koreyst) diff --git a/13-continued-learning/translations/ja-jp/README.md b/13-continued-learning/translations/ja-jp/README.md deleted file mode 100644 index 1331194c6..000000000 --- a/13-continued-learning/translations/ja-jp/README.md +++ /dev/null @@ -1,135 +0,0 @@ -# 追加のリソース - -以下は、生成 AI の学習と構築を続けるための他の素晴らしいリソースへのリンクです。 - -素晴らしいリソースが欠けている場合は、PRを提出してお知らせください！ - -## 🧠 すべてを制覇する一コレクション - -このコースを終了したら、[生成 AI 学習コレクション](https://aka.ms/genai-collection?WT.mc_id=academic-105485-yoterada)をチェックし、生成 AI の知識をさらに深めてください！ - -## レッスン1 - 生成 AI と大規模言語モデルの紹介 - -🔗 [GPTモデルの仕組み:誰でも理解できる](https://bea.stollnitz.com/blog/how-gpt-works/?WT.mc_id=academic-105485-yoterada) - -🔗 [生成 AI の基礎](https://learn.microsoft.com/training/modules/fundamentals-generative-ai?&WT.mc_id=academic-105485-yoterada) - -🔗 [GPTモデルの仕組み:誰でも理解できる](https://bea.stollnitz.com/blog/how-gpt-works?WT.mc_id=academic-105485-yoterada) - -🔗 [生成 AI:教育への影響と応用](https://arxiv.org/abs/2305.07605?wt.mc_id=github_S-1231_webpage_reactor) - -## レッスン2 - 様々なLLMの調査と比較 - -🔗 [Azure Machine Learningがキュレートするオープンソースの基盤モデルの使用方法 (プレビュー) - Azure Machine Learning | Microsoft Learn](https://learn.microsoft.com/azure/machine-learning/how-to-use-foundation-models?WT.mc_id=academic-105485-yoterada) - -🔗 [大規模言語モデル(LLM)インデックス | Sapling](https://sapling.ai/llm/index?WT.mc_id=academic-105485-yoterada) - -🔗 [[2304.04052] デコーダのみかエンコーダ-デコーダか？言語モデルを正則化されたエンコーダ-デコーダとして解釈する (arxiv.org)](https://arxiv.org/abs/2304.04052?WT.mc_id=academic-105485-yoterada) - -🔗 [Azure Machine Learningのプロンプトフローを使用した情報検索強化生成](https://learn.microsoft.com/azure/machine-learning/concept-retrieval-augmented-generation?WT.mc_id=academic-105485-yoterada) - -🔗 [LLMのグラウンディング](https://techcommunity.microsoft.com/t5/fasttrack-for-azure/grounding-llms/ba-p/3843857?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [大規模言語モデル(LLM)インデックス | Sapling](https://sapling.ai/llm/index?wt.mc_id=github_S-1231_webpage_reactor ) - -🔗 [[2304.04052] デコーダのみかエンコーダ-デコーダか？言語モデルを正則化されたエンコーダ-デコーダとして解釈する (arxiv.org)](https://arxiv.org/abs/2304.04052?wt.mc_id=github_S-1231_webpage_reactor) - -## レッスン3 - 責任ある生成 AI の利用 - -🔗 [責任ある生成 AI の基礎](https://learn.microsoft.com/training/modules/responsible-generative-ai/?&WT.mc_id=academic-105485-yoterada) - -🔗 [LLM のグラウンディング](https://techcommunity.microsoft.com/t5/fasttrack-for-azure/grounding-llms/ba-p/3843857?WT.mc_id=academic-105485-yoterada) - -🔗 [責任ある生成 AI の基礎](https://learn.microsoft.com/training/modules/responsible-generative-ai?WT.mc_id=academic-105485-yoterada) - -🔗 [生成 AI の責任ある使用](https://learn.microsoft.com/shows/ai-show/being-responsible-with-generative-ai?WT.mc_id=academic-105485-yoterada) - -🔗 [GPT-4システムカード](https://cdn.openai.com/papers/gpt-4-system-card.pdf?wt.mc_id=github_S-1231_webpage_reactor) - -## レッスン4 - プロンプト・エンジニアリングの基礎 - -🔗 [Azure OpenAIサービスの開始](https://https://learn.microsoft.com/training/modules/get-started-openai/?&WT.mc_id=academic-105485-yoterada) - -[Azure OpenAIサービスでのプロンプトエンジニアリングの適用](https://learn.microsoft.com/training/modules/apply-prompt-engineering-azure-openai/?&WT.mc_id=academic-105485-yoterada) - -[プロンプトエンジニアリングの紹介](https://learn.microsoft.com/azure/ai-services/openai/concepts/prompt-engineering?&WT.mc_id=academic-105485-yoterada) - -🔗 [プロンプトエンジニアリングの概要](https://learn.microsoft.com/semantic-kernel/prompt-engineering/?WT.mc_id=academic-105485-yoterada) - -🔗 [教育用Azure OpenAIプロンプト](https://techcommunity.microsoft.com/t5/education-blog/azure-openai-for-education-prompts-ai-and-a-guide-from-ethan-and/ba-p/3938259?wt.mc_id=github_S-1231_webpage_reactor ) - -🔗 [プロンプトエンジニアリングの紹介](https://learn.microsoft.com/azure/ai-services/openai/concepts/prompt-engineering?WT.mc_id=academic-105485-yoterada) - -🔗 [プロンプトエンジニアリングの概要](https://learn.microsoft.com/semantic-kernel/prompt-engineering?WT.mc_id=academic-105485-yoterada) - -🔗 [教育用Azure OpenAIプロンプト](https://techcommunity.microsoft.com/t5/e1.ucation-blog/azure-openai-for-education-prompts-ai-and-a-guide-from-ethan-and/ba-p/3938259?WT.mc_id=academic-105485-yoterada) - -## レッスン5 - 高度なプロンプトの作成 - -🔗 [プロンプトエンジニアリングの技術](https://learn.microsoft.com/azure/ai-services/openai/concepts/advanced-prompt-engineering?WT.mc_id=academic-105485-yoterada) - -## レッスン6 - テキスト生成アプリケーションの構築 - -🔗 [プロンプトエンジニアリングの技術](https://learn.microsoft.com/azure/ai-services/openai/concepts/advanced-prompt-engineering?pivots=programming-language-chat-completions&WT.mc_id=academic-105485-yoterada) - -## レッスン7 - チャットアプリケーションの構築 - -🔗 [大規模言語モデル(LLM)向けのシステムメッセージフレームワークとテンプレートの推奨事項](https://learn.microsoft.com/azure/ai-services/openai/concepts/system-message?WT.mc_id=academic-105485-yoterada) - -🔗 [GPT-35-TurboとGPT-4モデルの使用方法を学ぶ](https://learn.microsoft.com/azure/ai-services/openai/how-to/chatgpt?&WT.mc_id=academic-105485-yoterada) - -🔗 [人間の好みから言語モデルを微調整する](https://arxiv.org/pdf/1909.08593.pdf?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [Azure OpenAIサービスで自然言語ソリューションを構築する](https://learn.microsoft.com/training/modules/build-language-solution-azure-openai/?WT.mc_id=academic-105485-yoterada) - -🔗 [OpenAIの微調整](https://platform.openai.com/docs/guides/fine-tuning/when-to-use-fine-tuning?wt.mc_id=github_S-1231_webpage_reactor ) - -## レッスン8 - 検索アプリケーションの構築 - -🔗 [Azure Cognitive Search](https://learn.microsoft.com/training/modules/improve-search-results-vector-search?WT.mc_id=academic-105485-yoterada) - -🔗 [OpenAI埋め込みAPI](https://platform.openai.com/docs/api-reference/embeddings?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [コサイン類似度](https://en.wikipedia.org/wiki/Cosine_similarity?wt.mc_id=github_S-1231_webpage_reactor) - -## レッスン9 - 画像生成アプリケーションの構築 - -🔗[Azure OpenAIサービスで画像を生成する](https://learn.microsoft.com/training/modules/generate-images-azure-openai?WT.mc_id=academic-105485-yoterada) - -🔗 [OpenAIのDALL-EとCLIP 101:簡単な紹介](https://towardsdatascience.com/openais-dall-e-and-clip-101-a-brief-introduction-3a4367280d4e?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [OpenAIのCLIP論文](https://arxiv.org/pdf/2103.00020.pdf?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [OpenAIのDALL-EとCLIP 101:簡単な紹介](https://towardsdatascience.com/openais-dall-e-and-clip-101-a-brief-introduction-3a4367280d4e?WT.mc_id=academic-105485-yoterada) - -🔗 [OpenAIのCLIP論文](https://arxiv.org/pdf/2103.00020.pdf?WT.mc_id=academic-105485-yoterada) - -## レッスン10 - ローコード AI アプリケーションの構築 - -🔗 [Microsoft Copilot Studioでボットを作成する](https://learn.microsoft.com/training/paths/work-power-virtual-agents/?&WT.mc_id=academic-105485-yoterada) - -🔗 [AI BuilderとGPTを使用してインテリジェンスを追加する](https://learn.microsoft.com/training/modules/ai-builder-text-generation?&WT.mc_id=academic-105485-yoterada) - -🔗 [AI Builderの開始](https://learn.microsoft.com/training/modules/get-started-with-ai-builder?WT.mc_id=academic-105485-yoterada) - -🔗 [AI Builderでオブジェクトを検出する](https://learn.microsoft.com/training/modules/get-started-with-ai-builder-object-detection?WT.mc_id=academic-105485-yoterada) - -🔗 [Power AppsでCopilotを使用してキャンバスアプリソリューションを構築する](https://learn.microsoft.com/training/modules/build-canvas-app-real-estate-power-apps-copilot/?WT.mc_id=academic-105485-yoterada) - -🔗 [Power Platform Copilotプロンプトライブラリ](https://pnp.github.io/powerplatform-prompts/?wt.mc_id=github_S-1231_webpage_reactor&WT.mc_id=academic-109639-somelezediko) - -## レッスン11- Function Calling との統合 - -🔗 [OpenAI関数のドキュメンテーション](https://learn.microsoft.com/azure/ai-services/openai/how-to/function-calling?WT.mc_id=academic-105485-yoterada) - -## レッスン12 - AI アプリケーション用の UX デザイン - -🔗 [人間とAIのパートナーシップによる協調的UXを構築するためのベストプラクティス](https://learn.microsoft.com/community/content/best-practices-ai-ux?WT.mc_id=academic-105485-yoterada) - -🔗[人間中心のAI体験の設計:人工知能のための適用UX設計 by Akshay Kpre](https://www.linkedin.com/learning/ux-for-ai-design-practices-for-ai-developers?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [AI開発者のためのUX for AI:設計実践](https://www.youtube.com/watch?wt.mc_id=github_S-1231_webpage_reactor&v=Lkbkd_hkDLY) - -🔗 [AI時代の新しいスキル by John Maeda](https://www.amazon.com/Designing-Human-Centric-Experiences-Artificial-Intelligence/dp/1484280873?wt.mc_id=github_S-1231_webpage_reactor) - -🔗[人間中心のAI体験の設計:人工知能のための適用UX設計 by Akshay Kpre](https://www.amazon.com/Designing-Human-Centric-Experiences-Artificial-Intelligence/dp/1484280873?WT.mc_id=academic-105485-yoterada) diff --git a/13-continued-learning/translations/pt-br/README.md b/13-continued-learning/translations/pt-br/README.md deleted file mode 100644 index db1ec8335..000000000 --- a/13-continued-learning/translations/pt-br/README.md +++ /dev/null @@ -1,127 +0,0 @@ -# Recursos Adicionais - -Aqui estão links para outros ótimos recursos para continuar sua aprendizagem e construção com IA Generativa. - -Se você estiver sentindo falta de algum recurso, nos avise enviando um PR! - -## 🧠 Uma Coleção para Dominar tudo sobre o tópico - -Após completar este curso, confira nossa [Coleção de Aprendizagem de IA Generativa](https://aka.ms/genai-collection?WT.mc_id=academic-105485-koreyst) para continuar a aprimorar seu conhecimento em IA Generativa! - -## Lição 1 - Introdução à IA Generativa e LLMs - -🔗 [How GPT models work: accessible to everyone](https://bea.stollnitz.com/blog/how-gpt-works/?WT.mc_id=academic-105485-koreyst) - -🔗 [Fundamentals of Generative AI](https://learn.microsoft.com/training/modules/fundamentals-generative-ai?&WT.mc_id=academic-105485-koreyst) - -🔗 [How GPT models work: accessible to everyone](https://bea.stollnitz.com/blog/how-gpt-works?WT.mc_id=academic-105485-koreyst) - -🔗 [Generative AI: Implication and Applications for Education](https://arxiv.org/abs/2305.07605?wt.mc_id=github_S-1231_webpage_reactor) - -## Lição 2 - Explorando e Comparando Diferentes Tipos de LLM - -🔗 [How to use Open Source foundation models curated by Azure Machine Learning (preview) - Azure Machine Learning | Microsoft Learn](https://learn.microsoft.com/azure/machine-learning/how-to-use-foundation-models?WT.mc_id=academic-105485-koreyst) - -🔗 [The Large Language Model (LLM) Index | Sapling](https://sapling.ai/llm/index?WT.mc_id=academic-105485-koreyst) - -🔗 [[2304.04052] Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder (arxiv.org)](https://arxiv.org/abs/2304.04052?WT.mc_id=academic-105485-koreyst) - -🔗 [Retrieval Augmented Generation using Azure Machine Learning prompt flow](https://learn.microsoft.com/azure/machine-learning/concept-retrieval-augmented-generation?WT.mc_id=academic-105485-koreyst) - -🔗 [Grounding LLMs](https://techcommunity.microsoft.com/t5/fasttrack-for-azure/grounding-llms/ba-p/3843857?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [The Large Language Model (LLM) Index | Sapling](https://sapling.ai/llm/index?wt.mc_id=github_S-1231_webpage_reactor ) - -🔗 [[2304.04052] Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder (arxiv.org)](https://arxiv.org/abs/2304.04052?wt.mc_id=github_S-1231_webpage_reactor) - -## Lição 3 - Usando a IA Generativa de forma Responsável - -🔗 [Fundamentals of Responsible Generative AI](https://learn.microsoft.com/training/modules/responsible-generative-ai/?&WT.mc_id=academic-105485-koreyst) - -🔗 [Grounding LLMs](https://techcommunity.microsoft.com/t5/fasttrack-for-azure/grounding-llms/ba-p/3843857?WT.mc_id=academic-105485-koreyst) - -🔗 [Fundamentals of Responsible Generative AI](https://learn.microsoft.com/training/modules/responsible-generative-ai?WT.mc_id=academic-105485-koreyst) - -🔗 [Being Responsible with Generative AI](https://learn.microsoft.com/shows/ai-show/being-responsible-with-generative-ai?WT.mc_id=academic-105485-koreyst) - -🔗 [GPT-4 System Card](https://cdn.openai.com/papers/gpt-4-system-card.pdf?wt.mc_id=github_S-1231_webpage_reactor) - -## Lição 4 - Entendendo os Fundamentos da Engenharia de Prompt - -🔗 [Introduction to Prompt Engineering](https://learn.microsoft.com/azure/ai-services/openai/concepts/prompt-engineering?&WT.mc_id=academic-105485-koreyst) - -🔗 [Prompt Engineering Overview](https://learn.microsoft.com/semantic-kernel/prompt-engineering/?WT.mc_id=academic-105485-koreyst) - -🔗 [Azure OpenAI for Education Prompts](https://techcommunity.microsoft.com/t5/education-blog/azure-openai-for-education-prompts-ai-and-a-guide-from-ethan-and/ba-p/3938259?wt.mc_id=github_S-1231_webpage_reactor ) - -🔗 [Introduction to Prompt Engineering](https://learn.microsoft.com/azure/ai-services/openai/concepts/prompt-engineering?WT.mc_id=academic-105485-koreyst) - -🔗 [Prompt Engineering Overview](https://learn.microsoft.com/semantic-kernel/prompt-engineering?WT.mc_id=academic-105485-koreyst) - -🔗 [Azure OpenAI for Education Prompts](https://techcommunity.microsoft.com/t5/e1.ucation-blog/azure-openai-for-education-prompts-ai-and-a-guide-from-ethan-and/ba-p/3938259?WT.mc_id=academic-105485-koreyst) - -## Lição 5 - Criando Prompts Avançados - -🔗 [Prompt Engineering Techniques](https://learn.microsoft.com/azure/ai-services/openai/concepts/advanced-prompt-engineering?WT.mc_id=academic-105485-koreyst) - -## Lição 6 - Criando Aplicações de Geração de Texto - -🔗 [Prompt Engineering Techniques](https://learn.microsoft.com/azure/ai-services/openai/concepts/advanced-prompt-engineering?pivots=programming-language-chat-completions&WT.mc_id=academic-105485-koreyst) - -## Lição 7 - Criando Aplicações de Chat - -🔗 [System message framework and template recommendations for Large Language Models (LLMs)](https://learn.microsoft.com/azure/ai-services/openai/concepts/system-message?WT.mc_id=academic-105485-koreyst) - -🔗 [Learn how to work with the GPT-35-Turbo and GPT-4 models](https://learn.microsoft.com/azure/ai-services/openai/how-to/chatgpt?&WT.mc_id=academic-105485-koreyst) - -🔗 [Fine-Tuning language models from human preferences](https://arxiv.org/pdf/1909.08593.pdf?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [OpenAI Fine-Tuning](https://platform.openai.com/docs/guides/fine-tuning/when-to-use-fine-tuning?wt.mc_id=github_S-1231_webpage_reactor ) - -## Lição 8 - Criando Aplicações de Busca - -🔗 [Azure Cognitive Search](https://learn.microsoft.com/training/modules/improve-search-results-vector-search?WT.mc_id=academic-105485-koreyst) - -🔗 [OpenAI Embedding API](https://platform.openai.com/docs/api-reference/embeddings?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [Cosine Similarity](https://en.wikipedia.org/wiki/Cosine_similarity?wt.mc_id=github_S-1231_webpage_reactor) - -## Lição 9 - Criando Aplicações de Geração de Imagens - -🔗 [Generate Images with Azure OpenAI Service](https://learn.microsoft.com/training/modules/generate-images-azure-openai?WT.mc_id=academic-105485-koreyst) - -🔗 [OpenAI's DALL-E and CLIP 101: A Brief Introduction](https://towardsdatascience.com/openais-dall-e-and-clip-101-a-brief-introduction-3a4367280d4e?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [OpenAI's CLIP paper](https://arxiv.org/pdf/2103.00020.pdf?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [OpenAI's DALL-E and CLIP 101: A Brief Introduction](https://towardsdatascience.com/openais-dall-e-and-clip-101-a-brief-introduction-3a4367280d4e?WT.mc_id=academic-105485-koreyst) - -🔗 [OpenAI's CLIP paper](https://arxiv.org/pdf/2103.00020.pdf?WT.mc_id=academic-105485-koreyst) - -## Lição 10 - Criando Aplicações com Low Code - -🔗 [Add intelligence with AI Builder and GPT](https://learn.microsoft.com/training/modules/ai-builder-text-generation?&WT.mc_id=academic-105485-koreyst) - -🔗 [Get Started with AI Builder](https://learn.microsoft.com/training/modules/get-started-with-ai-builder?WT.mc_id=academic-105485-koreyst) - -🔗 [Detect Objects with AI Builder](https://learn.microsoft.com/training/modules/get-started-with-ai-builder-object-detection?WT.mc_id=academic-105485-koreyst) - -🔗 [Build a canvas app solution with Copilot in Power Apps](https://learn.microsoft.com/training/modules/build-canvas-app-real-estate-power-apps-copilot/?WT.mc_id=academic-105485-koreyst) - -🔗 [Power Platform Copilot Prompt Library](https://pnp.github.io/powerplatform-prompts/?wt.mc_id=github_S-1231_webpage_reactor&WT.mc_id=academic-109639-somelezediko) - -## Lição 11 - Integrando Aplicações com Chamadas de Função - -🔗 [OpenAI Functions Documentation](https://learn.microsoft.com/azure/ai-services/openai/how-to/function-calling?WT.mc_id=academic-105485-koreyst) - -## Lição 12 - Projetando UX para Aplicações de IA - -🔗 [Best practices for building collaborative UX with Human-AI partnership](https://learn.microsoft.com/community/content/best-practices-ai-ux?WT.mc_id=academic-105485-koreyst) - -🔗 [Designing Human-Centric AI Experiences: Applied UX Design for Artificial Intelligence by Akshay Kpre](https://www.linkedin.com/learning/ux-for-ai-design-practices-for-ai-developers?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [UX for AI: Design Practices for AI Developers](https://www.youtube.com/watch?wt.mc_id=github_S-1231_webpage_reactor&v=Lkbkd_hkDLY) - -🔗 [New skills in the age of AI by John Maeda](https://www.amazon.com/Designing-Human-Centric-Experiences-Artificial-Intelligence/dp/1484280873?wt.mc_id=github_S-1231_webpage_reactor) - -🔗 [Designing Human-Centric AI Experiences: Applied UX Design for Artificial Intelligence by Akshay Kpre](https://www.amazon.com/Designing-Human-Centric-Experiences-Artificial-Intelligence/dp/1484280873?WT.mc_id=academic-105485-koreyst) diff --git a/13-securing-ai-applications/README.md b/13-securing-ai-applications/README.md index 48e44d095..21ec4e6f5 100644 --- a/13-securing-ai-applications/README.md +++ b/13-securing-ai-applications/README.md @@ -24,17 +24,17 @@ As Artificial Intelligence (AI) and Machine Learning (ML) technologies increasin Here are key points to consider: -- **Impact of AI/ML**: AI and ML have significant impacts on daily life and as such safeguarding them has become essential. +- **Impact of AI/ML**: AI/ML have significant impacts on daily life and as such safeguarding them has become essential. - **Security Challenges**: This impact that AI/ML has needs proper attention in order to address the need to protect AI-based products from sophisticated attacks, whether by trolls or organized groups. - **Strategic Problems**: The tech industry must proactively address strategic challenges to ensure long-term customer safety and data security. Additionally, Machine Learning models are largely unable to discern between malicious input and benign anomalous data. A significant source of training data is derived from uncurated, unmoderated, public datasets, which are open to 3rd-party contributions. Attackers don’t need to compromise datasets when they're free to contribute to them. Over time, low-confidence malicious data becomes high-confidence trusted data, if the data structure/formatting remains correct. -This is why it is critical to ensure the integrity and protectiom of the data stores your models use to make decisions with. +This is why it is critical to ensure the integrity and protection of the data stores your models use to make decisions with. ## Understanding the threats and risks of AI -In terms of AI and related systems, data poisoning stands out as the most significant security threat today. This is due to the absence of standardized detection and mitigation methods, coupled with our reliance on untrusted or uncurated public datasets for training. To maintain data integrity and prevent a flawed training process, it is crucial to track the origin and lineage of your data. Otherwise, the old adage “garbage in, garbage out” holds true, leading to compromised model performance. +In terms of AI and related systems, data poisoning stands out as the most significant security threat today. Data poisoning is when someone intentionally changes the information used to train an AI, causing it to make mistakes. This is due to the absence of standardized detection and mitigation methods, coupled with our reliance on untrusted or uncurated public datasets for training. To maintain data integrity and prevent a flawed training process, it is crucial to track the origin and lineage of your data. Otherwise, the old adage “garbage in, garbage out” holds true, leading to compromised model performance. Here are examples of how data poisoning can affect your models: @@ -49,14 +49,14 @@ Here are examples of how data poisoning can affect your models: The MITRE Corporation has created [ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems)](https://atlas.mitre.org/?WT.mc_id=academic-105485-koreyst), a knowledgebase of tactics and techniques employed by adversaries in real-world attacks on AI systems. -> There are a growing number of vulnerabilities in AI-enabled systems, as the incorporation of AI increases the attack surface of existing systems beyond those of traditional cyber-attacks. We developed ATLAS to raise awareness of these unique and evolving vulnerabilities, as the global community starts to incorporate AI into more systems. ATLAS is modeled after the MITRE ATT&CK® framework and its tactics, techniques, and procedures (TTPs) are complementary to those in ATT&CK. +> There are a growing number of vulnerabilities in AI-enabled systems, as the incorporation of AI increases the attack surface of existing systems beyond those of traditional cyber-attacks. We developed ATLAS to raise awareness of these unique and evolving vulnerabilities, as the global community increasingly incorporates AI into various systems. ATLAS is modeled after the MITRE ATT&CK® framework and its tactics, techniques, and procedures (TTPs) are complementary to those in ATT&CK. Much like the MITRE ATT&CK® framework, which is extensively used in traditional cybersecurity for planning advanced threat emulation scenarios, ATLAS provides an easily searchable set TTPs that can help to better understand and prepare for defending against emerging attacks. Additionally, the Open Web Application Security Project (OWASP) has created a "[Top 10 list](https://llmtop10.com/?WT.mc_id=academic-105485-koreyst)" of the most critical vulnerabilities found in applications utilizing LLMs. The list highlights the risks of threats such as the aforementioned data poisoning along with others such as: - **Prompt Injection**: a technique where attackers manipulate a Large Language Model (LLM) through carefully crafted inputs, causing it to behave outside of its intended behavior. -- **Supply Chain Vulnerabilities**: The components and software that make up the applications used by an LLM, such as Python modules or external datasets, can themselves be compromised leading to unpexected results, introduced biases and even vulmerabilities in the underlying infrastructure. +- **Supply Chain Vulnerabilities**: The components and software that make up the applications used by an LLM, such as Python modules or external datasets, can themselves be compromised leading to unexpected results, introduced biases and even vulmerabilities in the underlying infrastructure. - **Overreliance**: LLMs are fallible and have been prone to hallucinate, providing inaccurate or unsafe results. In several documented circumstances, people have taken the results at face value leading to unintended real-world megative consequences. Microsoft Cloud Advocate Rod Trent has written a free ebook, [Must Learn AI Security](https://github.com/rod-trent/OpenAISecurity/tree/main/Must_Learn/Book_Version?WT.mc_id=academic-105485-koreyst), that dives deeply into these and other emerging AI threats and provides extensive guidance on how to best tackle these scenarios. @@ -65,7 +65,7 @@ Microsoft Cloud Advocate Rod Trent has written a free ebook, [Must Learn AI Secu Artificial intelligence (AI) is transforming various domains and industries, offering new possibilities and benefits for society. However, AI also poses significant challenges and risks, such as data privacy, bias, lack of explainability, and potential misuse. Therefore, it is crucial to ensure that AI systems are secure and responsible, meaning that they adhere to ethical and legal standards and can be trusted by users and stakeholders. -Security testing is the process of evaluating the security of an AI system or LLM by identifying and exploiting its vulnerabilities. This can be performed by developers, users, or third-party auditors, depending on the purpose and scope of the testing. Some of the most common security testing methods for AI systems and LLMs are: +Security testing is the process of evaluating the security of an AI system or LLM, by identifying and exploiting their vulnerabilities. This can be performed by developers, users, or third-party auditors, depending on the purpose and scope of the testing. Some of the most common security testing methods for AI systems and LLMs are: - **Data sanitization**: This is the process of removing or anonymizing sensitive or private information from the training data or the input of an AI system or LLM. Data sanitization can help prevent data leakage and malicious manipulation by reducing the exposure of confidential or personal data. - **Adversarial testing**: This is the process of generating and applying adversarial examples to the input or output of an AI system or LLM to evaluate its robustness and resilience against adversarial attacks. Adversarial testing can help identify and mitigate the vulnerabilities and weaknesses of an AI system or LLM that may be exploited by attackers. diff --git a/14-the-generative-ai-application-lifecycle/README.md b/14-the-generative-ai-application-lifecycle/README.md index c97ddef9e..f984edcaf 100644 --- a/14-the-generative-ai-application-lifecycle/README.md +++ b/14-the-generative-ai-application-lifecycle/README.md @@ -1,10 +1,13 @@ # The Generative AI Application Lifecycle -One important question that is important for all AI applications is about the relevance of the AI features, as AI is a fast envolving field, to ensure that your application remains relevant, reliable, and robust, you need to monitor, evaluate, and improve it continuously. This is where the generative AI lifecycle comes in. The generative AI lifecycle is a framework that guides you through the stages of developing, deploying, and maintaining a generative AI application. It helps you to define your goals, measure your performance, identify your challenges, and implement your solutions. It also helps you to align your application with the ethical and legal standards of your domain and your stakeholders. By following the generative AI lifecycle, you can ensure that your application is always delivering value and satisfying your users. +An important question for all AI applications is the relevance of AI features, as AI is a fast evolving field, to ensure that your application remains relevant, reliable, and robust, you need to monitor, evaluate, and improve it continuously. This is where the generative AI lifecycle comes in. + +The generative AI lifecycle is a framework that guides you through the stages of developing, deploying, and maintaining a generative AI application. It helps you to define your goals, measure your performance, identify your challenges, and implement your solutions. It also helps you to align your application with the ethical and legal standards of your domain and your stakeholders. By following the generative AI lifecycle, you can ensure that your application is always delivering value and satisfying your users. ## Introduction In this chapter, you will: + - Understand the Paradigm Shift from MLOps to LLMOps - The LLM Lifecycle - Lifecycle Tooling @@ -12,9 +15,9 @@ In this chapter, you will: ## Understand the Paradigm Shift from MLOps to LLMOps -LLMs are a new tool in the Artificial Intelligence arsenal, they are incredibly powerful in analysis and generation tasks for applications, however this power has some consequences in how we streamline AI and Classic Machine Learning tasks. +LLMs are a new tool in the Artificial Intelligence arsenal, they are incredibly powerful in analysis and generation tasks for applications, however this power has some consequences in how we streamline AI and Classic Machine Learning tasks. -With this, we need a new Paradigm to adapt this tool in a dynamic, with the correct incentives. We can categorize older AI apps as "ML Apps" and newer AI Apps as "GenAI Apps" or just "AI Apps", reflecting the mainstream technology and tecniques used at the time. This shifts our narrative in multiple ways, look at the following comparison. +With this, we need a new Paradigm to adapt this tool in a dynamic, with the correct incentives. We can categorize older AI apps as "ML Apps" and newer AI Apps as "GenAI Apps" or just "AI Apps", reflecting the mainstream technology and techniques used at the time. This shifts our narrative in multiple ways, look at the following comparison. ![LLMOps vs. MLOps comparison](./images/01-llmops-shift.png?WT.mc_id=academic-105485-koreys) @@ -23,18 +26,18 @@ Notice that in LLMOps, we are more focused in the App Developers, using integrat - Quality: Response quality - Harm: Responsible AI - Honesty: Response groundness (Makes sense? It is correct?) -- Cost: Solution Budger +- Cost: Solution Budget - Latency: Avg. time for token response ## The LLM Lifecycle -First, to understand the lifecycle and the modifications, let's note the next infographic. +First, to understand the lifecycle and the modifications, let's note the next infographic. ![LLMOps infographic](./images/02-llmops.png?WT.mc_id=academic-105485-koreys) As you may note, this is different from the usual Lifecycles from MLOps. LLMs have many new requirements, as Prompting, different tecniques to improve quality (Fine-Tuning, RAG, Meta-Prompts), different assessment and responsability with responsible AI, lastly, new evaluation metrics (Quality, Harm, Honesty, Cost and Latency). -For instance, take a look how we ideate. Using prompt engineering to experiment with various LLMs to explore possibilities to tests if their Hypothesis could be correct. +For instance, take a look how we ideate. Using prompt engineering to experiment with various LLMs to explore possibilities to tests if their Hypothesis could be correct. Note that this is not linear, but integrated loops, iterative and with an overacrching cycle. @@ -46,7 +49,7 @@ This may look a bit complicated, lets focus on the three big steps first. 1. Ideating/Exploring: Exploration, here we can explore according to our business needs. Prototyping, creating a [PromptFlow](https://microsoft.github.io/promptflow/index.html?WT.mc_id=academic-105485-koreyst) and test if is efficient enough for our Hypothesis. 1. Building/Augmenting: Implementation, now, we start to evaluate for bigger datasets implement tecnqiues, like Fine-tuning and RAG, to check the robustness of out solution. If it does not, re-implementing it, adding new steps in our flow or restructuring the data, might help. After testing our flow and our scale, if it works and check our Metrics, it is ready for the next step. -1. Operationalizing: Integration, now adding Monitoring and Alerts Systems to our system, deployment and application integration to our Application. +1. Operationalizing: Integration, now adding Monitoring and Alerts Systems to our system, deployment and application integration to our Application. Then, we have the overarching cycle of Management, focusing on security, compliance and governance. @@ -56,9 +59,9 @@ Now, what tools could we use? ## Lifecycle Tooling -For Tooling, Microsoft provides the [Azure AI Platform](https://azure.microsoft.com/solutions/ai/?WT.mc_id=academic-105485-koreys) and [PromptFlow](https://microsoft.github.io/promptflow/index.html?WT.mc_id=academic-105485-koreyst) facilitate and make your cycle easy to implement and ready to go. +For Tooling, Microsoft provides the [Azure AI Platform](https://azure.microsoft.com/solutions/ai/?WT.mc_id=academic-105485-koreys) and [PromptFlow](https://microsoft.github.io/promptflow/index.html?WT.mc_id=academic-105485-koreyst) facilitate and make your cycle easy to implement and ready to go. -The [Azure AI Platform](https://azure.microsoft.com/solutions/ai/?WT.mc_id=academic-105485-koreys), allows you to use [AI Studio](https://ai.azure.com/?WT.mc_id=academic-105485-koreys). AI Studio is a web portal allows you to Explore models, samples and tools. Managing your resources, UI development flows and SDK/CLI options for Code-First development. +The [Azure AI Platform](https://azure.microsoft.com/solutions/ai/?WT.mc_id=academic-105485-koreys), allows you to use [AI Studio](https://ai.azure.com/?WT.mc_id=academic-105485-koreys). AI Studio is a web portal allows you to Explore models, samples and tools. Managing your resources, UI development flows and SDK/CLI options for Code-First development. ![Azure AI possibilities](./images/04-azure-ai-platform.png?WT.mc_id=academic-105485-koreys) @@ -74,10 +77,9 @@ Construct, from Proof-of-Concept(POC) until large scale applications with Prompt ![LLMOps with PromptFlow](./images/06-llm-promptflow.png?WT.mc_id=academic-105485-koreys) - ## Great! Continue your Learning! -Amazing, now learn more how we structure an application to use the concepts with the [Contoso Chat App](https://nitya.github.io/contoso-chat/?WT.mc_id=academic-105485-koreys), to check how Cloud Advocacy adds those concepts in demonstations. For more content, check our [Ignite breakout session! +Amazing, now learn more how we structure an application to use the concepts with the [Contoso Chat App](https://nitya.github.io/contoso-chat/?WT.mc_id=academic-105485-koreyst), to check how Cloud Advocacy adds those concepts in demonstations. For more content, check our [Ignite breakout session! ](https://www.youtube.com/watch?v=DdOylyrTOWg) -Now, check Lesson 15, to understand how [Retrieval Augmented Generation and Vector Databases](#) impact Generative AI and to make more engaging Applications! \ No newline at end of file +Now, check Lesson 15, to understand how [Retrieval Augmented Generation and Vector Databases](#) impact Generative AI and to make more engaging Applications! diff --git a/15-rag-and-vector-databases/README.md b/15-rag-and-vector-databases/README.md index aed037d94..13b8161cf 100644 --- a/15-rag-and-vector-databases/README.md +++ b/15-rag-and-vector-databases/README.md @@ -2,7 +2,7 @@ -In the search applications lesson, we learnt briefly how you can integrate your own data into Large Language Models (LLMs). In this lesson, we will delve further into the concepts of grounding your data in your LLM application, the mechanics of the process and the methods for storing data, including both embeddings and text. +In the search applications lesson, we briefly learned how to integrate your own data into Large Language Models (LLMs). In this lesson, we will delve further into the concepts of grounding your data in your LLM application, the mechanics of the process and the methods for storing data, including both embeddings and text. > **Video Coming Soon** @@ -40,7 +40,7 @@ Users will be able to create practice quizzes from their notes, revision flash c ## Retrieval Augmented Generation (RAG) -An LLM powered chatbot processes user prompts to generate responses. It is designed to be interactive and engages with users on a wide array of topics. However, its responses are limited to the context provided and its foundational training data. For instance, GPT-4 knowledge cut off is September 2021, meaning, it lacks knowledge of events that have occurred after this period. In addition, the data used to train LLMs excludes confidential information such as personal notes or a company's product manual. +An LLM powered chatbot processes user prompts to generate responses. It is designed to be interactive and engages with users on a wide array of topics. However, its responses are limited to the context provided and its foundational training data. For instance, GPT-4 knowledge cutoff is September 2021, meaning, it lacks knowledge of events that have occurred after this period. In addition, the data used to train LLMs excludes confidential information such as personal notes or a company's product manual. ### How RAGs (Retrieval Augmented Generation) work @@ -62,7 +62,7 @@ The architecture for RAGs is implemented using transformers consisting of two pa Two approaches when implementing RAG according to the proposed paper: [Retrieval-Augmented Generation for Knowledge intensive NLP (natural language processing software) Tasks](https://arxiv.org/pdf/2005.11401.pdf?WT.mc_id=academic-105485-koreyst) are: -- ***RAG-Sequence*** using retrieved documents to predict the best possible answer to a user query +- **_RAG-Sequence_** using retrieved documents to predict the best possible answer to a user query - **RAG-Token** using documents to generate the next token, then retrieve them to answer the user's query @@ -82,7 +82,7 @@ Our application is based on our personal data i.e., the Neural Network lesson on A vector database, unlike traditional databases, is a specialized database designed to store, manage and search embedded vectors. It stores numerical representations of documents. Breaking down data to numerical embeddings makes it easier for our AI system to understand and process the data. -We store our embeddings in vector databases as LLMs have a limit of the number of tokens they accept as input. As you cannot pass the entire embeddings to an LLM, we will need to break them down into chunks and when a user asks a question, the embeddings most like the question will be returned together with the prompt. Chunking also reduces costs on the number of tokens passed through an LLM. +We store our embeddings in vector databases as LLMs have a limit of the number of tokens they accept as input. As you cannot pass the entire embeddings to an LLM, we will need to break them down into chunks and when a user asks a question, the embeddings most like the question will be returned together with the prompt. Chunking also reduces costs on the number of tokens passed through an LLM. Some popular vector databases include Azure Cosmos DB, Clarifyai, Pinecone, Chromadb, ScaNN,, Quadrants and DeepLake. You can create an Azure Cosmos DB model using Azure CLI with the following command: @@ -97,7 +97,7 @@ az cosmosdb list-keys -n -g Before we store our data, we will need to convert it to vector embeddings before it is stored in the database. If you are working with large documents or long texts, you can chunk them based on queries you expect. Chunking can be done at sentence level, or at a paragraph level. As chunking derives meanings from the words around them, you can add some other context to a chunk, for example, by adding the document title or including some text before or after the chunk. You can chunk the data as follows: -``` python +```python def split_text(text, max_length, min_length): words = text.split() chunks = [] @@ -116,7 +116,7 @@ def split_text(text, max_length, min_length): return chunks ``` -Once chunked, we can then embed our text using different embedding models. Some models you can use include: word2vec, ada-002 by OpenAI, Azure Computer Vision and many more. Selecting a model to use will depend on the languages you're using, the type of content encoded (text/images/audio), the size of input it can encode and length of the embedding output. +Once chunked, we can then embed our text using different embedding models. Some models you can use include: word2vec, ada-002 by OpenAI, Azure Computer Vision and many more. Selecting a model to use will depend on the languages you're using, the type of content encoded (text/images/audio), the size of input it can encode and length of the embedding output. An example of embedded text using OpenAI's `text-embedding-ada-002` model is: ![an embedding of the word cat](images/cat.png?WT.mc_id=academic-105485-koreyst) @@ -131,15 +131,15 @@ Retrieval happens when the system tries to quickly find the documents from the i There are several ways to perform search within our database such as: -- *Keyword* search - used for text searches +- **Keyword search** - used for text searches -- *Semantic* search - uses the semantic meaning of words +- **Semantic search** - uses the semantic meaning of words -- *Vector* search - converts documents from text to vector representations using embedding models. Retrieval will be done by querying the documents whose vector representations are closest to the user question. +- **Vector search** - converts documents from text to vector representations using embedding models. Retrieval will be done by querying the documents whose vector representations are closest to the user question. -- *Hybrid* - a combination of both keyword and vector search. +- **Hybrid** - a combination of both keyword and vector search. -A challenge with retrieval comes in when there is no similar response to the query in the database, the system will then return the best information they can get, however, you can use tactics like set up the maximum distance for relevance or use hybrid search that combines both keywords and vector search. In this lesson we will use hybrid search, a combination of both vector and keyword search. We will store our data into a dataframe with columns containing the chunks as well as embeddings. +A challenge with retrieval comes in when there is no similar response to the query in the database, the system will then return the best information they can get, however, you can use tactics like set up the maximum distance for relevance or use hybrid search that combines both keywords and vector search. In this lesson we will use hybrid search, a combination of both vector and keyword search. We will store our data into a dataframe with columns containing the chunks as well as embeddings. ### Vector Similarity @@ -236,13 +236,13 @@ chatbot(user_input) - Fluency - whether the response makes sense grammatically -## Use Cases for using RAG and vector databases +## Use Cases for using RAG (Retervival Augmented Generation) and vector databases There are many different use cases where function calls can improve your app like: - Question and Answering: grounding your company data to a chat that can be used by employees to ask questions. -- Recommendation Systems: where you can create a sysytem that matches the most similar values e.g. movies, restaurants and many more. +- Recommendation Systems: where you can create a system that matches the most similar values e.g. movies, restaurants and many more. - Chatbot services: you can store chat history and personalize the conversation based on the user data. @@ -264,4 +264,4 @@ Congratulations for completing the lesson 👏. ## Learning does not stop here, continue the Journey -After completing this lesson, check out our [Generative AI Learning collection](