GPT-RAG — Orchestrator Component

Note

A part of GPT-RAG.

Table of Contents

• Orchestrator Process
  • Orchestration Flow
• How-to: Developer
  • Redeploying the Orchestrator Component
  • Alternative Deployment of the Orchestrator Component
  • Running Locally
  • Evaluating the Orchestrator Component

Orchestrator Process

The Enterprise RAG Orchestrator efficiently manages user interactions by coordinating various modules and plugins to generate accurate responses. The core of its functionality revolves around the get_answer function in code_orchestration.py, which processes user queries through a structured workflow.

Orchestration Flow

1. Initialization

   • Conversation Management: Retrieves or creates a conversation record from Cosmos DB using the conversation_id.
   • Setup: Initializes necessary variables, loads the bot description, and prepares the Semantic Kernel for processing.

2. Processing User Input (get_answer function)

   • Input Handling: Captures the latest user query and appends it to the conversation history.
   • Guardrails: Performs initial checks, such as filtering blocked words to ensure content compliance.
   • RAG Flow:
     • Language Detection: Identifies the language of the user input.
     • Conversation Summarization: Summarizes the conversation history to maintain context.
     • Intent Triage: Determines the intent behind the user query (e.g., question answering, follow-up).
     • Data Retrieval: Utilizes retrieval plugins to fetch relevant information based on the identified intent.
     • Answer Generation: Generates a coherent response by integrating retrieved data and conversation context.
   • Final Guardrails: Ensures the generated answer meets quality standards by checking for blocked content and grounding.

3. Response Synthesis and Delivery

   • Updating Conversation: Saves the generated answer and relevant metadata back to Cosmos DB.
   • Delivery: Formats and sends the response to the user, completing the interaction cycle.

How-to: Developer

Redeploying the Orchestrator Component

• Provision the infrastructure and deploy the solution using the GPT-RAG template.

• Redeployment Steps:

  • Prerequisites:

    • Azure Developer CLI: Download azd for Windows, Other OS's.
    • PowerShell 7+ (Windows only): PowerShell.
    • Git: Download Git.
    • Python 3.11: Download Python.

• Redeployment Commands:

  azd auth login
  azd env refresh
  azd deploy

Important

Use the same environment name, subscription, and region as the initial deployment when running azd env refresh.

Alternative Deployment of the Orchestrator Component

If you deployed the GPT-RAG infrastructure manually, you can use Azure Functions Core Tools as an alternative to azd for deployment:

func azure functionapp publish FUNCTION_APP_NAME

After completing the deployment, run the following command to confirm that the function was successfully deployed:

func azure functionapp list-functions FUNCTION_APP_NAME

Note

Replace FUNCTION_APP_NAME with the name of your Orchestrator Function App before running the command.

Running Locally

For more information on how to test the Orchestrator component locally using VS Code, refer to the Local Deployment guide.

Evaluating the Orchestrator Component

For more information on how to evaluate the Orchestrator's performance, refer to the Load Testing the LLM Orchestrator App guide.