-
Notifications
You must be signed in to change notification settings - Fork 2.9k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Container app updates, fast api updates, fixes, infra updates (#157)
* inital aca work * container config updates * Fixing container startup issues (#129) * Fixing container startup issues * Fixing import --------- Co-authored-by: Andre Dewes <[email protected]> * update arch diagram * fix get response * path fixes * fix chat request * fix env vars * fix path for ai search * update to fastapi, prompty updates, start eval work, * some deployment fixes * fix api deployment, fix endpoint name * update eval py to load model config * evalution notebook * move evals * update infra to ai studio * update model version to deploy to east us2 * update evaluate * remove unsued deployment files * update text * fix models for infra * fix aca name --------- Co-authored-by: Andre Dewes <[email protected]> Co-authored-by: Andre Dewes <[email protected]>
- Loading branch information
3 people
authored
Sep 9, 2024
1 parent
451994c
commit 49e52c7
Showing
93 changed files
with
68,355 additions
and
4,332 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,4 @@ | ||
| name: Evaluate Chat Relevance, Fluency, Coherence, and Groundedness | ||
| name: Evaluate | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
|
|
@@ -7,7 +7,6 @@ on: | |
| # Set this to the mainline branch you are using | ||
| branches: | ||
| - main | ||
| - azd | ||
|
|
||
| # Set up permissions for deploying with secretless Azure federated credentials | ||
| # https://learn.microsoft.com/en-us/azure/developer/github/connect-from-azure?tabs=azure-portal%2Clinux#set-up-azure-login-with-openid-connect-authentication | ||
|
|
@@ -23,17 +22,24 @@ jobs: | |
| AZURE_TENANT_ID: ${{ vars.AZURE_TENANT_ID }} | ||
| AZURE_SUBSCRIPTION_ID: ${{ vars.AZURE_SUBSCRIPTION_ID }} | ||
| AZURE_CREDENTIALS: ${{ secrets.AZURE_CREDENTIALS }} | ||
| AZURE_OPENAI_ENDPOINT: ${{ secrets.AZURE_OPENAI_ENDPOINT }} | ||
| AZURE_OPENAI_API_VERSION: ${{ secrets.AZURE_OPENAI_API_VERSION }} | ||
| AZURE_DEPLOYMENT_NAME: "gpt-4" | ||
| AZURE_EMBEDDING_NAME: "text-embedding-ada-002" | ||
| COSMOS_ENDPOINT: "${{ secrets.CONTOSO_SEARCH_ENDPOINT }}" | ||
| AZURE_SEARCH_ENDPOINT: "${{ secrets.AZURE_SEARCH_ENDPOINT }}" | ||
| AZURE_OPENAI_ENDPOINT: ${{ vars.AZURE_OPENAI_ENDPOINT }} | ||
| AZURE_OPENAI_API_VERSION: ${{ vars.AZURE_OPENAI_API_VERSION }} | ||
| AZURE_OPENAI_4_EVAL_DEPLOYMENT_NAME: ${{ vars.AZURE_OPENAI_4_EVAL_DEPLOYMENT_NAME }} | ||
| AZURE_OPENAI_DEPLOYMENT_NAME: ${{ vars.AZURE_OPENAI_DEPLOYMENT_NAME }} | ||
| AZURE_OPENAI_35_TURBO_DEPLOYMENT_NAME: ${{ vars.AZURE_OPENAI_35_TURBO_DEPLOYMENT_NAME }} | ||
| AZURE_RESOURCE_GROUP: ${{ vars.AZURE_RESOURCE_GROUP }} | ||
| AZURE_SEARCH_ENDPOINT: ${{ vars.AZURE_SEARCH_ENDPOINT }} | ||
| AZURE_OPENAI_NAME: ${{ vars.AZURE_OPENAI_NAME }} | ||
| BING_SEARCH_ENDPOINT: ${{ vars.BING_SEARCH_ENDPOINT }} | ||
| BING_SEARCH_KEY: ${{ secrets.BING_SEARCH_KEY }} | ||
|
|
||
| steps: | ||
| - name: checkout repo content | ||
| uses: actions/checkout@v4 # checkout the repository content | ||
|
|
||
| - name: Install azd | ||
| uses: Azure/[email protected] | ||
|
|
||
| - name: setup python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
|
|
@@ -42,64 +48,38 @@ jobs: | |
| - name: install python packages | ||
| run: | | ||
| python -m pip install --upgrade pip | ||
| pip install -r requirements.txt | ||
| - name: install promptflow dev bits | ||
| run: | | ||
| # unstall old promptflow | ||
| pip uninstall -y promptflow promptflow-azure promptflow-core promptflow-devkit promptflow-tools promptflow-evals | ||
| # install dev packages | ||
| pip install promptflow-evals==0.2.0.dev125831637 --extra-index-url https://azuremlsdktestpypi.azureedge.net/promptflow | ||
| pip install azure_ai_ml==1.16.0a20240501004 --extra-index-url https://pkgs.dev.azure.com/azure-sdk/public/_packaging/azure-sdk-for-python/pypi/simple/ | ||
| pip install azure-cli | ||
| pip install bs4 | ||
| pip install ipykernel | ||
| pip install azure-cosmos==4.6.0 | ||
| pip install azure-search-documents==11.4.0 | ||
| pf version | ||
| pip install -r src/api/requirements.txt | ||
| - name: Azure login | ||
| uses: azure/login@v2 | ||
| with: | ||
| client-id: ${{ env.AZURE_CLIENT_ID }} | ||
| tenant-id: ${{ env.AZURE_TENANT_ID }} | ||
| subscription-id: ${{ env.AZURE_SUBSCRIPTION_ID }} | ||
|
|
||
| - name: Set az account | ||
| uses: azure/CLI@v2 | ||
| with: | ||
| inlineScript: | | ||
| az account set --subscription ${{env.AZURE_SUBSCRIPTION_ID}} | ||
| - name: Set Promptflow config | ||
| run: | | ||
| pf config set trace.destination=azureml://subscriptions/${{ env.AZURE_SUBSCRIPTION_ID }}/resourceGroups/rg-${{ vars.AZURE_ENV_NAME }}/providers/Microsoft.MachineLearningServices/workspaces/${{ vars.ML_WORKSPACE_NAME }} | ||
| - name: evaluate chat data using the eval sdk | ||
| working-directory: ./evaluations | ||
| - name: evaluate | ||
| working-directory: ./src/api | ||
| run: | | ||
| python evaluations_chat.py | ||
| python -m evaluators.evaluate | ||
| - name: Upload eval results as build artifact | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: eval_result | ||
| path: ./evaluations/eval_result.jsonl | ||
| path: ./src/api/evaluators/eval_results.jsonl | ||
|
|
||
| - name: GitHub Summary Step | ||
| if: ${{ success() }} | ||
| working-directory: ./evaluations | ||
| working-directory: ./src/api | ||
| run: | | ||
| studio_url=$(cat studio_url.txt) | ||
| echo $studio_url | ||
| echo "🔗 [View in Azure Studio Here]($studio_url)" >> $GITHUB_STEP_SUMMARY | ||
| echo "" >> $GITHUB_STEP_SUMMARY | ||
| echo "📊 Promptflow Evaluation Results" >> $GITHUB_STEP_SUMMARY | ||
| cat eval_result.md >> $GITHUB_STEP_SUMMARY | ||
| echo "📊 Evaluation Results" >> $GITHUB_STEP_SUMMARY | ||
| cat evaluate/eval_results.md >> $GITHUB_STEP_SUMMARY | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,13 +1,6 @@ | ||
| # yaml-language-server: $schema=https://raw.githubusercontent.com/Azure/azure-dev/main/schemas/v1.0/azure.yaml.json | ||
|
|
||
| name: contoso-chat | ||
| name: contoso-chat-openai-prompty | ||
| metadata: | ||
| template: [email protected] | ||
| workflows: | ||
| up: | ||
| steps: | ||
| - azd: provision | ||
| - azd: deploy | ||
| template: [email protected] | ||
| hooks: | ||
| postprovision: | ||
| posix: | ||
|
|
@@ -20,27 +13,5 @@ hooks: | |
| continueOnError: false | ||
| interactive: true | ||
| run: infra/hooks/postprovision.ps1 | ||
|
|
||
|
|
||
| ################################################################ | ||
| # The section below configures the new `ai.endpoint` resource | ||
| # used for the Azure AI based chat service (application) | ||
| ################################################################ | ||
| services: | ||
| chat: | ||
| host: ai.endpoint | ||
| language: python | ||
| config: | ||
| workspace: ${AZUREAI_PROJECT_NAME} | ||
| environment: | ||
| path: ./deployment/environment.yaml | ||
| model: | ||
| path: ./deployment/chat-model.yaml | ||
| deployment: | ||
| path: ./deployment/chat-deployment.yaml | ||
| overrides: | ||
| environment_variables.PRT_CONFIG_OVERRIDE: deployment.subscription_id=${AZURE_SUBSCRIPTION_ID},deployment.resource_group=${AZURE_RESOURCE_GROUP},deployment.workspace_name=${AZUREAI_PROJECT_NAME},deployment.endpoint_name=${AZUREAI_ENDPOINT_NAME},deployment.deployment_name=${AZUREAI_DEPLOYMENT_NAME} | ||
| environment_variables.AZURE_OPENAI_ENDPOINT: ${AZURE_OPENAI_ENDPOINT} | ||
| environment_variables.AZURE_OPENAI_API_VERSION: ${AZURE_OPENAI_API_VERSION} | ||
| environment_variables.COSMOS_ENDPOINT: ${COSMOS_ENDPOINT} | ||
| environment_variables.AZURE_SEARCH_ENDPOINT: ${AZURE_SEARCH_ENDPOINT} | ||
| infra: | ||
| provider: "bicep" | ||
Oops, something went wrong.