Skip to content

This repo hosts the code of a smart Javascript application integrating image analysis and generation capabilities

Notifications You must be signed in to change notification settings

carlotta94c/Analyze-and-generate-images-with-cognitive-services

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

19 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Analyze and generate images with Cognitive Services

This repository hosts the code of a smart Javascript application integrating image analysis and generation capabilities.

Empower your app with Computer Vision

Are you interested in how AI perceives the world around us? Do you know how AI can generate a picture of a seaside landscape in a photorealistic style? Look at this demo, to learn how you can embed computer vision capabilities into your applications using Azure services.

Pre-requisites

To run the app locally

To set up your local environment and run the web app you need to configure the following pre-requisites:

To deploy the app on the Cloud

In addition to the pre-requisites above, if you wish to deploy and run your app on Azure you'll need the following:

Description

The goal of this demo is to showcase how to empower your app with Computer Vision capabilities.

App functionalities

The React app of this demo provides the following functionalities:

  • Displays message if Azure key and endpoint for Cognitive Services Computer Vision or Azure OpenAI service isn't found. Key and endpoint not found

  • Allows you to analyze an image with Cognitive Services Computer Vision, by entering a public image URL or analyzing an image from collection.

    • When analysis is complete: it displays the analyzed image and Computer Vision JSON results.

      Image analysis result

  • Allows you to generate an image with DALL-E 2, by entering a textual prompt.

    • When generation process is complete: it displays the generated image and the corresponding url.

      Image generation result

Step-by-step guide

To run the app locally

  1. Fork this repository and clone it on your machine.

  2. Install all the required dependencies by running npm install in a terminal or bash window, from the project folder.

  3. Get your Computer Vision credentials and add them as environment variables in a terminal (use export rather than set if you are in a bash window).

    set REACT_APP_AZURE_COMPUTER_VISION_KEY=REPLACE-WITH-YOUR-KEY
    set REACT_APP_AZURE_COMPUTER_VISION_ENDPOINT=REPLACE-WITH-YOUR-ENDPOINT
  4. Get your OpenAI instance credentials and add them as environment variables in a terminal (use export rather than set if you are in a bash window).

    set REACT_APP_AZURE_OPENAI_KEY=REPLACE-WITH-YOUR-KEY
    set REACT_APP_AZURE_OPENAI_ENDPOINT=REPLACE-WITH-YOUR-ENDPOINT

If you are using an Azure OpenAI endpoint, you can retrieve your credentials from the Keys and Endpoint tab of your resource in the Azure Portal.

On the other hand, if you are using an OpenAI instance, you can retrieve your key from your user page, while your endpoint is https://api.openai.com/v1/.

To deploy the app on the Cloud

  1. Add environment variables to your remote environment, to enable GitHub actions to pass the secrets to the Static Web App. In a web browser, on your GitHub repository, select Settings -> Secrets and variables -> Actions -> New repository secrets. Enter the same name and value for the keys and endpoints you used in the previous section.

  2. In Visual Studio Code, open the project folder and select the Azure icon in the left-hand side menu. Select Static Web Apps -> Create New Static Web App. Select the GitHub repository you forked and the branch you want to deploy. Select React as the build preset.

  3. Pull down the changes made from creating the Azure resource to get the GitHub Action file. In the Visual Studio Code editor, edit the GitHub Action file found at ./.github/workflows/ to add the secrets (under the build_and_deploy_job).

    env:
      REACT_APP_AZURE_COMPUTER_VISION_ENDPOINT: ${{secrets.REACT_APP_AZURE_COMPUTER_VISION_ENDPOINT}}
      REACT_APP_AZURE_COMPUTER_VISION_KEY:  ${{secrets.REACT_APP_AZURE_COMPUTER_VISION_KEY}}
      REACT_APP_AZURE_OPENAI_ENDPOINT: ${{secrets.REACT_APP_AZURE_OPENAI_ENDPOINT}}
      REACT_APP_AZURE_OPENAI_KEY:  ${{secrets.REACT_APP_AZURE_OPENAI_KEY}} 
    
  4. Add and commit the change to the local main branch. Push the change to the remote repository, starting a new build-and-deploy action to your Azure Static web app.

  5. Monitor the build-and-deploy action in the Actions tab of your GitHub repository. Once the action is complete, you can view your deployed app by selecting the Browse to URL link in the Azure Static Web Apps section of the action.

Switching from an Azure OpenAI endpoint to OpenAI endpoint and viceversa

Azure OpenAI Services API and OpenAI API handles the construction of the endpoint and of the authentication header differently. This application can run with both services, but you need to set the api type to 'openai' in your environment variables if you are using non-Azure OpenAI endpoints (default is using Azure).

set REACT_APP_AZURE_OPENAI_API_TYPE=openai

Not sure what's the difference between the two services? Learn more here.

Troubleshooting

Are you getting ERR_OSSL_EVP_UNSUPPORTED error when running your app locally? This error message indicates that something within your web development stack isn’t compatible with the OpenSSL files you’re running.

This issue occurs due to an inconsistency with your development tools and the version of OpenSSL that you’re running. To fix this, you need to upgrade your development tools (Node.js, React.js, etc.) to the latest version. You can find a complete guide here.

If you don’t want to upgrade them for any reason, you can simply enable the OpenSSL 3.0 legacy provider as a workaround, with the command:

set NODE_OPTIONS=--openssl-legacy-provider

Useful Resources

About

This repo hosts the code of a smart Javascript application integrating image analysis and generation capabilities

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published