Skip to content

HimaxWiseEyePlus/HIMAX_WE2_Rabboni

Repository files navigation

Himax examples for Seeed Grove Vision AI Module V2

This is a repository which step by step teaches you how to build your own examples and run on Seeed Grove Vision AI Module V2. Finally, teach you how to restore to the original factory settings and run SenseCraft AI from Seeed Studio.

Outline

How to build the firmware?

This part explains how you can build the firmware for Grove Vision AI Module V2.

Build the firmware at Linux environment

Note: The following has been tested to work on Ubuntu 20.04 PC

  • Step 1: Install the following prerequisites
    sudo apt install make
    
  • Step 2: Download Arm GNU Toolchain (arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi.tar.xz)
    cd ~
    wget https://developer.arm.com/-/media/Files/downloads/gnu/13.2.rel1/binrel/arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi.tar.xz
    
  • Step 3: Extract the file
    tar -xvf arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi.tar.xz
    
  • Step 4: Add arm-gnu-toolchain-13.2.Rel1-x86_64-arm-none-eabi/bin/: to PATH
    export PATH="$HOME/arm-gnu-toolchain-13.2.Rel1-x86_64-arm-none-eabi/bin/:$PATH"
    
  • Step 5: Clone the following repository and go into HIMAX_WE2_Rabboni folder
    git clone https://github.com/HimaxWiseEyePlus/HIMAX_WE2_Rabboni
    cd HIMAX_WE2_Rabboni
    
  • Step 6: Compile the firmware
    cd EPII_CM55M_APP_S
    make clean
    make
    
  • Output elf file: ./obj_epii_evb_icv30_bdv10/gnu_epii_evb_WLCSP65/EPII_CM55M_gnu_epii_evb_WLCSP65_s.elf alt text
  • Step 7: Generate firmware image file
    cd ../we2_image_gen_local/
    cp ../EPII_CM55M_APP_S/obj_epii_evb_icv30_bdv10/gnu_epii_evb_WLCSP65/EPII_CM55M_gnu_epii_evb_WLCSP65_s.elf input_case1_secboot/
    ./we2_local_image_gen project_case1_blp_wlcsp.json
    
  • Output firmware image: ./output_case1_sec_wlcsp/output.img alt text

Build the firmware at MacOS environment

Note: The steps are almost the same as the Linux environment except Step 7.

  • Step 7: Generate firmware image file (using ./we2_local_image_gen_macOS_arm64 for MacOS)
    cd ../we2_image_gen_local/
    cp ../EPII_CM55M_APP_S/obj_epii_evb_icv30_bdv10/gnu_epii_evb_WLCSP65/EPII_CM55M_gnu_epii_evb_WLCSP65_s.elf input_case1_secboot/
    ./we2_local_image_gen_macOS_arm64 project_case1_blp_wlcsp.json
    

Build the firmware at Windows environment

  • Step 1: Install the make command for prerequisites
  • Step 2: Download Arm GNU Toolchain arm-gnu-toolchain-13.2.rel1-mingw-w64-i686-arm-none-eabi.zip
  • Step 3: Extract the file
    tar -xvf arm-gnu-toolchain-13.2.rel1-mingw-w64-i686-arm-none-eabi.zip
    
  • Step 4: Add arm-gnu-toolchain-13.2.rel1-mingw-w64-i686-arm-none-eabi/bin/: to PATH
    setx PATH "%PATH%;[location of your gnu-toolchain-13.2 ROOT]\arm-gnu-toolchain-13.2.rel1-mingw-w64-i686-arm-none-eabi\bin"
    
  • Step 5: Clone the following repository and go into HIMAX_WE2_Rabboni folder
    git clone https://github.com/HimaxWiseEyePlus/HIMAX_WE2_Rabboni
    cd HIMAX_WE2_Rabboni
    
  • Step 6: Compile the firmware
    cd EPII_CM55M_APP_S
    make clean
    make
    
  • Output elf file: ./obj_epii_evb_icv30_bdv10/gnu_epii_evb_WLCSP65/EPII_CM55M_gnu_epii_evb_WLCSP65_s.elf alt text
  • Step 7: Generate firmware image file
    cd ../we2_image_gen_local/
    cp ../EPII_CM55M_APP_S/obj_epii_evb_icv30_bdv10/gnu_epii_evb_WLCSP65/EPII_CM55M_gnu_epii_evb_WLCSP65_s.elf input_case1_secboot/
    we2_local_image_gen project_case1_blp_wlcsp.json
    
  • Output firmware image: ./output_case1_sec_wlcsp/output.img alt text

How to flash the firmware?

This part explains how you can flash the firmware to Grove Vision AI Module V2.

System Requirement

  • Driver
    • If you find that the Grove Vision AI V2 is not recognised after connecting it to your computer, you should install the driver which can reference here.
  1. Grove Vision AI Module V2
  2. Connection cable
    • Micro usb cable: connect to EVB (as Power/UART)
  3. Software Tools Serial terminal emulation application
    • In the following description, TeraTerm and Minicom will be used.
      • Serial terminal emulation application Setting
        • Baud Rate 921600 bps
        • Data 8 bit
        • Parity none
        • Stop 1 bit
        • Flow control none
        • please check xmodem protocol is supported.
      • Minicom (for Linux PC)
        • Install minicom command
          sudo apt-get install minicom
          
        • Burn application to flash by using xmodem send application binary.
          • Minicom will extra install "lrzsz" package to support xmodem protocol alt text
          • If you did not have “lrzsz” instruction, please install by following instruction.
            sudo apt-get install lrzsz #(to support xmodem protocol)
            
        • Open the permissions to access the device
          sudo setfacl -m u:[USERNAME]:rw /dev/ttyUSB0
          # in my case
          # sudo setfacl -m u:kris:rw /dev/ttyACM0
          
          alt text
        • Open minicom
          sudo minicom -s
          
          alt text
        • Setup serial port and COM Port name alt text
          • Tips for finding the COM Port name.
            • You can use google chrome to connect to Seeed SenseCraft AI, select Grove Vision(V2) and press Connect. alt text
            • Then, you would get the COM Port name. alt text
      • TeraTerm (for Windows PC)
        • Setup serial port alt text alt text

Flash Image Update at Linux Environment

Following steps update application in the flash.

  • Step 1: Open Minicom, setup serial port and COM Port name-> connect to Grove Vision AI Module V2. (Please reference the minicom part of System Requirement) alt text
  • Step 2: Hold down any key on the keyboard (except the Enter key) and press the reset button to reset Grove Vision AI Module V2 and the startup options will be displayed. alt text
  • Step 3: Press button “1” and Grove Vision AI Module V2 will enter receiving mode after then.
    alt text
  • Step 4: Press Ctrl+A on keyboard to enter minicom menu, and then press s on keyboard to upload file and select xmodem. alt text
  • Step 5: Select the firmware image at HIMAX_WE2_Rabboni\we2_image_gen_local\output_case1_sec_wlcsp\output.img and press enter to burn. alt text
  • Step 6: After burning is compelete, press any key to be continue. alt text
  • Step 7: Then, you will see the message "Do you want to end file transmission and reboot system? (y)" is displayed. Press button y to restart. alt text
  • Step 8: You will see the uart on minicom which is runing your algorithm. alt text

Flash Image Update at Windows Environment

Following steps update application in the flash.

  • Step 1: Open TeraTerm and select File -> New connection, connect to Grove Vision AI Module V2. alt text
  • Step 2: Hold down any key on the keyboard (except the Enter key) and press the reset button to reset Grove Vision AI Module V2 and the startup options will be displayed. alt text
  • Step 3: Press button “1” and Grove Vision AI Module V2 will enter receiving mode after then. Select target flash image(output.img) by File->Transfer->XMODEM->Send. alt text
  • Step 4: After the firmware image burning is completed, the message "Do you want to end file transmission and reboot system? (y)" is displayed. Press button y to restart. alt text
  • Step 5: You will see the uart on TeraTerm which is runing your algorithm. alt text

Flash Image Update at Linux Environment by python code

  • Prerequisites for xmodem
  • Disconnect Minicom
  • Make sure your Seeed Grove Vision AI Module V2 is connect to PC.
  • Open the permissions to acceess the deivce
sudo setfacl -m u:[USERNAME]:rw /dev/ttyUSB0
# in my case
# sudo setfacl -m u:kris:rw /dev/ttyACM0

alt text

  • Open Terminal and key-in following command
  • port: the COM number of your Seeed Grove Vision AI Module V2, for example,/dev/ttyACM0
  • baudrate: 921600
  • file: your firmware image [maximum size is 1MB]
    python3 xmodem/xmodem_send.py --port=[your COM number] --baudrate=921600 --protocol=xmodem --file=we2_image_gen_local/output_case1_sec_wlcsp/output.img
    
    # example:
    # python3 xmodem/xmodem_send.py --port=/dev/ttyACM0 --baudrate=921600 --protocol=xmodem --file=we2_image_gen_local/output_case1_sec_wlcsp/output.img
    
  • It will start to burn firmware image. alt text
  • Please press reset buttun on Seeed Grove Vision AI Module V2. alt text
  • It will success to run the algorithm.

Flash Image Update at Windows Environment by python code

  • Prerequisites for xmodem
  • Disconnect Tera Term
  • Make sure your Seeed Grove Vision AI Module V2 is connect to PC.
  • Open CMD and key-in following command
  • port: the COM number of your Seeed Grove Vision AI Module V2
  • baudrate: 921600
  • file: your firmware image [maximum size is 1MB]
    python xmodem\xmodem_send.py --port=[your COM number] --baudrate=921600 --protocol=xmodem --file=we2_image_gen_local\output_case1_sec_wlcsp\output.img 
    # example:
    # python xmodem\xmodem_send.py --port=COM123 --baudrate=921600 --protocol=xmodem --file=we2_image_gen_local\output_case1_sec_wlcsp\output.img 
    
  • It will start to burn firmware image automatically. alt text
  • Please press reset buttun on Seeed Grove Vision AI Module V2. alt text
  • It will success to run the algorithm.

How to restore to the original factory settings

Linux Environment

  • Update the flash image Seeed_SenseCraft_AI*.img to Grove Vision AI Module V2 and press reset buttun. alt text

  • Disconnect the Minicom:

    • Please press Ctrl+A on keyboard and press z on keyboard to go to the menu of minicom. alt text
    • Then, press q on keyboard to quit with no reset minicom, and press yes to leave. alt text
  • Open the permissions to acceess the deivce

    sudo setfacl -m u:[USERNAME]:rw /dev/ttyUSB0
    # in my case
    # sudo setfacl -m u:kris:rw /dev/ttyACM0
    

    alt text

  • After doing the above steps, you can run the SenseCraft AI on Grove Vision AI Module V2.

    1. Introduction : https://wiki.seeedstudio.com/grove_vision_ai_v2/
    2. Connect Grove Vision AI Module to NB USB port
    3. Open "Google Chrome" browser
    4. Open SenseCraft Homepage
    5. Select "Grove Vision AI(WE2)" and connect (serial port) alt text alt text alt text

Windows Environment

  • Update the flash image Seeed_SenseCraft AI*.img to Grove Vision AI Module V2 and press reset buttun. alt text
  • Disconnect the TeraTerm. alt text
  • After doing the above steps, you can run the SenseCraft AI on Grove Vision AI Module V2.
    1. Introduction : https://wiki.seeedstudio.com/grove_vision_ai_v2/
    2. Install CH343 UART driver (CH343SER.ZIP) (Optional)
    3. Connect Grove Vision AI Module to NB USB port
    4. Open "Microsoft Edge" browser
    5. Open SenseCraft Homepage
    6. Select "Grove Vision AI(WE2)" and connect (serial port)

How to build face mesh scenario_app and run on WE2?

Linux Environment

  • Change the APP_TYPE to tflm_fd_fm at makefile
    APP_TYPE = tflm_fd_fm
    
  • Build the firmware reference the part of Build the firmware at Linux environment
  • How to flash firmware image and model at model_zoo?
    • Prerequisites for xmodem
    • Disconnect Minicom
    • Make sure your Seeed Grove Vision AI Module V2 is connect to PC.
    • Open the permissions to acceess the deivce
      sudo setfacl -m u:[USERNAME]:rw /dev/ttyUSB0
      # in my case
      # sudo setfacl -m u:kris:rw /dev/ttyACM0
      
      alt text
    • Open Terminal and key-in following command
      • port: the COM number of your Seeed Grove Vision AI Module V2, for example,/dev/ttyACM0
      • baudrate: 921600
      • file: your firmware image [maximum size is 1MB]
      • model: you can burn multiple models "[model tflite] [position of model on flash] [offset]"
        • Position of model on flash is defined at ~/tflm_fd_fm/common_config.h
          python3 xmodem/xmodem_send.py --port=[your COM number] --baudrate=921600 --protocol=xmodem --file=we2_image_gen_local/output_case1_sec_wlcsp/output.img --model="model_zoo/tflm_fd_fm/0_fd_0x200000.tflite 0x200000 0x00000" --model="model_zoo/tflm_fd_fm/1_fm_0x280000.tflite 0x280000 0x00000"  --model="model_zoo/tflm_fd_fm/2_il_0x32A000.tflite 0x32A000 0x00000"
          
          # example:
          # python3 xmodem/xmodem_send.py --port=/dev/ttyACM0 --baudrate=921600 --protocol=xmodem --file=we2_image_gen_local/output_case1_sec_wlcsp/output.img --model="model_zoo/tflm_fd_fm/0_fd_0x200000.tflite 0x200000 0x00000" --model="model_zoo/tflm_fd_fm/1_fm_0x280000.tflite 0x280000 0x00000"  --model="model_zoo/tflm_fd_fm/2_il_0x32A000.tflite 0x32A000 0x00000"
          
      • It will start to burn firmware image and model automatically. alt text
    • Please press reset buttun on Seeed Grove Vision AI Module V2. alt text
    • It will success to run the algorithm. alt text alt text

Windows Environment

  • Change the APP_TYPE to tflm_fd_fm at makefile
    APP_TYPE = tflm_fd_fm
    
  • Build the firmware reference the part of Build the firmware at Windows environment
  • How to flash firmware image and model at model_zoo?
    • Prerequisites for xmodem
    • Disconnect Tera Term
    • Make sure your Seeed Grove Vision AI Module V2 is connect to PC.
    • Open CMD and key-in following command
      • port: the COM number of your Seeed Grove Vision AI Module V2
      • baudrate: 921600
      • file: your firmware image [maximum size is 1MB]
      • model: you can burn multiple models "[model tflite] [position of model on flash] [offset]"
        • Position of model on flash is defined at ~/tflm_fd_fm/common_config.h
          python xmodem\xmodem_send.py --port=[your COM number] --baudrate=921600 --protocol=xmodem --file=we2_image_gen_local\output_case1_sec_wlcsp\output.img --model="model_zoo\tflm_fd_fm\0_fd_0x200000.tflite 0x200000 0x00000" --model="model_zoo\tflm_fd_fm\1_fm_0x280000.tflite 0x280000 0x00000"  --model="model_zoo\tflm_fd_fm\2_il_0x32A000.tflite 0x32A000 0x00000"
          
          # example:
          # python xmodem\xmodem_send.py --port=COM123 --baudrate=921600 --protocol=xmodem --file=we2_image_gen_local\output_case1_sec_wlcsp\output.img --model="model_zoo\tflm_fd_fm\0_fd_0x200000.tflite 0x200000 0x00000" --model="model_zoo\tflm_fd_fm\1_fm_0x280000.tflite 0x280000 0x00000"  --model="model_zoo\tflm_fd_fm\2_il_0x32A000.tflite 0x32A000 0x00000"
          
      • It will start to burn firmware image and model automatically. alt text
    • Please press reset buttun on Seeed Grove Vision AI Module V2. alt text
    • It will success to run the algorithm. alt text alt text

Send image and meta data by UART

  • Disconnect the uart at your Tera Term or Minicom first.
  • You can use the Himax AI web toolkit which we provide, download it and unzip it to local PC, and double click index.html.
  • Please check you select Grove Vision AI(V2) and press connect button alt text
  • Select your own COM. alt text
  • You will see the preview result on website. alt text
  • Tip
    • Windows:
      • Please use "Microsoft Edge" browser
    • Linux:
      • Open the permissions to acceess the deivce
        sudo setfacl -m u:[USERNAME]:rw /dev/ttyUSB0
        # in my case
        # sudo setfacl -m u:kris:rw /dev/ttyACM0
        
      • Please use "Google Chrome" browser

Model source link

How to add support for raspberry pi camera?

You can reference the scenario app allon_sensor_tflm , allon_sensor_tflm_freertos and tflm_fd_fm. Take allon_sensor_tflm for example, you should only modify the allon_sensor_tflm.mk from cis_ov5647 to cis_imx219 or cis_imx477.

#CIS_SUPPORT_INAPP_MODEL = cis_ov5647
CIS_SUPPORT_INAPP_MODEL = cis_imx219
#CIS_SUPPORT_INAPP_MODEL = cis_imx477

So that, it can support cis_imx219 or cis_imx477 camera.

How to use CMSIS-NN at the project?

  • Modify the setting at the makefile
    • Enable the flag LIB_CMSIS_NN_ENALBE to build CMSIS-NN library
      LIB_CMSIS_NN_ENALBE = 1
      
    • You can reference the scenario app example about allon_sensor_tflm_cmsis_nn which is the example running the model without passing vela and using the CMSIS-NN library.
      • Change the APP_TYPE to allon_sensor_tflm_cmsis_nn at the makefile
        APP_TYPE = allon_sensor_tflm_cmsis_nn
        

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published