-
Notifications
You must be signed in to change notification settings - Fork 27
add aitk whisper sample (step 1); check dataset #209
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This pull request adds AITK (AI Toolkit) sample files for the OpenAI Whisper Large v3 Turbo model, enabling model optimization and inference using ONNX Runtime with the QNN Execution Provider. The PR includes Python scripts for workflow orchestration, model loading, and inference, along with configuration files for FP32 and quantized model generation.
Key changes:
- Implementation of Whisper model encoder/decoder loading and inference logic
- QNN workflow orchestration with support for static quantization using real data
- Infrastructure updates to support the new AitkPython pass type in validation scripts
Reviewed changes
Copilot reviewed 26 out of 26 changed files in this pull request and generated 20 comments.
Show a summary per file
| File | Description |
|---|---|
| openai-whisper-large-v3-turbo/aitk/winml.py | Windows ML execution provider registration utility |
| openai-whisper-large-v3-turbo/aitk/whisper_encoder_load.py | Encoder model loader with data loading utilities |
| openai-whisper-large-v3-turbo/aitk/whisper_decoder_load.py | Decoder model loader with data loading utilities |
| openai-whisper-large-v3-turbo/aitk/whisper_large_v3_turbo_encoder_*.json | Configuration files for encoder model conversion (FP32 and QDQ) |
| openai-whisper-large-v3-turbo/aitk/whisper_large_v3_turbo_decoder_*.json | Configuration files for decoder model conversion (FP32 and QDQ) |
| openai-whisper-large-v3-turbo/aitk/qnn_workflow.py | Main workflow orchestration script for model generation |
| openai-whisper-large-v3-turbo/aitk/qnn_run.py | Script for running inference and generating quantization data |
| openai-whisper-large-v3-turbo/aitk/qnn_app.py | Core Whisper inference application with ONNX Runtime |
| openai-whisper-large-v3-turbo/aitk/qnn_workflow.json* | Workflow configuration and UI templates |
| openai-whisper-large-v3-turbo/aitk/model_project.config | Model project configuration |
| openai-whisper-large-v3-turbo/aitk/info.yml | Model metadata and recipe information |
| openai-whisper-large-v3-turbo/aitk/inference_sample.ipynb | Placeholder notebook for inference examples |
| openai-whisper-large-v3-turbo/aitk/README.md | Documentation for model optimization workflow |
| openai-whisper-large-v3-turbo/aitk/.gitignore | Git ignore patterns for generated files |
| .aitk/scripts/sanitize/parameters.py | Enhanced parameter validation for dataset handling |
| .aitk/scripts/sanitize/model_parameter.py | Added support for AitkPython pass type validation |
| .aitk/scripts/sanitize/generator_dml.py | Code formatting improvements |
| .aitk/scripts/sanitize/generator_common.py | Added AitkPython pass handling and optimization path logic |
| .aitk/scripts/sanitize/file_validation.py | Updated validation for AitkPython workflows |
| .aitk/scripts/sanitize/constants.py | Added AitkPython constants and removed unused Coming status |
| .aitk/requirements/requirements-WCR-QAI.txt | Added dependencies for Qualcomm AI Hub integration |
| .aitk/configs/model_list.json | Registered new Whisper model and librispeech dataset |
| .aitk/configs/checks.json | Updated check counters for new validations |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
No description provided.