Separate ML model from NS image #66
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
It seems like the speech recognition DSP task which is collecting the input data from ROM/VSI audio source needs to be a bit slower as in case of using software to do the ML inference the performance is way slower than using the Ethos-NPU which means the DSP task must be delayed for sometime as to allow some extra time for the software inference to be finished. Signed-off-by: Ahmed Ismail <[email protected]>
The idea is to move the ML model to the DDR memory region, generate a separate image for the DDR memory region where the model can be over the air updated without the need to update the NS image. To do so the following changes are needed: * Modify the linker scripts for all platforms to remove the RAM load address of the ML model image as it should be loaded to DDR memory region. * Some linker sections were renamed to be more descriptive. * A new CMake function is introduced to extract sections from input AXF file, this function is used to extract DDR section which contains the ML model out of the NS image to be loaded separately. Signed-off-by: Ahmed Ismail <[email protected]>
Signed-off-by: Ahmed Ismail <[email protected]>
Signed-off-by: Devaraj Ranganna <[email protected]>
aggarg
approved these changes
Apr 16, 2024
amazonKamath
approved these changes
Apr 16, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
The idea is to move the ML model to the DDR memory region, generate a separate image for the DDR memory region. To do so the following changes are needed:
loaded to DDR memory region.
which contains the ML model out of the NS image to be loaded separately.
Test Steps
Validated changes in the CI, no regression observed.
Checklist:
Related Issue
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.