OpenVINO NN Module Functions

[danielenricocahall]

Support for:

• max pooling
• average pooling
• CTC loss
• one hot encoding
• CELU
• log sigmoid

Validated via running the nn_test tests locally - although currently keras/src/ops/nn_test.py is marked as an exclusion for CI. Should the nn_test tests be augmented to skip certain tests based on backend, similar to numpy_test?
Note:

• dot_product_attention and ctc_decode are also probably doable as the operations are available in OpenVINO, but figured that can be done separately.

[gemini-code-assist]

Summary of Changes

Hello @danielenricocahall, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly expands the functional capabilities of the Keras OpenVINO backend by introducing support for several key neural network operations. These additions, including new activation functions, pooling layers, and specialized loss functions, enhance the compatibility of Keras models with OpenVINO, allowing for a broader range of models to be deployed and optimized using the OpenVINO toolkit.

Highlights

• New Activation Function: The CELU (Continuously Differentiable Exponential Linear Unit) activation function has been implemented for the OpenVINO backend, allowing for its use in Keras models targeting OpenVINO.
• Pooling Operations: Support for both max_pool and average_pool operations has been added, enabling these common neural network layers to be executed via the OpenVINO backend.
• One-Hot Encoding: The one_hot encoding function is now supported, which is crucial for various categorical data processing tasks within the OpenVINO backend.
• CTC Loss Implementation: The ctc_loss function, commonly used in sequence-to-sequence models like speech recognition, has been implemented for the OpenVINO backend.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces support for several new functions in the OpenVINO backend, including celu, max_pool, average_pool, one_hot, and ctc_loss. The implementations are mostly correct, but I've identified a few areas for improvement. Specifically, there are opportunities to enhance code readability in celu, fix a copy-paste error in max_pool, improve the implementation of one_hot for correctness and efficiency, and address a critical bug in ctc_loss related to incorrect input processing. My detailed feedback and suggestions are provided in the comments below.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 6.38298% with 44 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.60%. Comparing base (08f102d) to head (357f5e9).
⚠️ Report is 3 commits behind head on master.

Files with missing lines	Patch %	Lines
keras/src/backend/openvino/nn.py	6.38%	44 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #21803      +/-   ##
==========================================
- Coverage   82.63%   82.60%   -0.03%     
==========================================
  Files         577      577              
  Lines       59415    59465      +50     
  Branches     9313     9317       +4     
==========================================
+ Hits        49097    49124      +27     
- Misses       7913     7941      +28     
+ Partials     2405     2400       -5     

Flag	Coverage Δ
keras	82.43% <6.38%> (-0.03%)	⬇️
keras-jax	63.28% <6.38%> (-0.05%)	⬇️
keras-numpy	57.52% <0.00%> (-0.04%)	⬇️
keras-openvino	34.32% <6.38%> (+0.01%)	⬆️
keras-tensorflow	64.08% <6.38%> (-0.04%)	⬇️
keras-torch	63.58% <6.38%> (-0.05%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[danielenricocahall]

/gemini review

[gemini-code-assist]

Code Review

This pull request adds support for several new neural network functions to the OpenVINO backend, including max_pool, average_pool, ctc_loss, one_hot, and celu. The implementations are generally correct and follow the patterns in the existing backend code. I've provided a few suggestions to improve the code. Specifically, I recommend simplifying the celu implementation by using the native OpenVINO operation, refactoring max_pool and average_pool to reduce code duplication, and adding a check in one_hot to handle cases where dtype is None for increased robustness.

[danielenricocahall]

/gemini review

[gemini-code-assist]

Code Review

This pull request adds support for several new neural network functions to the OpenVINO backend, including pooling operations, celu, one_hot, and ctc_loss. The approach of using a shared _pool helper for both max_pool and average_pool is good for code reuse. However, I've identified a few critical issues that could lead to runtime errors in the _pool and one_hot implementations. I've also suggested a simplification for the celu function to use the native OpenVINO operator. Additionally, the PR description mentions support for log_sigmoid, but it is not implemented in the code; it would be great to add this for completeness.

[danielenricocahall]

/gemini review

[gemini-code-assist]

Code Review

This pull request adds support for several neural network functions to the OpenVINO backend, including max_pool, average_pool, ctc_loss, one_hot, celu, and log_sigmoid. The implementations are generally well-structured, and the refactoring to use a shared _pool function for pooling operations is a good approach. However, I've identified a critical issue in the _pool function that will cause a runtime error for max_pool operations. Additionally, the celu function can be simplified by using a native OpenVINO operation. My review includes suggestions to address these points.

[danielenricocahall]

/gemini review

[gemini-code-assist]

Code Review

This pull request adds support for several neural network functions to the OpenVINO backend, including pooling operations, CTC loss, one-hot encoding, and various activation functions like CELU and log-sigmoid. The changes are generally good, with a nice refactoring for pooling operations.

I've identified two main issues:

1. A critical bug in the pooling implementation where an incorrect keyword argument is used, which will lead to a runtime error.
2. The celu activation function is implemented manually, while a more efficient native OpenVINO operation is available. Using the native op would improve performance and simplify the code.

Details and code suggestions are provided in the specific comments.

[danielenricocahall]

/gemini summary

[gemini-code-assist]

Summary of Changes

This pull request significantly enhances the Keras OpenVINO backend by integrating several fundamental neural network operations. These additions, including new activation functions, pooling layers, and specialized loss functions, broaden the compatibility of Keras models with OpenVINO, allowing for a wider array of models to be deployed and optimized using the OpenVINO toolkit. The changes aim to reduce NotImplementedError instances and improve the functional parity of the OpenVINO backend.

Highlights

• New Activation Functions: Implemented support for celu (Continuously Differentiable Exponential Linear Unit) and log_sigmoid activation functions within the OpenVINO backend. The silu function was also updated to use the native OpenVINO swish operator.
• Pooling Operations: Added support for max_pool and average_pool operations, replacing previous NotImplementedError messages. A new internal helper function, _pool, was introduced to consolidate common logic for these pooling operations.
• One-Hot Encoding: The one_hot encoding function is now supported, including handling for sparse=True (which raises a ValueError as sparse tensors are not supported) and proper dtype conversion for on_value and off_value.
• CTC Loss: Implemented the ctc_loss function, enabling its use with the OpenVINO backend. This includes correctly converting all input tensors to OpenVINO format.

Changelog

• keras/src/backend/openvino/nn.py
  • Added import for OPENVINO_DTYPES.
  • Implemented the celu activation function.
  • Updated silu implementation to use ov_opset.swish.
  • Implemented the log_sigmoid activation function.
  • Implemented max_pool functionality.
  • Implemented average_pool functionality.
  • Introduced a new private helper function _pool to abstract common pooling logic.
  • Implemented one_hot encoding, including sparse argument validation and dtype handling.
  • Implemented ctc_loss function, ensuring all inputs are correctly converted to OpenVINO tensors.

Activity

• An initial summary of changes was provided by gemini-code-assist[bot], highlighting new activation functions, pooling, one-hot encoding, and CTC loss.
• A Codecov report was posted by codecov-commenter, indicating low patch coverage (6.38%) for keras/src/backend/openvino/nn.py.
• The author, danielenricocahall, requested /gemini review multiple times throughout the PR's activity.
• Critical feedback from gemini-code-assist[bot] on ctc_loss regarding incorrect log_softmax application and missing OpenVINO tensor conversions was addressed by the author.
• Medium priority suggestions from gemini-code-assist[bot] for celu variable naming were incorporated by the author.
• Medium priority suggestion from gemini-code-assist[bot] to create a _pool helper function for pooling operations was implemented by the author.
• Critical and high priority feedback from gemini-code-assist[bot] on one_hot regarding sparse=True handling, dtype=None default, and input tensor conversion were addressed by the author.
• Critical feedback from gemini-code-assist[bot] on the _pool helper function regarding kernel_shape vs kernel and strides=None handling was partially addressed; strides=None is handled, but the author noted that the current OpenVINO API uses kernel_shape for the pooling functions.
• Repeated suggestions from gemini-code-assist[bot] to use a native ov_opset.celu operator were met with the author's response that opset14 does not contain such a function, leading to the manual implementation being retained.