Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[TF FE][MOC] Fuse Keras LSTM to LSTMSequence and Optimize TF While with TensorList ops #25170

Merged
merged 16 commits into from
Jun 27, 2024
Merged

[TF FE][MOC] Fuse Keras LSTM to LSTMSequence and Optimize TF While with TensorList ops #25170

merged 16 commits into from
Jun 27, 2024

Conversation

rkazants
Copy link
Contributor

@rkazants rkazants commented Jun 22, 2024 •
edited
Loading

Details: Fuse Keras LSTM to LSTMSequence and Optimize TF While with TensorList ops.
Loop operations with TensorListSetItem transformed to ConcatOutput outputs.
Loop operations with TensorListGetItem transformed to SlicedInput inputs.

It helps to fuse six loops with LSTMCell to six LSTM sequence model. It reduces customer model size by twice and increase throughput by 1.97x.

Tickets: TBD

@rkazants rkazants requested a review from a team as a code owner June 22, 2024 11:32
@github-actions github-actions bot added category: TF FE OpenVINO TensorFlow FrontEnd category: TFL FE OpenVINO TensorFlow Lite FrontEnd labels Jun 22, 2024
@rkazants rkazants requested a review from a team as a code owner June 24, 2024 11:24
@github-actions github-actions bot added the category: transformations OpenVINO Runtime library - Transformations label Jun 24, 2024
@rkazants
Fix build
d2c7b1f 
Signed-off-by: Kazantsev, Roman <[email protected]>
@rkazants rkazants requested a review from evkotov June 24, 2024 11:29
@rkazants
Fix build
02605ee 
Signed-off-by: Kazantsev, Roman <[email protected]>
evkotov
evkotov reviewed Jun 25, 2024
View reviewed changes
Copy link
Contributor

@evkotov evkotov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have performance measurements? Did this transformation improve?

src/common/transformations/src/transformations/op_conversions/convert_ti_to_sequences.cpp Show resolved Hide resolved
@rkazants
Fix LSTMCell pattern
c4dbc34 
Signed-off-by: Kazantsev, Roman <[email protected]>
@rkazants
Avoid deprecated LSTMSequence version
f499406 
Signed-off-by: Kazantsev, Roman <[email protected]>
@rkazants
Copy link
Contributor Author

Do you have performance measurements? Did this transformation improve?

Yes, 2x for customer model

@rkazants
Implement unit-tests for transformations
0b2745b 
Signed-off-by: Kazantsev, Roman <[email protected]>
@rkazants
Add callback
866bea7 
Signed-off-by: Kazantsev, Roman <[email protected]>
@rkazants
Copy link
Contributor Author

@popovaan, @itikhono, let me fix your comments in separate PR.

@rkazants rkazants added this to the 2024.3 milestone Jun 27, 2024
@andrei-kochin andrei-kochin added this pull request to the merge queue Jun 27, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jun 27, 2024
@rkazants rkazants added this pull request to the merge queue Jun 27, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jun 27, 2024
@rkazants rkazants added this pull request to the merge queue Jun 27, 2024
Merged via the queue into openvinotoolkit:master with commit 07cbea2 Jun 27, 2024
120 checks passed
@rkazants rkazants deleted the rkazants/lstm_keras_fusion branch June 27, 2024 19:48
@rkazants rkazants mentioned this pull request Jun 27, 2024
Merged
github-merge-queue bot pushed a commit that referenced this pull request Jun 28, 2024
@rkazants
[TF FE][MOC] Fix leftovers for Keras LSTM fusion transformation (#25268)
f9cf396 
**Details:** Fix leftovers for Keras LSTM fusion transformation
#25170

**Tickets:** TBD

Signed-off-by: Kazantsev, Roman <[email protected]>
github-merge-queue bot pushed a commit that referenced this pull request Jun 28, 2024
@rkazants
[TF FE][MOC] Fix leftovers for Keras LSTM fusion transformation (#25268)
8cdbedc 
**Details:** Fix leftovers for Keras LSTM fusion transformation
#25170

**Tickets:** TBD

Signed-off-by: Kazantsev, Roman <[email protected]>
AsyaPronina pushed a commit to AsyaPronina/openvino that referenced this pull request Jul 1, 2024
@rkazants @AsyaPronina
[TF FE][MOC] Fuse Keras LSTM to LSTMSequence and Optimize TF While wi…
5253d2e 
…th TensorList ops (openvinotoolkit#25170)

**Details:** Fuse Keras LSTM to LSTMSequence and Optimize TF While with
TensorList ops.
Loop operations with TensorListSetItem transformed to ConcatOutput
outputs.
Loop operations with TensorListGetItem transformed to SlicedInput
inputs.

It helps to fuse six loops with LSTMCell to six LSTM sequence model. It
reduces customer model size by twice and increase throughput by 1.97x.

**Tickets:** TBD

---------

Signed-off-by: Kazantsev, Roman <[email protected]>
AsyaPronina pushed a commit to AsyaPronina/openvino that referenced this pull request Jul 1, 2024
@rkazants @AsyaPronina
[TF FE][MOC] Fix leftovers for Keras LSTM fusion transformation (open…
b7c22ef 
…vinotoolkit#25268)

**Details:** Fix leftovers for Keras LSTM fusion transformation
openvinotoolkit#25170

**Tickets:** TBD

Signed-off-by: Kazantsev, Roman <[email protected]>
AsyaPronina pushed a commit to AsyaPronina/openvino that referenced this pull request Jul 1, 2024
@rkazants @AsyaPronina
[TF FE][MOC] Fuse Keras LSTM to LSTMSequence and Optimize TF While wi…
ab1a407 
…th TensorList ops (openvinotoolkit#25170)

**Details:** Fuse Keras LSTM to LSTMSequence and Optimize TF While with
TensorList ops.
Loop operations with TensorListSetItem transformed to ConcatOutput
outputs.
Loop operations with TensorListGetItem transformed to SlicedInput
inputs.

It helps to fuse six loops with LSTMCell to six LSTM sequence model. It
reduces customer model size by twice and increase throughput by 1.97x.

**Tickets:** TBD

---------

Signed-off-by: Kazantsev, Roman <[email protected]>
AsyaPronina pushed a commit to AsyaPronina/openvino that referenced this pull request Jul 1, 2024
@rkazants @AsyaPronina
[TF FE][MOC] Fix leftovers for Keras LSTM fusion transformation (open…
d7d9770 
…vinotoolkit#25268)

**Details:** Fix leftovers for Keras LSTM fusion transformation
openvinotoolkit#25170

**Tickets:** TBD

Signed-off-by: Kazantsev, Roman <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@popovaan popovaan popovaan left review comments

@itikhono itikhono itikhono approved these changes

@andrei-kochin andrei-kochin andrei-kochin approved these changes

@evkotov evkotov evkotov approved these changes

@mvafin mvafin Awaiting requested review from mvafin

Assignees

@AKochin AKochin

@andrei-kochin andrei-kochin

Labels
category: TF FE OpenVINO TensorFlow FrontEnd category: TFL FE OpenVINO TensorFlow Lite FrontEnd category: transformations OpenVINO Runtime library - Transformations
Projects
None yet
Milestone
2024.3
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@rkazants @itikhono @andrei-kochin @popovaan @evkotov @AKochin