Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add TFLite + Transformer example #179

Merged
merged 3 commits into from
Aug 19, 2024
Merged

Add TFLite + Transformer example #179

merged 3 commits into from
Aug 19, 2024

Conversation

calren
Copy link
Contributor

@calren calren commented Jul 16, 2024

Demonstrates applying an Effect generated by TFLite's Artistic Style Transfer on a video using Transformer API.

style_transfer.mp4

@calren calren changed the title [Not ready for review] Add TFLite + Transformer example Add TFLite + Transformer example Jul 17, 2024
@calren
Copy link
Contributor Author

calren commented Jul 17, 2024

cc @droid-girl @lethargicpanda

@droid-girl
Copy link
Contributor

I checked-out the project. One minor comment: should it be possible to repeat the export with another style? At the moment the selector is available but the export button is disabled because the export completed already.
A proposal:

  1. enable export button again when the first one completed
  2. if you want to keep it disabled, maybe also disable the style selector

float[] identityMatrix = GlUtil.create4x4IdentityMatrix();
glProgram.setFloatsUniform("uTexTransformationMatrix", identityMatrix);
glProgram.setFloatsUniform("uTransformationMatrix", identityMatrix);
// glProgram.setFloatsUniform("uRgbMatrix", identityMatrix);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove commented out code?

bitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
bitmap.copyPixelsFromBuffer(pixelBuffer);

Log.w("DEBUG", "Process frame at " + (presentationTimeUs / 1000) + " ms");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we follow the same convention as in other classes and define a TAG const and use it instead of "Debug"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that makes sense, will do

import javax.microedition.khronos.opengles.GL10;

@UnstableApi
final class StyleTransferShaderProgram extends BaseGlShaderProgram {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we aim to convert it to Kotlin? Do you aim to keep this repo Kotlin only or any guidelines on this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, we should convert this to Kotlin even though we don't have any strict guidelines on keeping this repo kotlin only.

so i do this in a separate PR to keep it easier to review or just do it as part of this PR?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am ok with having a follow up PR with a Kotlin version as long as the repository owner is fine with it.

… Remove unused code, added TAG constant, and added TODO for migrating to Kotlin
@calren
Copy link
Contributor Author

calren commented Jul 22, 2024

addressed comments, please take another look!

should it be possible to repeat the export with another style?
I disabled the radio buttons when the export is started. I tried exporting with another style after an initial export, but the result looked glitchy, and I'm not sure why. It seems like only the first export gives a good result, and subsequent ones always looks glitchy.

@calren
Copy link
Contributor Author

calren commented Jul 30, 2024

trying to make builds pass again on this repo before merging this (#185)

@calren
Copy link
Contributor Author

calren commented Aug 7, 2024

@SigmanZero can you give this an LGTM? It still needs an approval from someone with write access on this repo

@calren
Copy link
Contributor Author

calren commented Aug 7, 2024

@ksemenova @lethargicpanda can you take a quick look at this PR as well?

checking out gradle/libs.versions.toml will probably be the most useful, to make sure we're using recommended dependencies

@calren
Copy link
Contributor Author

calren commented Aug 19, 2024

merging this for now but @ksemenova @lethargicpanda let me know if there are concerns on the TFLite front and I can make follow up PRs to address

@calren calren merged commit f6714a6 into main Aug 19, 2024
2 of 3 checks passed
@calren calren deleted the caren/tf_lite branch August 19, 2024 19:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants