-
Notifications
You must be signed in to change notification settings - Fork 76
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Has the demo gif for text generation been sped up? #12
Comments
Hi @farazk86, I know that this message (and repo) is rather old but I'm testing this demo and struggle to find a way to make it working with GPU delegate. Would you mind sharing what you did ? My understanding is that the model is not adapted to run on GPU but I can't even start the app without crash, so I'm curious to know how you did it. Without that modification below, the app runs perfectly and outputs about 1 word/sec. If anyone else has insights about that, I would be really grateful as well (@Pierrci ? @sayakpaul ?). Sorry if that's a very noob question ! I had some difficulties related to gradle / TF version but now I can build a valid APK supporting GPU with the following modifs : GPT2Client.kt
and of course adding in build.gradle
But when I run the app it crashes on startup with the following error (tflite 2.3)
With tflite 2.4 it's a bit different :
|
Hi,
I cannot achieve the speed demonstrated in the gif: https://github.com/huggingface/tflite-android-transformers/tree/master/gpt2
It takes about 7 seconds to generate a single word on my build. I am even using gpuDelegate to run interpreter on GPU rather than CPU and its still slower.
Has the gif been sped up? am I the only one having this poor performance?
Thanks
The text was updated successfully, but these errors were encountered: