PR review Request: Reduce backend preparation overhead by serializing model only once #8299
IceTDrinker
started this conversation in
Ideas / Feature Requests
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hello,
This is an issue I ran into with large models, the serialization is costly and is done twice currently to prepare backends (once by check_model and once more to actually run the model). I have a PR here : #8270
Comments and feedbacks welcome as I don't seem able to ping the code owner of this file.
Cheers,
IceTDrinker
Edit: I don't know how to ping the relevant code owners for review, sorry about that
Beta Was this translation helpful? Give feedback.
All reactions