You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Measure RAM size and performance for a comparison. Caching will result in some faster responses, but slows down every request and will increase ram usage.
Problem: Uvicorn will spawn different processes in parallel, which will have difficulties keeping a shared cache. Redis and other external solutions might be nice, but have huge overhead. Might be able to cache requests in middleware before passing them to backend
People that are trying this service out are likely to queue the same result multiple times.
Caching these values will really save some execution time.
This is somehow relevant to #24
The text was updated successfully, but these errors were encountered: