Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Looks like json encoder uses huge amount of memory #22

Open
avi3tal opened this issue Jun 28, 2019 · 0 comments
Open

Looks like json encoder uses huge amount of memory #22

avi3tal opened this issue Jun 28, 2019 · 0 comments

Comments

@avi3tal
Copy link

avi3tal commented Jun 28, 2019

I am working with large size json body and i have encountered that python-simple-rest-client just takes around *4 memory than i would expected.

I have decided to reimplement my http client using requests directly and looks like i have solved the problem.

After debugging i cam to realise that the pure Json library is ineffective in terms of memory and this is what caused my memory to blow.

I read your code and looks like you are using json_encoder which also uses the pure json lib (unless you choose to change it - which in your case you didn't).

My recommendation:

  1. Do not use json_encoder as is but change the json lib to more effective one.
  2. maybe try to avoid even loading the json if this is only for the reason of constructing Response class. Maybe you should pass the original response.json() as is instead of converting to your own body() function.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant