Skip to content

Looks like json encoder uses huge amount of memory #22

Description

@avi3tal

I am working with large size json body and i have encountered that python-simple-rest-client just takes around *4 memory than i would expected.

I have decided to reimplement my http client using requests directly and looks like i have solved the problem.

After debugging i cam to realise that the pure Json library is ineffective in terms of memory and this is what caused my memory to blow.

I read your code and looks like you are using json_encoder which also uses the pure json lib (unless you choose to change it - which in your case you didn't).

My recommendation:

  1. Do not use json_encoder as is but change the json lib to more effective one.
  2. maybe try to avoid even loading the json if this is only for the reason of constructing Response class. Maybe you should pass the original response.json() as is instead of converting to your own body() function.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions