Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fork for JSON content > 2 GB #21

Open
develancer opened this issue Sep 11, 2020 · 1 comment
Open

Fork for JSON content > 2 GB #21

develancer opened this issue Sep 11, 2020 · 1 comment

Comments

@develancer
Copy link

Hi guys,

Thank you for this amazing library. It does work fast and surprisingly stable :-)

However, I've had issues with reading JSON files larger than 2 GB. The reason is simple—one cannot create a String object of such size. I therefore forked this project and adapted it to reading directly from files: https://github.com/develancer/LazyJSON64

There are some downsides, at the moment—for example, the structures are read-only and there is no „dirty” mode.

Do you think that integrating these changes into mainstream some day would be useful? We can leave it as a separate fork for now.

@kasperjj
Copy link
Contributor

Absolutely! Super interesting fork... I had started down the same path a while ago to allow for parsing json without bringing it all into memory and to allow lazy decoding of unicode characters.... ie. we don't actually need to spend time decoding the whole data chunk, only string keys and values when needed. Let me dig into my old branch next week and see if we can make a shared path forward that accomplishes both!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants