[BUG] Performance issue with Yojson

[vouillon]

Parsing a large string (~ 1MiB) with Yojson is really slow. We should investigate why.

[hhugo]

How slow is really slow ? Are you sure the slowness comes from yojson ? Can you describe the issue a bit more ?

[vouillon]

The function Yojson.Safe.from_string is slow on large strings.

The main issue is that Buffer.add_sub_bytes is assuming that Bytes.unsafe_to_string is cheap (https://github.com/ocaml/ocaml/blob/42adb2793657f4ef4f72eb9fc362c73e8eb2eb9f/stdlib/buffer.ml#L162-L163).

Additionally, this function has the side effect of changing the representation of the byte string, which is then converted back from a string to an array in caml_lex_engine.

js_of_ocaml/runtime/lexing.js

Line 56 in 1edf056

var buffer = caml_uint8_array_of_bytes(lexbuf[lex_buffer]);

[hhugo]

I see. I was able to reproduce.

• Yojson.Raw.from_string is not (much less) affected because it processes the whole string at once.
• --disable use-js-string solves the issue
• We should probably fix theses patterns in the ocaml/ocaml codebase (Remove some String to/form Bytes conversion to behave better with jsoo ocaml/ocaml#13543)
• We should revisit the decision to change the underlying representation of bytes when converting to string (when use-js-string is enabled) (Runtime: don't change the shape of bytes when converting to string #1712)

[hhugo]

@vouillon, see ocaml/ocaml#13543

[vouillon]

Cool! Thanks!

[hhugo]

• Remove some String to/form Bytes conversion to behave better with jsoo ocaml/ocaml#13543 has been merged.
• runtime: do not force array shape for bytes in lexing.js #1713 alone fixes most of the issue reported here
• Runtime: don't change the shape of bytes when converting to string #1712 might need more testing to understand the impact